Define a helper function that returns a tuple with four elements: isBST (boolean), size (int), min (int), and max (int).
Perform a post-order traversal: first visit the left child, then the right child, and finally the current node.
At each node, check if the left child is a BST and if the right child is a BST.
If both children are BSTs and the current node's value is greater than the maximum value in the left subtree and less than the minimum value in the right subtree, then the current subtree is a BST.
If the current subtree is a BST, calculate its size and update the maximum BST size if necessary.
If the current subtree is not a BST, return a range that ensures the parent will not be a BST.
After the traversal, return the maximum BST size found.