Binary search trees ------------------- Another important use of binary trees is to store values that we may want to look up later. For instance, a binary search tree could be used to store a dictionary of words. A binary search tree satisfies the following property at every node v: all values in the subtree rooted at v that are smaller than the value stored at v lie in the left subtree of v and all values in the subtree rooted at v that are larger than the value stored at v lie in the right subtree of v. To emphasize that the values in the tree can be ordered, we write a elaborate slightly on the Haskell definition of binary trees to describe search trees. data (Ord a) => STree a = Nil | Node (STree a) a (STree a) Observe that the structure of an STree is identical to that of a normal Tree, but there is a type class dependence, similar to the ones we have seen for polymorphic functions such as mergesort and quicksort. Here are two examples of search trees over the values [1,2,3,4,5,6]. 4 3 / \ / \ 2 5 2 5 / \ \ / / \ 1 3 6 1 4 6 Both trees look reasonably well "balanced". This is not always the case. For instance, here is a highly unbalanced search tree over the same set of values. 6 / 5 / 4 / 3 / 2 / 1 To find a value in a binary search tree, we start at the root. At each node, if we have not already found the value we are looking for, we can use the search tree property to decide whether to search in the right subtree or the left subtree. We keep walking down the tree in this fashion till we find the value we seek or we reach a leaf node from where we cannot descend further. Thus, each lookup in a binary search tree traverses, in the worst case, a single path from the root to a leaf node. How much time does it take to look up a value in a search tree with n nodes? Let us say that a tree is balanced if at each node the size of the left subtree differs from the size of the right subtree by at most 1. Initially, we search for the value in the entire tree, with n nodes. If we do not find the value at the root, we search either the left or the right subtree. Since the tree is balanced, the number of nodes in each of these subtrees is at most n/2. In this way, we successively search trees of size n, n/2, n/4, ... till we reach a leaf node, a subtree of size 1. The length of this sequence is clearly bounded by log n. Another way of stating this is that the height of a balanced search tree with n nodes is log n. Here is a Haskell definition of the search procedure we just described: findtree :: (Stree a) -> a -> Bool findtree Nil x = False findtree (Node tleft y tright) x | x == y = True | x < y = findtree tleft x | otherwise = findtree tright x Observe that a search tree does not contain duplicate values. Exercise: What does inorder traversal of a search tree yield? Search trees are not static objects. In general, we have to insert new values into search trees and remove stale values from search trees. There are two aspects to keep in mind while defining these operations: a) We have to ensure that the updated tree remains a search tree b) We have to preserve the balanced structure of the tree For the moment we concentrate on the first aspect. We will tackle the problem of maintaining the balanced structure later. Where should we insert a value into a search tree? From the definition of a search tree, there is only one possibility. Search for the value in the tree. If it already exists, there is nothing to be done. Otherwise, we reach a leaf node. This is the same path that we would have to follow to find the new value after it has been inserted. So, insert the new value as a left or right child of the leaf node where the unsuccessful search terminates. inserttree :: (Stree a) -> a -> (Stree a) inserttree Nil x = Node Nil x Nil inserttree (Node tleft y tright) x | x == y = Node tleft y tright | x < y = Node (inserttree tleft x) y tright | otherwise = Node tleft y (inserttree tright x) Clearly, the maximum number of steps required to insert a value into a search tree is equal to the length of the longest path in the tree. Thus, if the search tree is balanced and has n nodes, inserttree takes time log n, but will not in general preserve the balanced structure of the tree. How do we delete a value from a tree? First, we have to agree on what happens if the value to be deleted does not occur in the tree. One approach is to declare this an error. It is easier, how ever, to interpret "delete x from t" as "delete x from t if the value exists in t", so if we try to delete x and it is not found in t, the tree t is unchanged. Suppose we want to delete a value x from a tree whose root is y. If x < y, we inductively delete x from the left subtree of y. Similarly, if x > y, we inductively delete x from the right subtree of y. So, the interesting case is when x==y. y == x / \ w z / \ / \ t1 t2 t3 t4 If we remove y, we have a "hole" at the root of this tree. It is tempting to move either w (or z) into this place and recursively delete w from the left subtree (or z from the right subtree). However, this would not preserve the structure of the tree --- for instance, if we move w up to the root, values in the tree t2, which are bigger than w, will end up to the left of w. The correct solution is to move the largest value from the left subtree of y (or the smallest value from the right subtree of y) in place of y. The largest value in a search tree can be found easily, by following the rightmost path in the tree. Removing this value from a tree is also a relatively easy operation. Here is a function that removes the maximum value from a nonempty tree, returning both the value and the modified tree, after deletion. deletemax :: (STree a) -> (a,STree a) deletemax (Node t1 y Nil) = (y,t1) deletemax (Node t1 y t2) = (z, Node t1 y tz) where (z,tz) = deletemax t2 We can now rewrite deletetree as follows: deletetree :: (Stree a) -> a -> (Stree a) deletetree Nil x = Nil deletetree (Node tleft y tright) x | x < y = Node (deletetree tleft x) y tright | x > y = Node tleft y (deletetree tright x) -- In all cases below, we must have x == y deletetree (Node Nil y tright) x = tright deletetree (Node tleft y tright) x = Node tz z tright where (z,tz) = deletemax tleft Exercise: Define the function deletemin and change the definition of deletetree in the "interesting case" to move the smallest value from the right subtree in place of the node being deleted. Balanced binary trees --------------------- Having described how to preserve the inductive structure of a search tree after insert and delete operations, it remains to be seen how we can maintain the balanced nature of the tree. A balanced tree is one in which, at each node, the sizes of the left and right subtrees differ by at most one. This definition of balance is difficult to maintain without incurring a heavy rebalancing cost. A less stringent definition is to ask that the heights of the left and right subtrees differ by at most one. Such trees are called height-balanced trees or AVL trees, after Adelson-Velskii and Landis, the first people to propose the use of such balanced trees. It is easy to see that a size-balanced tree is always height-balanced. The converse is not true in general. For instance, the following tree is height-balanced but not size-balanced. The left subtree of the root has size 3 while the right subtree has size 1. 4 / \ 2 5 / \ 1 3 To use height-balanced trees in place of size-balanced trees, we have to ensure that the height of a height-balanced tree is logarithmic in the number of nodes in a tree. To establish this, we turn the problem around and consider the following: for a fixed height h, what is the size S(h) of the smallest height-balanced tree with height h? Starting with h=1, we can compute S(h) and draw the corresponding tree T(h), ignoring the actual values: h S(h) T(h) 1 1 * 2 2 * / * 3 4 * / \ * * / * ... ... ... k S(k-1)+S(k-2)+1 * / \ T(k-1) T(k-2) Thus, the "worst" tree of height k is inductively constructing by fusing together the worst trees of height k-1 and k-2. The recurrence for S(h), given by S(1) = 1 S(2) = 2 S(k) = S(k-1) + S(k-2) + 1, k > 2 is very similar to that for the Fibonacci numbers F(1) = 1 F(2) = 1 F(k) = F(k-1) + F(k-2), k > 2 In the case of the Fibonacci numbers, we can show that F(k) is exponentially large with respect to k. Analogously, we can show that S(k) grows exponentially with respect to k. This means that even the "most skewed" height-balanced tree of height k has a size that is exponential in k. In other words, the "most skewed" height-balanced tree with n nodes has height logarithmic in n. Let us assume that we have written a function rebalance that reestablished the height-balanced property after an insert or delete. Here is how we would modify our functions inserttree and deletetree. inserttree :: (Stree a) -> a -> (Stree a) inserttree Nil x = Node Nil x Nil inserttree (Node tleft y tright) x | x == y = Node tleft y tright | x < y = rebalance (Node (inserttree tleft x) y tright) | otherwise = rebalance (Node tleft y (inserttree tright x)) deletetree :: (Stree a) -> a -> (Stree a) deletetree Nil x = Nil deletetree (Node tleft y tright) x | x < y = rebalance (Node (deletetree tleft x) y tright) | x > y = rebalance (tleft y (deletetree tright x)) -- In all cases below, we must have x == y deletetree (Node Nil y tright) x = tright deletetree (Node tleft y tright) x = rebalance (Node tz z tright) where (z,tz) = deletemax tleft Let us define the slope of a node to be the difference in heights between the left and right subtrees of the node. In a height-balanced tree, the slope of each node is in the range {-1,0,+1}. Inserting or deleting a node can at most change the slope by 1. In the modified functions inserttree and deletetree, we apply rebalancing bottom up, starting from the lowest node where we potentially disrupt the balance. Thus, when we need to rebalance a node, we can assume that its slope is either 2 or -2 and its subtrees are inductively height-balanced. Suppose a node has slope 2 (the case -2 will be symmetric) with both its subtrees height-balanced. Then, the height of the left subtree is two more than that of the right subtree. We shall consider two cases. a) The slope at the root of the left subtree is 0 or 1. This situation corresponds to the following picture, where t[h] denotes that t has height h x / \ y t3[h] / \ t1[h+1] t2[h or h+1] We can rotate this tree to the right as follows y / \ t1[h+1] x / \ t2[h or h+1] t3[h] It is easy to observe that the slope at y is 0 or -1 and the slope at x is either 1 or 0, so the rotated tree is height-balanced (recall that t1, t2 and t3 were inductively assumed to be height-balanced). We have to verify that the resulting tree remains a search tree. To this, it is sufficient to verify that the inorder traversals of the two trees match. It is easy to verify that the inorder traversal for both trees is given by (inorder t1) ++ [y] ++ (inorder t2) ++ [x] ++ (inorder t3) b) The slope at the root of the left subtree is -1. This situation corresponds to the following picture, where t[h] denotes that t has height h x / \ y t3[h] / \ t1[h] t2[h+1] where t2 looks like z / \ t21[h or h-1] t22[h or h-1] with at least one of t21 or t22 being of height h. We can now rotate the subtree rooted at y to the left, as follows. x / \ z t3[h] / \ y t22[h or h-1] / \ t1[h] t21[h or h-1] First, we verify that in both cases, the inorder traversal of the left subtree of x is given by (inorder t1) ++ [y] ++ (inorder t21) ++ [z] ++ (inorder t22) so this left rotation preserves the search tree property. Notice that at this stage, the slope at z may be 2 (because it is possible that t1[h], t21[h] and t22[h-1]). However, we now apply a right rotation at x to get _____________z____________ / \ y x / \ / \ t1[h] t21[h or h-1] t22[h or h-1] t3[h] At this point, clearly the slope of x is -1 or 0, the slope of y is 1 or 0 and the slope of z is 0. Further, the inorder traversal of this tree yields (inorder t1)++[y]++(inorder t21)++[z]++ (inorder t22)++[x]++(inorder t3) which is the same as that of the original tree, before the two rotations. We can now write the function rebalance: rebalance :: (Stree a) -> (Stree a) rebalance (Node t1 y t2) | abs (sy) < 2 = Node t1 y t2 | sy == 2 && st1 /= -1 = rotateright (Node t1 y t2) | sy == 2 && st1 == -1 = rotateright (Node (rotateleft t1) y t2) | sy == -2 && st2 /= 1 = rotateleft (Node t1 y t2) | sy == -2 && st2 == 1 = rotateleft (Node t1 y (rotateright t2)) where sy = slope (Node t1 y t2) st1 = slope t1 st2 = slope t2 rotateright (Node (Node t1 y t2) x t3) = Node t1 y (Node t2 x t3) rotateleft (Node t1 x (Node t2 y t3)) = Node (Node t1 y t2) x t3 How do we compute the slope? A naive implementation would be as follows slope :: (Stree a) -> Int slope Nil = 0 slope (Node t1 x t2) = (height t1) - (height t2) height :: (Stree a) -> Int height Nil = 0 height (Node t1 x t2) = 1 + (max (height t1) (height t2)) Clearly, computing height requires examining each node in the tree, so computing the slope of a tree is proportional to n, rather than log n. The solution is to augment the information stored at each node to inductively maintain the height of a node. data (Ord a) => STree a = Nil | Node Int (STree a) a (STree a) where a node of the form (Node m t1 x t2) records that the value at this node is x and the subtree rooted at this node has height m. With this additional information, we have height :: (Stree a) -> Int height Nil = 0 height (Node m t1 x t2) = m Thus, computing the slope at any node takes constant time. It remains to modify all the functions we have written so far---insert, delete, rebalance---to correctly compute the height value at a node after each update. This is left as an exercise.