Tower cranes are very complex mechanical systems and have been the subject of research investigations for several decades. Research on tower cranes has focused on the development of dynamical models (linear and nonlinear) as well as control techniques to reduce the swaying of the payload. Inherently, the dynamical model of the tower crane is highly nonlinear and classified as under-actuated. The crane system has potentially six degrees of freedom but only three actuators. Also, the actuators are far from the payload which makes the system non-colocated. The dynamic model describing the motion of the payload from point to point is affected by uncertainties, time delays and external disturbances which may lead to inaccurate positioning, reduce safety and efficacy of the overall system. It is proposed here to use an H based adaptive fuzzy control technique to control the swaying motion of a tower crane. This technique will overcome modeling inaccuracies, such as drag and friction losses, effect of time delayed disturbances, as well as parameter uncertainties. The proposed control law for payload positioning is based on indirect adaptive fuzzy control. A fuzzy model is used to approximate the dynamics of the tower crane; then, an indirect adaptive fuzzy scheme is developed for overriding the nonlinearities and time delays. The advantage of employing an adaptive fuzzy system is the use of linear analytical results instead of estimating nonlinear system functions with an online update law. The adaptive fuzzy scheme fuses a Variable Structure (VS) scheme to resolve the system uncertainties, and the external disturbances such that H tracking performance is achieved. A control law is derived based on a Lyapunov criterion and the Riccati-inequality to compensate for the effect of the external disturbances on tracking error so that all states of the system are uniformly ultimately bounded (UUB). Therefore, the effect can be reduced to any prescribed level to achieve H tracking performance. Simulations are presented here to illustrate the performance of the proposed control design.