HVAC systems of large commercial buildings consume energy in excess of the sum total of the building loads. This excess energy use is due to the fact that a single air-handler unit in a HVAC system, having to provide conditioned air at different supply temperatures to different zones in the building, can do so only by resorting to either (a) a certain amount of mixing of cold and hot air streams as in dual-duct systems or (b) to terminal reheating in single-duct systems. This mixing of cold and hot air streams or terminal reheating result in an energy penalty which can be minimized by say, converting a constant air volume (CAV) system to a variable air volume (VAV) system, but it cannot be entirely eliminated. This paper proposes an index, called the Energy Delivery Efficiency (EDE), which characterizes this penalty and rates the energy performance of HVAC systems on an absolute scale. We develop the mathematical basis of the EDE approach for both one-zone and two-zone buildings, that allows determining the variation of the ideal EDE with outdoor temperature for a specific building. Year-long measured whole-building cooling and heating energy use data from two retrofitted buildings are finally used to illustrate differences between actual EDE plots of CAV and VAV systems, how they compare with the ideal EDE of a two-zone building, and how the approach can provide diagnostic insights into HVAC system efficiency in specific buildings.