The effect of roughness on the frictional energy dissipation in fretting contact of nominally flat rough surfaces is studied. The contact is modeled as the statistical sum of asperity tip junctions. A mathematical analysis with a probability distribution of asperity heights in the form of a delta sequence is conducted to analytically show that a rougher surface dissipates more energy than a smoother surface. Numerical simulations with three typical measured surface roughness profiles are presented, validating the analytical finding that rougher surfaces dissipate more energy than smoother surfaces in fretting contact. The proposed statistical approach is compared with so called direct calculation methods, which analytically model discrete asperity contacts, and the differences regarding the energy dissipation in fretting are discussed.