Carbon nanofibers/tubes (CNF/Ts) are very strong and stiff and as a result, are expected to be capable of enhancing the mechanical properties of cementitious materials significantly. Yet there are practical issues concerning the utilization of CNF/Ts in cementitious materials. This study summarizes some of the past efforts made by different investigators for utilizing carbon nanofilaments in cementitious materials and also reports recent experimental research performed by the authors on the mechanical properties of CNF-reinforced hardened cement paste. The major difficulties concerning the utilization of CNF/Ts in cementitious materials are introduced and discussed. Most of these difficulties are related to the poor dispersibility of CNF/Ts. However, the findings from the research presented in this work indicate that, despite these difficulties, carbon nanofilaments can significantly improve the mechanical properties of cementitious materials. The results show that CNFs, even when poorly dispersed within the cementitious matrix, can remarkably increase the flexural strength and cracking resistance of concrete subjected to drying conditions.