The Kirchhoff-law-Johnson-noise (KLJN) scheme is a statistical/physical secure key exchange system based on the laws of classical statistical physics to provide unconditional security. This dissertation contains three main studies of the KLJN system. The first study presents the refutation of a physical model, proposed by Gunn, Allison and Abbott (GAA), to utilize electromagnetic waves for eavesdropping on the KLJN secure key distribution. The correct mathematical model of the GAA scheme is deduced, which is based on impedances at the quasi-static limit. Mathematical analysis and simulation results confirm our approach and prove that GAA's experimental interpretation is incorrect too. The second study analyzes one of the passive (listening) attacks against the KLJN system, the cable capacitance attack. In practical situations, due to the non-idealities of the building elements, there is a small information leak, which can be mitigated by privacy amplification or other techniques so that unconditional (information-theoretic) security is preserved. The industrial cable and circuit simulator LTSPICE is used to validate the information leak due to one of the non-idealities in KLJN, the parasitic (cable) capacitance. Simulation results show that privacy amplification and/or capacitor killer (capacitance compensation) arrangements can effectively eliminate the leak. The third study explores one of the major active (invasive) attacks, the current injection attack. The LTSPICE is used to emulate the attack against the ideal and a practical KLJN system, respectively. It is shown that two security enhancement techniques, namely, the instantaneous voltage/current comparison method, and a simple privacy amplification scheme, independently and effectively eliminate the information leak and successfully preserve the system's unconditional security.