This chapter deals with the problem of non-linear and non-Gaussian states and parameters estimation using Bayesian methods. The performances of various conventional and state-of-the-art state estimation techniques are compared when they are utilized to achieve this objective. These techniques include the Extended Kalman Filter (EKF), Unscented Kalman Filter (UKF), and Particle Filter (PF). In the current work, the authors consider two systems (biological model and power system) to perform evaluation of estimation algorithms. The results of the comparative studies show that the UKF provides a higher accuracy than the EKF due to the limited ability of EKF to accurately estimate the mean and covariance matrix of the estimated states through lineralization of the nonlinear process model. The results also show that the PF provides a significant improvement over the UKF because, unlike UKF, PF is not restricted by linear-Gaussian assumptions which greatly extends the range of problems that can be tackled.