A recent thrust in turbulence closure modeling research is to incorporate machine learning (ML) elements, such as neural networks, for the purpose of enhancing the predictive capability to cover a broader class of flows. For generalizability to unseen flows, we submit that the data-driven ML approaches must preserve certain fundamental physical principles and closure tenets incumbent in physics-based (PB) models. We propose and investigate three elements to ensure the physical underpinnings of ML turbulence closures: (i) characteristic physical features and constraints that all (PB and ML) closure models must strive to satisfy; (ii) ML training scheme that infuses and preserves selected PB constraints; and (iii) physics-guided formulation of ML loss (objective) function to optimize models predictions. Current ML training and implementation strategies that can potentially cause significant physical incompatibilities and internal inconsistencies are identified. Means of mitigating inconsistencies and improving compatibility between different physical elements of the modeled system are developed. First, key closure constraints dictated by the model system dynamics are derived. Then a closed loop training procedure for enforcing the constraints in a self-consistent manner is proposed. Finally, the simple test case of turbulent channel flow is used to highlight the deficiencies in current ML methods and demonstrate improvements stemming from the proposed mitigation measures. In summary, this work addresses the need for physics-dictated guidance in the development of ML-enhanced turbulence closure models.