Friction stir processing (FSP) is considered to be a promising sustainable technique for grain refinement of metallic alloys. The heat generated during FSP promotes dynamic recrystallization in processed material which is essential for grain sub-division process. However, excessive heat generation can lead to high temperatures of >300C that may cause abnormal grain growth in the processed material. On the other hand, repetitive high temperature heating cycles can reduce the lifetime of the FSP tool. Therefore, it is essential to manage the process heat not only to achieve homogeneity and finer grain sizes in the processed material but also to reduce tool wear. In this work, friction stir processing of AZ31B Mg with an internally cooled FSP tool is simulated by a three-dimensional CFD model. We have studied the effect of rapid tool cooling on temperature and flow stress distribution in processed material. Additionally, the grain size and hardness of the processed material is estimated by using Zener-Holloman and Hall-Petch based relationships. It was found that FSP with internally cooled tool is a promising approach that effectively controls temperature levels during processing. Therefore it enables the achievement of better mechanical properties by effective grain refinement and has a positive effect on tool life.