“Industry 4.0” has become the future direction of manufacturing industry. To prepare for this upgrade, it is important to study the automation of semiconductor failure analysis. In this paper, the sample polishing activity was studied for upgrading to a smart polishing process. Two major issues were identified in implementing the smart polishing process: the optimization of current polishing recipes and the capability of making decisions based on live feedback. With the help of Solver add-in, the current polishing recipes were optimized. To make decisions based on live images captured during polishing, strategies were explored based on finger polishing process study. Our investigation showed that a grey scale line profile analysis on images can be used to build the vision capability of our smart polishing system, on which a decision- making capability can be developed.