Linear elastic fracture mechanics principles are widely applied for the analysis of crack problems in rock fracture mechanics. Rock indentation is an important and complicated problem among rock engineering issues. In this paper, in addition to the fracture criterion of maximum tangential stress adjacent to crack tip, the higher order displacement discontinuity method (which is a version of the indirect boundary element method) has been used for modeling the crack propagation mechanism under blunt indenters. In order to achieve more accurate results, higher order boundary elements i.e. quadratic elements, has been used to calculate displacement discontinuities and also to reduce the singularities of stress and displacement fields near the crack tip, the special crack tip elements has been used to calculate the stress intensity factors (SIF) at the crack tips. In this modeling, the effect of crack angle on stress intensity factors has been investigated. The numerical results of stress intensity factors obtained from some example problems were compared to the theoretical and experimental results cited in the literature which always show a percentage error less than one percent. The simulated results may pave the way for increasing the efficiency of mining and drilling by improving the design of tools and indentation equipments.