The failure mechanism of YXR3, a kind of matrix-type high speed steel (HSS), which is widely used in cold forging tooling, was studied in this paper. Through metallographic analysis of YXR3 in different conditions, such as before and after heat treatment and mechanical tests, the metallurgical structures and changes of the steel were analyzed. The failure analysis was conducted to specimens of uniaxial tension, low cycle fatigue, and impact tests by observing the macroscopic and microscopic fractographs and fracture morphology features of YXR3 steel in different loading conditions. The morphology and distribution of all phases in YXR3 steel were clearly found in the SEM micrographs by EBSD (Electron Back-Scattering Diffraction), and the major cracking modes and crack origins were investigated by observing the microcracks, which would be beneficial for finding out the fracture mechanism of YXR3 steel from microscopic perspective. The results of this study could help the tool designer to improve the tool design and manufacture which can avoid the abnormal appearance of tool fracture during cold forging.