Abrasion plays an important role in several types of applications, such as: manufacturing processes involving material removal; attainment of engineered surfaces applied to different tribosystems; selection of surface finishing to promote an adequate contact between surfaces; wear and failure of components; among others. Moreover, the abrasive mechanisms in service will, in part, depend on materials and their properties, including their microstructures, which are often responsible for mechanical and damage behaviors. In this aspect, a study in micro-scale can contribute to improve the design of materials and allow the prediction of their response in different applications. The aim of the present work is to investigate the most relevant parameters and conditions involving the abrasive mechanisms in steels. Therefore, a methodology was developed to quantify the tribological and mechanical results in the micro-scale. To reach this goal, FEM numerical models considering different material microstructures were developed, which were validated through experimental tests. The tribological behavior was evaluated using micro-scratch tests under constant normal load, using a cono-spherical tip with 10 μm diameter, similar to an abrasive particle, which slide over five groups of ferrous materials: (i) homogeneous soft material; (ii) heterogeneous soft material with soft second phase particles: (iii) heterogeneous soft material with hard second phase particles; (iv) homogeneous hard material; and (v) heterogeneous hard material with hard second phase particles. The range of normal load studied was selected based on the dominant abrasive micro-mechanisms, from micro-ploughing to micro-cutting. Damage model with element deletion was also applied to evaluate the material removal due to abrasion and the analysis was based on factors such as worn volume and specific energy during the scratch. The results indicate that: (a) hard second phase particles promote a local decrease of depth of penetration and volume removed and, as a consequence, an increase in the specific energy; (b) soft second phase particles tend to follow the deformation behavior of the matrix, and they can provide some oscillations of the specific energy depending on the applied normal load; and (c) computational approach is validated with experimental findings, indicating that main differences can be assigned to adhesion and crystallographic orientation of the grains, which have not been considered in the modelling. The numerical results allowed drawing a quantitative map of the abrasion resistance of the different steels evaluated as a function of the ratio between hardness of deformed materials and attack angle. The microstructure effects on the abrasion are established and can be incorporated into analyses to improve the design of materials.