The non-linear behavior of infilled frames is very complex. The behavior of this structure may be studied by experimental and numerical approaches. An experimental test can provide a more realistic output but has the disadvantages of high costs, relatively long time and specific room usage. A numerical analysis can be an alternative to analyze the behavior of infilled frames. One of the most powerful numerical approaches is meso-modeling. This approach has the advantage of being able to capture local damage to the panel. For this reason, the progressive damage identified in the meso-model can be used as a basis for determining damage state criteria. The grouping of damage states is proposed based on the initial identification in the form of local damage linked to global damage, i.e., IDR. This study’s proposed level of infilled frame damage is DS1 = 0.17%, DS2 = 0.52%, DS3 = 0.79% and DS4 = 1.99%. However, the quantification results of the structural damage level cannot be generalized because many complex factors influence the behavior of infilled frames. Subsequently, a parametric study was carried out to determine the contribution of the mechanical properties of the infilled frame material to the degree of structural damage.