PurposeThe purpose of this paper is to identify how the design of a performance appraisal system (PAS) affects the perceived justice of academic employees (AE) about their performance appraisal (PA) and how this is associated with organizational effectiveness in terms of organizational leadership (OL).Design/methodology/approachThe study subjects are two economic faculties of two Estonian public universities. The data for the study were collected using the PA Survey with a total of 82 AEs, OL Capability Questionnaire with a total of 72 AEs and the organizations' documents to analyze PAS. Assessment and analysis of the data included: the measurement of PAS design; the measurement of perceived justice from PA; the measurement of organizational leadership capability; analysis of the results gained from studying perceived justice from different PAS designs and organizational effectiveness in terms of OL.FindingsUltimately, the study reveals that PAS design affects academic employees' perception of distributive justice and organizational external effectiveness in terms of OL but does not affect academic employees' perception of procedural justice and organizational internal effectiveness in terms of OL.Research limitations/implicationsThis study suggests that organizational effectiveness depends on perceived justice of employees from the design of PAS. However, the results of this study are valid in the arrangements of academic jobs in universities and in similar or close context of Estonian culture.Originality/valueThis paper demonstrates the role of PAS design in conditions of intellectual job arrangement in universities with its influence on organizational effectiveness in the context of OL.