Decisions made by various artificial intelligence (AI) systems greatly influence our day-to-day lives. With the increasing use of AI systems, it becomes crucial to know that they are fair, identify the underlying biases in their decision-making, and create a standardized framework to ascertain their fairness. Biases in AI systems lead to unintended ethical, social and even legal issues. In this paper, we propose a novel and versatile Fairness Score and Bias Index for measuring the fairness of a supervised learning AI system. We also propose a standard operating procedure (SOP) for issuing Fairness Certification for such data-driven applications. Fairness Score and audit process standardization will ensure quality, reduce ambiguity, enable comparison and improve the trustworthiness of the AI systems. It will also provide a framework to operationalize the concept of fairness and facilitate the commercial deployment of such systems. Furthermore, a Fairness Certificate issued by a designated third-party auditing agency following the standardized process would boost the conviction of the organizations in the AI systems that they intend to deploy. The Bias Index proposed in this paper reveals comparative bias amongst the various protected attributes within the dataset, and the Fairness Score provides overall fairness with an ideal value of 1. To substantiate the proposed framework, we iteratively train a model on biased and unbiased data using multiple labelled datasets and check that the Fairness Score, Bias Indexes and the proposed process correctly identify the biases and judge the fairness.

Problem statement: Standardisation of AI fairness rules and benchmarks is challenging because AI fairness and other ethical requirements depend on multiple factors, such as context, usage case, type of the AI system, and so on. In this paper, we elaborate that the AI system is prone to biases at every stage of its lifecycle, from inception to its use, and that all stages require due attention for mitigating AI bias. We need a standardised approach to handle AI fairness at every stage. Gap analysis: While AI fairness is a hot research topic, a holistic strategy for AI fairness is generally missing. The AI fairness assessment is often developer-centric. Standard processes are required for independent audits for the benefit of other stakeholders. Also, most researchers focus only on a few facets of AI model-building. Peer review shows a focus on biases in the datasets, fairness metrics, and algorithmic bias. In the process, other aspects affecting AI fairness get ignored. The solution proposed: We propose a comprehensive approach in the form of a novel seven-layer model, inspired by the Open System Interconnection (OSI) model, to standardise AI fairness handling. Despite the differences in the various aspects, most AI systems have similar model-building stages. The proposed model splits the AI system lifecycle into seven abstraction layers, each corresponding to a well-defined AI model-building or usage stage. We also provide checklists for each layer and deliberate on potential sources of bias in each layer and their mitigation methodologies. This work will facilitate layer-wise standardisation of AI fairness rules and benchmarking parameters.