Ethical AI in Software Systems: A Framework for Bias Detection and Mitigation in Machine Learning Pipelines

Abstract

The paper contains a comprehensive roadmap of how to locate and fix algorithmic bias in machine learning pipeline in three domains of use which are credit scoring, a hiring system, and a recommendation system. The proposed framework will involve a combination of fairness indicators, bias detection strategies, and mitigation strategies at any stage of an ML lifecycle, including data collection, trainer, and implementation. Examples provided by experimental results show that the framework makes the levels of bias to be reduced to an average of 12 percent as compared to an average of 41 percent with an overall reduction of approximately 70 percent. These senses of the Disparate Impact Ratio rose, to averages, of 0.58, to 0.91 which represent far fairer outcomes. This is much better, but it still reduced the accuracy of models by only 3-5 percent making performance within reachable parameters. The follow up was also constant and reduced the bias to 12 percent after repeated assessments of the evaluations. It shows that ethical AI models can deal with the amount of fairness and performance without compromising the functionality of a system. It is also partially a contribution to responsible software engineering in the sense that it can be used to provide a scalable and practical way of bias reduction in the real-life machine learning systems