Machine learning is a subset of AI that enables systems to learn from data. I’m particularly interested in how bias enters the machine learning process. It starts at the beginning, at problem formation stage, and can creep in at any stage through to, and post, model or product deployment.
Bias can leak into machine learning processes because models learn patterns that are encoded in the training data. If the dataset is biased (ie. not representative of the target population) the machine learns this bias and perpetuates it in its decisions or representations. For example, in Gen AI outputs:
Datasets with mostly American English dialects can make speech recognition systems such as chatbots battle with diverse accents
Datasets with mostly images of white male doctors can perpetuate stereotypes that competent doctors are mostly male and White.
Datasets with mostly Western cultural artefacts can skew product, gift or style recommendations
This is referred to as algorithmic bias. But even if datasets are representative, people have cognitive biases that they bring into the labelling of data and during the feature engineering stage of machine learning. This requires a view of what type of bias could exist at what stage of the process, and how to detect whether it is occurring. Thankfully there are software programs and statistical measures that can tell you a lot about your data – whether its balanced or imbalanced or whether the impact on end-users will be fair or unequal. Plus, there are ways to remedy it at each stage, but preferably don’t wait until the deployment stage as this could be a time-consuming and costly exercise.
Because training models is a time intensive and expensive endeavour, most companies use existing LLM models and then use retrieval-augmented generation (RAG). If the model itself is biased, the bias will naturally pass onto the RAG process, however the RAG process itself can be biased. Fortunately, there are detection and mitigation actions that can be taken during the RAG process as well.
I’ve developed an 8-point integrative framework which provides a quick view of the types of bias found in the ML and RAG processes. Depending on where you are in the model development stage, there are methods to test for bias and strategies to mitigate it. With regulations becoming tighter in terms of responsible AI which includes algorithmic fairness, companies should ensure that potential bias is assessed at all stages of the ML and RAG pipeline.
Fig. 1: 8-point bias framework
Fig 2. Table provides description of the relevant biases and the fairness measures used to detect it.
