Last week, the UK witnessed chaos over exam results (GCSE and A-levels).
The fiasco also provided a textbook case on how to build a biased algorithm.
Sadly, this will be the shape of things to come.
These decisions were overturned because of the backlash – but many more decisions made by biased algorithms will go unchallenged globally.
Here are some insights you can learn when people intentionally set out to design a biased algorithm.
One thing is for sure, this is not the last time we have heard of biased algorithms
Designing a biased algorithm – what we can learn from the UK education results algorithm
- Biased data vs biased intent: There are two primary ways to introduce bias in an algorithm. Either through the data or through the intent. The data often reflects the status quo and reflects the current bias in society (for better or for worse). The data also reflects gaps. For example, if there are less people from (say) Iceland in the UK – the data will not reflect that population – leading to a bias. More seriously, the algorithm could be designed to be intentionally reflect a view in its logic. The later was at play here
- Transparency was not the issue: The algorithm was ‘transparently biased’. The workings of the algorithm were revealed in great detail in a 319-page report. So, whatever bias was not hidden.
- Intent is the issue: The algorithm had two primary goals – to reduce grade inflation and to maintain continuity. For this, it primarily gave greater emphasis to historical data. These design intents were the root of the problem and they were not ‘algorithmic’ – but rather human-created.
- Ignore the experts – The UK has no shortage of technical expertise or business expertise. The house of lords has been holding a consultation on AI for the last couple of years. Reports were produced based on the collective wisdom. So, the risks were known
- Not for the first time: There was another issue of an algorithm biased by intent in the visa application process towards some nationalities.
- Conflicting criteria – A father called the algorithm ‘punishment by statistics' with “virtually no chance of providing grades to the students in a way that satisfies the double criteria of being fair to the individuals and controlling grade inflation nationally”.
- No individual results: Because the results were standardized, the results were not reflective of individual effort – thereby ruining careers
- Ignore the outliers: The standardization process ignored the outliers. Worse – because the process relied on historical performance – it penalized the hard-working student from a low achieving school neighborhood aka the outliers. Based on prior performance of the school, about 40 percent of the results were downgraded
- Ignore the humans in the loop: In this case, the teachers
- Choose colourful words to obfuscate the issue: At various times, our prime minister has called the algorithm ‘robust’ or ‘mutant’. The real issue is actually quite easy to understand as listed above and its neither robust nor mutant (whatever meaning you ascribe to these words)
So, did we increase confidence in any entity?
The NGOs did a great job and engendered trust – especially upReach, a social mobility charity and foxglove which challenges digital injustices
The only thing we can be sure of is: we will see more of this situation in future
More support would be needed for such non-profit and unbiased initiatives.
Note that the views expressed in this article are my own.
Image source BBC