In continuing my trend for AI ethics from the last post, Spinoza: Building an AI Ethics Framework From First Principles, I found some early research about AI ethics audit.
As proposed by the paper, Ethics‑Based Auditing to Develop Trustworthy AI, the idea itself is a bit
vague where the idea of AI ethics auditing is defined as “a governance mechanism that can be used by organizations that design and deploy AI systems to control or influence the behavior of AI systems.”
So, the proposal is to audit the behavior of the organization as a structured process by which an entity’s behavior is assessed for consistency with relevant principles or norms.
However, defining the norms is the challenge.
In a nutshell, the audits focus on the rationale behind the decision, code audits entail reviewing the source code, and impact audits investigate the effects of an algorithm’s outputs. According to the paper, by promoting procedural regularity and strengthening institutional trust, ethics-based auditing can help by providing a consistent set of criteria for the following outcomes for AI and ethics:
The process of ethics-based auditing should be continuous, holistic, dialectic, strategic and design-driven. Hence, the audits need to be continuously monitored and the output must be evaluated. The holistic impact of AI, including alternatives to AI decision-making, should be taken into account. The ethics framework should ensure that the right questions are asked and the process should ensure that the default actions are the right ones from an ethical standpoint. Finally, trustworthy AI is about design. Hence, interpretability and robustness should be built into systems from the start. Ethics-based auditing support this aim by providing active feedback to the continuous (re-)design process.
The paper Ethics-based auditing of automated decision-making systems: interve... gives additional considerations for automated decision-making systems (ADMS)
On the face of it, all this sounds like a good idea – but could it end up enriching large consulting companies without providing real value?
GDPR comes to mind i.e. the intentions are good – but there are really very few benefits that we, as customers, see for such regulation.
Nevertheless, an AI ethics audit may well be on the way in the future and if it does appear we are getting a glimpse of what it could look like
Image source pixabay