The Limits of Applying the Hippocratic Oath to Data Science
---------------------
Robbie Fordyce
01 Feb 2023
---------------------
Can we have a hippocratic oath for data science? What might this mean for data scientists? What would this mean for you and me?
---------------------
- Overview
- Limits
- Do No Harm
Mannell, K., Fordyce, R. and Jethani, S. (2023) Oaths and the ethics of automated data: limits to porting the Hippocratic oath from medicine to data science. Cultural Studies. 37 (1): 168-189.
DOI: https://doi.org/10.1080/09502386.2022.2042577
01. Overview
This paper was a genuine pleasure to write with two of my dearest colleagues. I even went so far as to render the historical context in meme format (see below).
The paper itself responds to an idea that’s circulated in regulatory circles that suggest the solution to data ethics issues is a kind of Hippocratic Oath. The core concept is to adopt the Hippocratic Oath from medicine for the purposes of trying to make sure that data science, and ‘big data’ companies are more ethical. The concept would seem to be “if we ensure that the people doing the science don’t do anything bad, then we’re killing the problem at its source”. These sorts of discussions often focus on the mantra of “Do No Harm” as the core principle. You can see a range of news outlets covering this idea, including Wired, Forbes (albeit written by an independent consultancy), the Guardian, among others. This is also supported closer to the regulatory space, including from the EU’s European Data Protection Supervisor Chief Giovanni Buttarelli (sadly deceased a few months after his comments) as well as EU Commissioner Věra Jourová.
02. Limits
The thing is, we don’t think the Hippocratic Oath will work for data science, for quite a few reasons.
The crux of our position is that the ethics in medicine’s Hippocratic Oath are deontological rather than utilitarian in nature. Deontological ethics is generally associated with Kant and focuses on duty and virtue above all else, where one acts in the best manner they believe should be applied in all cases. Utilitarian ethics is the process of trying to work out what decision to make through a sort of ethical calculus, and is generally associated with Jeremy Bentham and John Mill. They each have wild variations and major changes and developments since Kant/Mill/Bentham. Apologies to any moral philosophers in advance, but our shorthand on these is that we see deontological ethics as being effective in n=1 contexts, while utilitarian ethics is effective in n=many contexts.
This might seem like a somewhat arcane and academic observation, but it effectively means that the physician is beholden to the patient as a duty of care. They are working out what is best for the patient at that time and in that moment. Despite some critiques of the way that medical consent is managed, the process operates with a principle of informed consent and participation by the patient, who is given a sense of what the costs and benefits are for accepting or refusing care. In comparison, a utilitarian ethics would involve dealing with 200 patients and working out which ones to save and which to let die (or, less dramatically, which ones to help first). Triage is governed by other ethics and other economics of care.
03. Do No HarmSo, consider the phrase that is often focused upon for adopting the Hippocratic Oath: “Do No Harm”. Despite the fact that this phrase does not appear in any of the modern Hippocratic Oaths we could find (either the Lasagna version of 1964, or later modifications), this is not feasible under a utilitarian ethics. In any situation other than a perfectly equitable division of resources to a perfectly identical set of people, harm will be done. Consider a tool for assigning welfare payments to a group. Some will win, some will lose. Some will receive more, others might be punished less, some will receive a bill. Any kind of data science is reliant on sorting information within a group, of assigning resources or punishments, or targeting some individuals and not others.
At its most literal, the Hippocratic Oath would simply not work even in a situation where all other principles of data ethics could be followed. Even then, the Hippocratic Oath is not the sum total of ethical management in medicine. It’s almost at the level of performance relative to the way that regulation actually manages , and it’s backed up by a major commitment from the practitioner – literally years of highly exclusive professional education and practice with multiple layers of oversight and exclusivity. There is also a massive national and institutional regulation, as well as insurance systems and the moral conviction of the medical community.
There’s so much more to say on this, including a history of the Hippocratic Oath, and we do so in the article, linked below. We have some free e-prints that I can provide if anyone would like a free copy.