Embedding human judgement into regulation, before, during and after validation.

And this is where I think the ethical discussion becomes particularly relevant to Quality.

At first sight, spending time discussing philosophy, human judgement, responsibility or the ethical implications of AI could appear slightly unproductive. A business focused mind could reasonably ask: what exactly does this add? Does this discussion improve productivity? Does it reduce costs? Does it help us manufacture medicines faster?

But, in a strange way, could the same criticism not be made about Quality?

Let’s look at Quality from a very superficial and purely commercial perspective. We ask people to document what they are doing. We check their work. Sometimes we check it twice. We investigate deviations. We challenge assumptions. We ask for evidence. We stop processes when something does not look right. We introduce controls that can make manufacturing slower and, at least in the short term, more expensive.

Anyone looking only at immediate productivity could describe many of these activities as a nuisance. But we know why they are there. The endpoint is not simply producing something faster.

The endpoint is producing a medicine that is safe, effective and consistently manufactured; understanding our processes better; identifying risks before they become problems; and ultimately protecting patients and public health. Once we look at Quality in this way, its value becomes difficult to question.

So I wonder whether we should apply some of the same thinking to artificial intelligence. Perhaps ethical reflection should not be something that happens only after an AI system has already been designed, validated and implemented. Perhaps questions about human judgement, accountability, bias, responsibility and the limits of automation should be considered alongside validation and throughout the lifecycle of the system.

Not because we want to slow AI down. But because, as decades of pharmaceutical Quality have shown us, sometimes slowing down enough to ask the right questions is what allows us to move forward with greater confidence.

And perhaps the most important question is not simply how much AI can do for us.

It is deciding which decisions we are comfortable allowing it to make, and which ones we are not. Because one day this will probably become normal. In the same way that today we trust a calculator to perform a calculation, we may increasingly trust AI systems to perform parts of our analysis and decision-making. But can a risk assessment alone determine that level of confidence?

And what happens when we move into areas traditionally based on professional judgement? Could this become something similar to “QP discretion”, but applied to a system whose reasoning the QP may not be able to fully understand? Are we moving towards a point where a QP, or a Quality professional more generally, also needs to become partly an IT or AI expert simply to understand the tools supporting their decisions?

And if we accept that we may never fully understand every step of an AI system’s reasoning, the question becomes even more important:

Which decisions are we prepared to make with systems whose reasoning we may not be able to fully understand, and which decisions should remain firmly within human judgement?