If AI becomes better than humans at finding weaknesses, could it also become better at testing its own reliability? And what would that mean for our responsibilities in GMP?
A few days ago, I came across an interesting interview with Sam Altman, CEO of OpenAI, at Stanford Graduate School of Business. No, he was not talking about GMP, or even validation. But it gave me something to think about. During the discussion, Altman raised concerns about cybersecurity and how increasingly capable AI models are discovering vulnerabilities that traditional approaches may struggle to address.
At one point, he said: “I no longer think we’re going to be able to just patch all the bugs.” His argument was that AI models are becoming capable of identifying increasingly complex vulnerabilities, potentially including weaknesses that fall outside our traditional understanding of cybersecurity. And if AI continues improving at this pace, relying on people to identify and fix every vulnerability may no longer be realistic.
This made me think about something slightly different. If AI becomes better than humans at discovering software vulnerabilities, could it not also become better at identifying weaknesses in our validation strategies? And if AI can identify a problem, suggest a correction and test whether that correction works, could we eventually use AI to validate other AI systems, or even itself? Of course, finding a software vulnerability is not the same as demonstrating that a system is suitable for GMP use. But perhaps there is something worth exploring here. What would it actually mean for AI to validate itself? I think we need to distinguish between#;
- using AI to support validation and
- allowing AI to perform the entire validation process.
Imagine we are validating an AI-enabled system used to generate Product Quality Reviews. We define its intended use, establish acceptance criteria and prepare test cases to demonstrate that it retrieves the correct data, performs accurate calculations and generates reliable reports. Now imagine using another AI system to review our validation strategy. It could identify missing test scenarios, suggest unusual combinations of input data or discover inconsistencies in calculations that we had not considered. It mcould even generate test cases, execute them and identify unexpected failures..
Could one AI system independently validate another?
Let’s imagine a future where we have two AI systems. The first is a static AI model, used to analyse manufacturing data and identify potential quality trends. The second is a more advanced autonomous AI validation agent, capable of planning and executing tests, using different testing tools and adapting its approach when it identifies unexpected results. This second system could challenge the first model using thousands of manufacturing scenarios, including unusual combinations of process parameters, missing data, incorrect inputs and situations never encountered during the original training. It could compare the results against independently established reference data, identify failures, suggest corrections and repeat the testing until the required performance criteria are met. Potentially, the entire testing process could take place with very limited human intervention. And why stop there? We could imagine another independent AI agent challenging the validation strategy itself, looking for weaknesses in the tests and identifying scenarios the first validation agent overlooked.
This is hypothetical, of course, but it illustrates how AI could eventually go beyond simply supporting traditional validation activities. The problem is that independence would still need to be demonstrated. If the systems share similar training data, assumptions or weaknesses, they might consistently fail to detect the same errors. Two AI systems agreeing does not necessarily mean they are both correct. Could AI make validation better than we can?
There is also another side to this discussion. We should not assume that human-generated validation tests are automatically better. People make mistakes, overlook scenarios and sometimes design tests around what they already expect a system to do. Imagine an AI identifying a combination of manufacturing conditions when used on other products, they caused an outcome that resulted in a oong term OOS, something that the validation team had never considered has not perhaps able to recognise patterns in different product that share similar steps on manufacturing. This to say that if we agree that AI can explore far more scenarios than a human team, perhaps refusing to use that capability could itself mean missing opportunities to prevent harm. However, generating one million tests does not necessarily provide better assurance than a smaller number of well-designed tests.The vnalue is not simply in how much AI can test, but in whether it can discover what we would otherwise miss. Are we reducing human work or increasing human responsibility?
This is probably the part I find most interesting.
We often discuss AI as a technology that will allow us to do more with fewer resources. Faster investigations, faster document reviews, faster validation and faster decisions. Imagine AI allowing a validation team to perform in one day what previously required several weeks. But someone still needs to understand the results, assess the failures and decide whether the remaining risks are acceptable. And as the volume and complexity of the evidence increase, these decisions may become more frequent and difficult rather than easier. In previous posts, I questioned whether the people responsible for reviewing AI outputs would always have the knowledge, time and authority to challenge them. We introduce AI to reduce workload and improve efficiency. Yet the more capable AI becomes, the more activities we can perform, the more evidence we can generate and the more decisions we may need to justify. Are these decision cause of people stress?
We may be able to do more, faster and potentially better than before. But if we remain accountable for the results, is AI actually reducing our responsibilities, or is it giving us even more responsibility than we had before?
References
- Sam Altman — View From The Top, Stanford Graduate School of Business
- Draft Annex 22 — Artificial Intelligence, EudraLex Volume 4
- EMA Multistakeholder Workshop Report — October 2026
Leave a Reply