UK AI Safety Institute releases 'Inspect' toolset to evaluate AI models
It will be available under an open source MIT licence
UK's AI Safety Institute on Friday released a toolset designed to evaluate AI models.
Dubbed Inspect, the open source tool is the first state-backed AI safety testing platform to be made publicly available.
"As part of the constant drumbeat of UK leadership on AI safety, I have cleared the AI Safety Institute's testing platform - called Inspect - to be open sourced," said secretary of state for Science, Innovation and Technology, Michelle Donelan.
"This puts UK ingenuity at the heart of the global effort to make AI safe, and cements our position as the world leader in this space. The reason I am so passionate about this, and why I have open sourced Inspect, is because of the extraordinary rewards we can reap if we grip the risks of AI."
AI safety evaluations lack a consistent and accessible approach. Inspect aims to fill this gap by providing a software library for testers of all backgrounds – from startups and researchers to developers and government bodies.
It allows for a standardised way to assess the capabilities of individual AI models across various aspects, including core knowledge, reasoning abilities and autonomous functionalities.
The toolset functions through three core components:
- datasets provide sample test scenarios
- solvers execute the tests
- scorers analyse the results and generate metrics
The platform is extensible, with a permissive MIT licence. Third-party developers can create and integrate additional testing methods using Python, allowing Inspect to adapt and evolve alongside the ever-changing AI landscape.
AI Safety Institute chair Ian Hogarth said: "Successful collaboration on AI safety testing means having a shared, accessible approach to evaluations, and we hope Inspect can be a building block for AI Safety Institutes, research organisations and academia."
The move by the UK aligns with the growing international focus on AI safety.
In May last year, leaders of the G7 economies issued a joint statement advocating for the adoption of standards to create trustworthy AI and the establishment of a ministerial forum known as the Hiroshima AI process. These initiatives aim to ensure responsible and secure development and deployment of AI technologies worldwide.
Last month, the UK and the US announced a joint effort to develop advanced AI model testing methods. This builds upon commitments made at the UK's AI Safety Summit last November, where discussions centred on mitigating risks associated with AI development.
The US National Institute of Standards and Technology (NIST) recently launched NIST GenAI, a programme focused on evaluating the safety and security of generative AI models. NIST GenAI aims to develop methods for detecting fake or misleading AI-generated content.
The UK AI Safety Institute, along with Incubator for AI (i.AI) and Number 10, are also spearheading an initiative to bring together leading AI talent from diverse fields.
According to the government, this group will work collaboratively to further develop and refine open source AI safety tools.
Amanda Brock, CEO of OpenUK, commented: "The positioning for the UK in the AI safety space is key to the current strategy and the success of the Safety Institute's new platform can now only be measured by the number of the companies who have already committed to their AI platforms being tested who actually start to go through this process.
"With the UK's slow position on regulating - which I agree with - this platform simply has to be successful for the UK to have a place in the future of AI. All eyes will now be on South Korea and the next safety summit to see how this is received by the world."