CESER Releases New Testbed to Advance LLM and Agentic AI Evaluation for Critical Infrastructure

The U.S. Department of Energy’s (DOE) Office of Cybersecurity, Energy Security, and Emergency Response (CESER) and Lawrence Livermore National Laboratory (LLNL) have developed Stormbreaker, a dynamic testbed for rapidly evaluating large language models (LLMs) and agentic artificial intelligence (AI) in power systems and operational technology (OT) environments. Stormbreaker builds on CESER’s Mjölnir AI testbed at LLNL, extending the AI testbed environment to support dynamic testing of LLMs and evaluating their performance in critical infrastructure settings.

AI has the potential to play a pivotal role in securing America’s energy sector. Critical infrastructure operators face new challenges as AI evolves and industries adopt LLMs and agentic AI. These challenges include understanding what LLMs and agentic AI systems can do, where they are reliable, and how adversaries could leverage them in power systems and OT environments.

Testing LLMs and agentic AI is complex and depends on a variety of factors, including the underlying model, system instructions, user prompts, skill guidance, tool availability, and target environment. Traditional benchmarking of LLMs applies different LLMs to a static suite of challenges, which is useful for apples-to-apples comparisons, but it only changes one of several variables. As a result, it can fail to measure how an AI system will perform once deployed in a changing operational setting.

Stormbreaker is designed for evaluating LLMs and agentic AI as dynamically as the environments in which these systems operate. Rather than relying on a single static threshold, Stormbreaker enables users to run tests across many configurations and environments, helping them build a deeper understanding of model behavior, robustness, and limitations.

Stormbreaker highlights two core evaluation approaches:

• Configuration transfer allows users to systematically vary one or more components, such as instructions, prompts, skills, tools, or environment, and measure the impact in a controlled experiment.
• Limit discovery pushes a scenario progressively harder until the model fails, revealing the boundaries of its capability. 

These approaches help users understand not just whether a model succeeds, but how and why it succeeds, and where it begins to break down. Critical infrastructure operators can use Stormbreaker to determine how well AI can support defensive tasks or be used in offensive or adversarial contexts. 

Stormbreaker is intended to support a wide range of stakeholders interested in safe and effective AI deployment, including utilities, grid operators, energy technology providers, National Laboratories, and other organizations. Organizations interested in engagement, demonstrations, or collaboration opportunities can email AI-FORTS@doe.gov.