Multimodal large language models—AI systems that process text, numbers, and images together—show promise in automating power grid diagnosis by analyzing topology diagrams, measurement data, and incident reports simultaneously. However, a correct answer alone does not guarantee the model used the right reasoning or evidence to reach that conclusion. This gap between accuracy and trustworthiness poses a significant risk in grid operations, where engineers need to understand *why* an AI system recommends a particular action.
Researchers have now proposed a comprehensive auditing framework to verify that AI models for grid diagnosis actually rely on task-appropriate evidence. The approach registers what evidence should matter for each diagnostic task, then tests the model's behavior by removing different inputs one at a time—topology data, measurements, or text reports—to see which ones the model truly depends on. The framework compares three layers of reliance: what the model claims to use, how it actually behaves when evidence is removed, and what domain engineers say should matter.
When the framework detects discrepancies—cases where the model's actual reasoning doesn't match engineering requirements—it applies a correction mechanism. The system regenerates failed responses while constraining the model to use only the registered evidence, then independently re-audits to confirm improvement without sacrificing accuracy.
Testing on standard IEEE grid scenarios with 39 and 118 buses, researchers validated the framework across three different model sizes. Results demonstrate the approach can reliably identify when models fail to use appropriate evidence, diagnose the specific failure mode, and fix the problem. This work addresses a critical need: as AI moves from research into real grid operations, utilities must ensure their AI advisors are not just accurate but also explainable and grounded in sound engineering reasoning.
