Helping a leading UK energy provider adopt customer-facing AI agents safely
Challenge
A leading UK energy provider wanted to introduce customer-facing AI agents but needed to make sure they were safe, reliable and compliant before going live.
Approach
Resillion built a risk-based AI assurance approach, covering test strategy, expected behaviour, realistic test scenarios and repeatable testing for chat and voice agents.
Results
The organisation now has a repeatable way to test AI agents, providing evidence for go-live decisions, stronger governance and a foundation for wider AI adoption.
Building confidence in customer-facing AI
The client is a major UK energy provider serving millions of homes and businesses through essential customer services.
As part of its digital transformation, the organisation wanted to introduce conversational AI agents to support real customer journeys, not back-office experiments. These agents would help customers complete everyday but sensitive tasks such as identity checks, billing, payments, meter management, Direct Debit setup, tariff changes and support during vulnerable situations.
The opportunity was faster, more consistent service at scale. For customers, the benefit was easier access to support when they needed it. But in a regulated energy environment, that could only happen if the agents could be trusted to behave safely, fairly and consistently.
Resillion helped the client build an AI testing and assurance capability from the ground up, giving business, technology, compliance and security teams a common way to assess AI.
The challenge: Testing AI in a regulated environment
AI agents cannot be tested in exactly the same way as traditional software. Their responses can vary, even when they receive the same question, and they can use information from earlier parts of a conversation to decide how to respond.
For a regulated energy provider, this creates new risks. An AI agent could give incorrect information, mishandle personal data, fail to follow a required process, ignore business rules, respond badly to a vulnerable customer or be manipulated through prompt injection.
These failures could lead to customer harm, reputational damage, compliance issues or disruption to services.
The energy provider therefore needed more than a one-off test. It needed a repeatable way to define acceptable AI behaviour, test agents against it and provide clear evidence for go-live and governance decisions.
AI assurance gives organisations the evidence they need to adopt AI responsibly. It turns questions about safety and compliance into structured testing and informed decisions.
Our approach: Building a repeatable way to test AI
Resillion led the AI testing and assurance work from end to end, helping the client put in place the strategy, testing approach and governance needed to use customer-facing AI safely.
Resillion created an AI testing strategy that gave business, technology and compliance teams a common understanding of what needed to be tested and what good behaviour looked like.
Rather than checking whether an AI agent gives one exact answer, the approach assesses whether it behaves appropriately across realistic customer conversations, business rules and different risk scenarios.
Resillion helped create and manage test data, including “golden datasets” containing approved examples of how an AI agent should respond in different situations.
These provide an agreed benchmark that can be used to test AI agents consistently and identify where their behaviour falls short of expectations.
Resillion developed a risk-based approach covering areas such as customer journeys, business rules, guardrails, hallucinations, prompt injection, data protection, security, compliance and vulnerable customers.
This gives the client a consistent way to assess AI agents before they go live, rather than relying on ad-hoc testing or manual review.
Resillion also demonstrated how AI testing could be automated and run repeatedly.
Running the same scenarios multiple times helps identify inconsistent behaviour that a single test might miss, while creating a more scalable approach as the organisation introduces more AI services.
The testing approach was connected to the client’s technology, compliance, security and governance processes. Resillion provided assurance evidence and reporting to help stakeholders make informed decisions about AI readiness and deployment.
This means AI assurance can continue as the organisation develops and introduces new AI services, rather than being treated as a one-off exercise.
Resillion’s work gave the client confidence that AI agents could be introduced into sensitive customer interactions without increasing risk. By testing the agents across realistic journeys, business rules and higher-risk scenarios, the client could check whether AI would give accurate guidance, protect personal data, follow required steps and respond appropriately to vulnerable customers.
This matters because customer-facing AI is only valuable if people can rely on it. The assurance approach helped the organisation move closer to faster, more scalable customer service while protecting customer trust, regulatory confidence and brand reputation.
Results: Greater confidence in customer-facing AI
By putting structured AI assurance in place, the client moved from having no repeatable way to test customer-facing AI agents to having a working capability that supports safer deployment and future growth.
Key outcomes include:
- Greater confidence to put AI agents in front of customers safely
- Stronger protection for customers in sensitive journeys such as billing, payments, tariff changes and vulnerable-customer support
- Clear evidence for go-live decisions, governance forums and regulatory scrutiny
- A repeatable foundation for scaling AI beyond the first wave of chat and voice agents
- More consistent service quality as AI adoption expands across customer channels
- Better collaboration between business, technology, compliance, security and testing teams
The Total Quality perspective
For organisations using AI in regulated customer environments, quality is about more than whether the technology works. It is about making sure AI behaves safely, consistently and appropriately for customers and the business.
Resillion brings together quality engineering, AI assurance, security, governance and industry expertise to help organisations adopt AI with confidence.
By creating a repeatable way to test customer-facing AI agents, Resillion helped the client turn questions about AI safety and compliance into structured testing, clear evidence and informed decisions.
As AI becomes part of more critical customer journeys, this approach can help organisations innovate while keeping customers, the business and the wider organisation protected.
For more information take a look at our AI and Machine learning capabilities.
