Six places your connected product assurance is quietly breaking
In 1968, 2001: A Space Odyssey gave us HAL 9000, a machine capable of communicating, interpreting information, and making decisions on behalf of its human crew. More than five decades later, our everyday devices may not have reached the same capabilities, but they face a similar challenge: technology can work as designed and still fail the person using it.
A connected device can have the right sensors, software, integrations, and connectivity and still fail the person using it. Sometimes the failure is technical. More often, it happens somewhere between the product, the ecosystem around it, and the moment when a user simply expects it to work.
In October 2025, an AWS outage took down multiple cloud-dependent products. One of the major victims were Eight Sleep, a smart sleep technology company known for its smart mattress system. Its users reported overheating issues and beds becoming stuck in odd positions.
The devices themselves were not necessarily defective. A failure in cloud infrastructure and DNS resolution disrupted services that those products depended on to deliver their intended experience.
The problem wasn’t necessarily inside any one component. It sat in the dependency between them.
The evidence can be valid. The product experience can still fail.
The product has escaped the device.
The AWS outage illustrates a broader problem with connected products: the product the customer experiences is larger than the device that gets shipped.
It is the combination of hardware, firmware, mobile software, identity, cloud services, APIs, networks, third-party platforms, suppliers and support commitments that together deliver the customer’s experience.
Those elements change at different speeds. Hardware might remain unchanged for years. Firmware may change quarterly. The app may change monthly. Cloud services can change continuously. A phone operating system can alter permissions or background behaviour without the device manufacturer changing anything at all.
None of this complexity is visible to the customer. They see one product, one promise and one experience. Your users don’t experience these components separately. They expect the product to connect, authenticate, control, measure, alert, update and recover when required.
That changes what it means to assure a connected product. The unit of assurance is no longer just the ‘passed’ device. It is the outcome the connected system is expected to deliver.
That is why departmental sign-off is necessary, but not sufficient.
Yale’s smart lock earned Matter certification in two weeks. But passing certification did not settle every question about how it would work in customers’ homes. Yale also asked Resillion to test its Wi-Fi behaviour in real-life scenarios. This one example where the standard was met but confidence in the customer experience needed more evidence.
Six fault lines where product confidence breaks.
The most difficult failures are often not defects inside a single component. They appear at the seams:
The promise: A product promise isn’t always translated into a testable end-to-end requirement, so teams can validate individual functions without proving the intended customer outcome.
The component: Hardware, firmware, the app and cloud services can each work as intended but fail in a particular sequence, state or recovery path.
The ecosystem: A certified implementation can encounter a router, phone, hub or third-party condition that wasn’t represented in the evidence.
The update: A firmware, software or operating-system update can change an assumption on which earlier evidence depended.
The external dependency: A supplier, API, cloud service or platform can change something outside the manufacturer’s direct control, affecting the customer experience even when the device itself hasn’t changed.
The ownership: Every function can defend its own evidence while nobody owns whether the complete product outcome is sufficiently assured.
“Passing” answers one question at a point in time. It doesn’t automatically show that the product remains sufficiently assured after its dependencies, configuration or environment change.
A certification result, interoperability test, security assessment or Quality Engineering report is valuable. But every piece of evidence answers a defined question under defined conditions, relating to the product and version, configuration, environment, requirements and applicable standard or use case.
Change is where those boundaries matter. The answer isn’t to retest everything after every change. Nor is it to assume previous evidence remains valid. The better question is impact:
- What changed?
- Which evidence does that change touch?
- Which assumptions or dependencies have moved?
- What decision needs to be renewed?
Assurance has to follow the product through its lifecycle, not stop when an individual test or certification milestone is complete.
- At architecture and design: Is the intended outcome clear, with its dependencies and owners?
- At integration and pre-certification: Does the configuration under test represent what is intended to ship?
- At release: Is the evidence current for the exact build and supported environment?
- After a material change: What previous evidence or assumptions could the change invalidate, and what needs to be refreshed?
The whole point is to keep the consistent connection between the product outcome, the evidence supporting it and the decisions being made about it.
Continuous assurance does not mean continuous testing of everything.
You can stay ahead of every curveball when you have a current view of what changed, what that change touches, and what decision needs to be renewed.
That does not mean you need to create another department or subject every change to a test cycle. It means making the assurance impact of change visible and giving accountability to a senior leader act on it.
At Resillion, we call this ‘Release Confidence Owner’. The title may vary by organisation. It might be held by a Product Director, VP of Engineering, or another authorised leader. What matters is the mandate.
The Release Confidence Owner needs to:
- Own outcome: What must reliably work for the customer or business, including degraded and recovery behaviour?
- Own dependencies: Which hardware, firmware, app, cloud, identity, network, supplier and ecosystem components create that outcome?
- Own change: What has moved since confidence was last established?
- Own evidence: What does each specialist function know, and where are the gaps or assumptions?
- Own decision: Is that evidence sufficient for this product, configuration and moment in the lifecycle? If not, what is the smallest credible assurance activity needed, and who owns the decision?
The Release Confidence Owner does not replace those disciplines or own their technical evidence. They own the decision that the evidence, taken together, is sufficient for the product outcome.
A conformance result remains a conformance result. A cyber security assessment remains a cyber security assessment. Quality Engineering remains distinct.
The leadership responsibility is not to collapse those disciplines into a single score. It is to understand how their evidence combines, where the gaps or assumptions are, and whether it is sufficient for the decision at hand.
The conclusion
Passing the product is just a milestone. Owning the outcome is the job.
And the question after “Did it pass?” should be:
What are we relying on that we have not actually demonstrated for this product, in this configuration, after this change?
Connected products are never truly finished. Their dependencies change, their environments change and their customers find new ways to use them.
Their assurance cannot be a one-time event either.
If you can’t yet answer that question for your next release, start with the evidence you already hold. Our Conformance and Interoperability team can help you map what your certification covers, what it doesn’t, and what decision the gap affects.
➔ Talk to Resillion’s Conformance and Interoperability team about where your evidence stops.
