March 9, 2026 · Updated September 5, 2026
When an automation stops, the team needs answers: what completed, what changed, and what should happen next? A browser that disappears into an error leaves a colleague reconstructing the work. A clear interruption can be handled as part of the agency's process.
Carrier workflows can change through navigation, required questions, authentication, or underwriting rules. The right evaluation is how the system handles those changes, not whether it is labeled a script or an AI agent.
Different methods have different failure modes
Automation may use structured integrations, browser selectors, visual interpretation, language models, or a combination. A well-designed scripted step can be dependable. An AI-assisted step can adapt to some variations and still misunderstand a field or act on an incorrect assumption.
Microsoft's Power Automate documentation describes UI elements and selectors, showing why scripted automation should not be reduced to fixed mouse coordinates.
The implementation needs to verify the result of an action. For example, after entering an address, did the site accept it? After requesting a quote, did the system receive terms, a referral, or an error? Continuing to click is not proof of completion.
Ask what happens when an answer is missing
A newly required question may ask for a fact that is absent from the client file. The correct behavior may be to stop and ask the team. Automatically choosing an answer can make the workflow appear successful while creating an inaccurate application.
Use a test case with an intentionally missing answer. Ask the provider to show the question, source information available, and the place where the reviewer supplies or confirms the answer. The system should not hide uncertainty to keep the animation moving.
Inspect the last completed step
A useful interruption record identifies the account, task, site, last confirmed action, outstanding question, and available result. It should be clear whether something was saved or submitted before the interruption.
That matters when a colleague resumes the work. Repeating a submission or overwriting information can be worse than a delay. Ask how the product prevents duplicate actions and how a person takes over safely within the configured process.
| Scenario | What to ask the provider to demonstrate |
|---|---|
| Session expires | How authentication is restored and work state is checked |
| Required question appears | How unsupported information reaches the team |
| Field rejects an answer | How the validation error is surfaced and corrected |
| Carrier returns a referral | How the outcome differs from a completed quote |
| Task stops after a save | How the next attempt avoids duplicating work |
| Site changes | Who validates and maintains the supported workflow |
Treat maintenance as part of the product
Ask who owns diagnosis, what information the agency receives, how fixes are tested, and when the workflow can be used again. A patch is not evidence that a product is inherently poor; a vague maintenance process is the problem.
If a provider publishes reliability metrics, ask for the definition, measurement period, supported scope, and treatment of interruptions. Uptime of the website, successful task completion, and correct output are different measures. Do not substitute one for another.
Contract duration is also a commercial term, not proof of reliability. Evaluate support commitments and the process your team will actually experience.
Test Relay on a defined task
Relay's browser workflows should be evaluated for the specific carrier, line, state, authentication, and action configured for your agency. Inspect the input, progress, returned result, and review path. Do not infer universal support or a guaranteed self-healing capability.
Start with how Relay works and the AI Quoting workflow. A useful demonstration includes the work your team can use and a clear explanation of what happens when it needs help.
See a Relay workflow your team can inspect from start to result.