Skip to content
RelayGet started

Carrier Portal Automation Benchmark

Relay scored head-to-head against leading general-purpose AI agents on real HO-3 carrier portal work.

Start with Relay

Scorecard

What we measured.

The run tested source-backed field fill, replacement cost, and rate completion on a live HO-3 carrier portal in June 2026.

91.9%

field accuracy on live portal work

3.6 min

median time to quote

1st

of 4 AI agents tested head-to-head

Metric
Relay
ChatGPT Codex
Claude Code
Gemini CLI

Field accuracy

Correct-fill rate averaged with error-free rate. Higher is better.

91.9%
84.1%
73.4%
68.6%

Time to quote

Median wall-clock per lead, steady state. Lower is better.

3.6 min
11.2 min
7.5 min
Incomplete

Output tokens per run

Tokens the model generates per run. Lower is better.

40K
236K
82K
36K*

HO-3 new business quoting on a live carrier portal, June 2026. Source-backed field fill, replacement cost, and rate, scored head-to-head.

*Gemini CLI never completed a run, so its token count reflects unfinished work.

Methodology

How the run worked.

  • Run the same HO-3 new business quote task against a live carrier portal.
  • Score source-backed field fill, replacement cost, and rate completion.
  • Measure median wall-clock time per lead after steady-state setup.
  • Track output tokens because lower token volume means lower cost and less wasted model work.
  • Keep the carrier and portal vendor anonymous on the public site.

Limits

What this does not claim.

  • This is one carrier-portal workflow, not a claim that every carrier or line behaves the same way.
  • The benchmark measures portal execution, not coverage judgment or binding authority.
  • A human still reviews intake and returned quotes before anything goes to a client or carrier.
  • These June 2026 results do not measure the current Workbench release or establish coverage for every carrier.
Historical research · June 2026

See the current quoting workflow.

This scorecard records an earlier evaluation. Current product pages explain Workbench, supported workflows, and the setup your agency needs. The result is not a promise of your quoting time.

See AI Quoting Get started

Your next step starts with one email.

Join the waitlist. We’re inviting agencies regularly and will email you when access is available.

Join the waitlist