How Vortex IQ Measures Execution: TTF, FAR, FRR

Version 1, September 2026. Any change to a definition is dated in the change log at the end, so that figures from different periods stay comparable.

What this page is for

The three numbers we hold ourselves to are only useful if they mean the same thing every time they are quoted. This page is the definition of record. The companion article, The execution gap, explains why we chose them. This page explains exactly how each one is calculated, so that any figure a merchant sees in their workspace can be traced to a rule written here, and so that any provider quoting the same names can be asked whether they mean the same thing.

The unit of measurement

The unit is a defined corrective job. A job begins when an issue is first recorded, is owned by one crew, and ends when a correction has been verified in the live store against a completion criterion set when the job was opened.

Not everything the AI workforce for ecommerce does is a fix. A research brief, a new product launch and a full platform migration are projects with their own measures: accepted delivery, launch completeness, verified recovery. They are not counted here, and forcing them into a fix scorecard would flatter the numbers while telling the merchant nothing. A remediation inside one of those projects, such as repairing a broken redirect discovered during a migration, is a fix and is measured as one.

Six crews do the work. Pulse for Store Health, CodeCraft for Store Development and Bridge for Staging and Migration stop revenue leaks. Beacon for SEO and AI Visibility, Muse for Store Experience and Compass for Ads Performance accelerate growth. The measures below apply to corrective work in every crew, and are reported per crew as well as in total.

The record every fix leaves behind

Every job produces one record. The three numbers are calculated from it and from nothing else.

Detection. The time the issue was first recorded, and the source: monitor, shopper report, manual entry.
Priority. The business weight assigned at detection, so counts can be read against importance.
First proposal. The time it was ready, the evidence supporting it, the proposed change, and the expected effect.
Review decision. Who decided, when, and the outcome: approved unchanged, revised, or rejected, with a reason category.
Execution. What ran, when, against what, and the rollback point retained where the workflow supports one.
Reuse. Whether a previously validated solution was used, which one, and how its fit was checked for this store.
Verification. The check performed, the time it passed or failed, and the evidence retained.
Reopen. Any recurrence within the validation window, with the original detection time preserved.

A number that cannot be traced back to one of these fields is not reported.

Time to Fix (TTF)

Definition. The elapsed time between the first recorded detection of an issue and verification that the intended correction is live and working.

Calculation. For one job: TTF = verification time minus first detection time

The start. The clock starts at first detection, whatever came after. If a ticket was opened two days later, those two days count. The merchant carried the problem for them.

The finish. The clock stops at evidence, not at activity. Deploying code, updating a field, or receiving a successful API response are intermediate steps. Completion requires a check appropriate to the issue: a passing journey test for a checkout fault, a confirmed live state for a catalogue correction, a render check for a template change. The check type is set when the job opens and recorded when it runs.

Approval time is included, and shown separately. The headline figure contains the time spent waiting for a person, because the merchant experiences that wait. It is also reported as one of three components, so that the figure is actionable:

Preparation: detection to first proposal
Approval: first proposal to review decision
Execution and verification: decision to passed check

A long preparation time points to investigation or capability. A long approval time usually points to unclear evidence or an unclear owner. A long verification time exposes weak testing or an unsuitable completion criterion. The response to a long approval time is to make review easier, never to remove the reviewer.

Failure keeps the clock running. If verification fails, the job is not complete and TTF continues to accrue. If an issue reopens within its validation window, the record is flagged, the original detection time is retained, and the resolution is measured from that original time. A fix that did not hold is not a fast fix.

Reporting. Over any period we report the median TTF and the 90th percentile across completed jobs, never the mean alone, because one difficult job should not hide inside an average. Alongside them we report the number and age of issues still open, so that a fast median among finished work cannot conceal a growing backlog of hard work.

Figures are segmented by issue type, severity, platform and whether human approval was required. Repairing a missing image reference and resolving an intermittent checkout failure are different work, and combining them into one headline without showing the mix conceals more than it explains.

What TTF does not claim. It measures how long a problem persisted. It does not measure how much the problem cost, which depends on traffic, margin and what was affected. It is read alongside business priority, not instead of it.

Fix Acceptance Rate (FAR)

Definition. The percentage of reviewed issues whose first submitted fix proposal was approved without human edits or a request for revision.

Calculation. FAR = first proposals approved unchanged, divided by first proposals that received a review decision, times 100

One first proposal per issue. A proposal that is revised and later approved does not become a first-pass acceptance. The original decision stands in FAR; the eventual approval is recorded in the completion record. Counting resubmissions as fresh opportunities would hide the work the reviewer actually had to do, which is the thing FAR exists to show.

Pending is excluded. A proposal awaiting a decision is reported as pending and enters the calculation only once decided.

Unchanged means unchanged. Approved without alteration to the proposed change and without a request for revision. A reviewer's comment that does not alter the change is not an edit. A change to scope, wording, target, timing or implementation is.

Standing approvals are reported separately. Where a merchant has authorised a class of change to proceed under a standing policy, those approvals are counted and shown on their own line. They are never blended into FAR, which measures a human decision.

Every non-acceptance carries a reason. Revisions and rejections are categorised at the point of decision:

Wrong scope: the proposal addressed more or less than the issue
Missing context: the proposal did not account for something the store knows
Brand or tone: the change was technically correct and commercially unacceptable
Commercial rule: the change conflicted with a pricing, stock or promotion rule
Risk: the reviewer judged the change unsafe to apply as proposed
Technical error: the proposed change would not have worked
Other, with a note

The reasons are more useful than the percentage. A low FAR with reasons clustered on missing context is a data problem. Clustered on scope, it is an instruction problem. We treat every reason as evidence about what the workforce needs to understand better.

Reporting. Reviewed count, pending count, approved unchanged, revised, rejected, the reason breakdown, and the verification outcomes of the approved proposals. A high FAR is meaningful only when reviewers had enough evidence and time to make a real decision, so approval time from TTF is shown beside it.

What FAR does not claim. Acceptance is not correctness. An approved proposal still has to execute and pass verification. FAR measures the quality of the handover to the reviewer, not the quality of the outcome.

Fix Reuse Rate (FRR)

Definition. The percentage of verified completed fixes that materially used a previously validated solution.

Calculation. FRR = verified fixes using a validated solution, divided by all verified completed fixes, times 100

Validated has a specific meaning. A solution counts as validated only if it was verified on an earlier job and its record includes the conditions under which it applies, the checks required before applying it, the change itself, and the evidence required afterwards. A suggestion that resembles an earlier answer is not a validated solution.

Material use has a specific meaning. The reused element must contribute to resolving the specific issue: a procedure, an implementation pattern, or a substantive component of the change, applied under its recorded conditions and checked in the current store. Reusing a generic prompt, opening the same connector, or producing something that looks like an earlier fix does not count.

Within-store and across-store reuse are reported separately. Reuse within one store shows retained context. Reuse across stores shows that a method travels. A claim that the workforce learns across the merchant base rests on the second figure, and it is never made from the first.

Methods travel. Merchant data does not. A transferable method is separated from the data of the store it was learned on and applied elsewhere only where reuse is permitted and the conditions match. Each store keeps its own context, permissions and checks. No merchant's information moves to another merchant's store.

A rising FRR is evidence of reuse, not of benefit. To establish that reuse saves anything, the figure must be accompanied by lower measured delivery cost or less human effort on comparable work, with quality maintained. Reuse carries its own validation and maintenance costs, and a report that shows FRR without those is showing a number, not a result.

Novelty is not failure. A store with a different configuration, a new commercial requirement or a genuinely new problem needs a new solution. Recognising that, rather than forcing the job into an existing pattern, is part of competent execution, and a fix that correctly declines to reuse is not a mark against the workforce.

Reporting. Verified fixes, fixes with reuse, within-store and across-store split, and for each reused fix the reference to the solution used and the check that confirmed its fit.

Reading the three together

The most useful question is how they move in relation to each other over comparable work.

TTF falls, FAR falls. Proposals are reaching people faster and worse. Look at whether speed is creating correction work.
FAR rises, TTF stays high. Proposal quality is up and something else is slow. Look at the approval queue and deployment constraints.
FRR rises, TTF and cost unchanged. Recorded reuse is too shallow to save work. Look at whether effort has moved to checks and coordination.
All three improve on comparable work. Execution is genuinely getting better. Check that the work mix and priority have stayed constant.

The underlying work must still matter. Counts and rates are read against business impact and against the work left undone.

The reporting standard

Any report of these figures states:

The observation period
The number of stores and the number of issues
The mix of work by type and severity
The approval policy in force, including any standing approvals
The verification window applied to each issue type
The version of these definitions used

Unfinished, failed and reopened work is reported with the same prominence as successful completions. A method earns credibility by explaining its delays and rejections as clearly as its successes.

Where the numbers appear

Each merchant sees their own figures in their Vortex IQ workspace, per crew and in total, updated as jobs complete, with a monthly summary. The record behind any figure is one step away from the figure itself. A merchant who asks how a number was reached is shown the jobs, not a footnote.

Change log

Version 1, September 2026. First published.

Frequently asked questions

What counts as a fix?

A fix is a defined corrective job. It starts when an issue is first recorded, is owned by one crew, and ends when the correction is verified in the live store against a completion criterion set when the job opened. Research briefs, launches and full migrations are projects with their own measures and are not counted as fixes.

Does time waiting for approval count in Time to Fix?

Yes. The merchant experiences that wait, so it is inside the headline figure. It is also shown as a separate component alongside preparation time and execution and verification time, so a long Time to Fix can be traced to its cause rather than argued about.

What happens to TTF if a fix fails verification or the issue comes back?

The clock keeps running until verification passes. If the issue reopens within its validation window, the record is flagged and the resolution is measured from the original detection time. A fix that did not hold is never counted as a fast fix.

Are automatic or policy-based approvals included in Fix Acceptance Rate?

No. Fix Acceptance Rate measures a human decision on a first proposal. Approvals made under a standing policy the merchant has authorised are counted and reported on their own line, and never blended into FAR.

What counts as reuse for Fix Reuse Rate?

A previously validated solution, applied under its recorded conditions and checked in the current store, that materially contributes to the fix. Reusing a generic prompt, opening the same connector, or producing something that resembles an earlier fix does not count.

How often are the figures updated, and can I see the jobs behind them?

Figures update as jobs complete and are summarised monthly in each merchant's Vortex IQ workspace. Every figure links to the jobs it was calculated from, so a merchant who asks how a number was reached is shown the record, not a footnote.

How Vortex IQ measures execution