Accurate against what?
Every accuracy claim is a comparison, which means somewhere there is an answer key. The first question is who wrote it. A scope graded against a file an experienced estimator wrote after walking the loss, that was submitted and paid, is a real result. A scope graded against another model's output, or against the vendor's own past exports, is a system grading itself and will look excellent forever.
Ask directly: who wrote the comparison estimates, did they see the property, and were those estimates approved. If the answer is vague, the number is decoration. This is not a hostile question — it is the same question a carrier asks you about your own documentation.
Line items or dollars?
Dollar accuracy is close to meaningless in this industry and it is the figure most often quoted, because it is the one that sounds biggest. Xactimate reprices an imported file against the price list for the area, so an identical scope produces materially different totals in two different markets. A vendor reporting dollar accuracy is partly reporting how well their price list matches yours, which has nothing to do with whether the scope was right.
Line items are the unit that survives. Did the file contain the work that needed to be done, at the right quantities, with the right activity? Score that. If a vendor can only give you a dollar figure, they either have not measured line items or did not like what they saw when they did.
A dollar accuracy claim is measuring the price list, not the scope.
Mitigation or rebuild, and which invoice?
These are different jobs with different scopes and they cannot share an answer key. Tear-outs, drying equipment, PPE, containment, and antimicrobial application belong to the mitigation invoice. Cabinets, flooring, paint, and trim belong to the rebuild. Grading a mitigation scope against a rebuild estimate produces a terrible number that says nothing, and grading the reverse produces a flattering one that says nothing either.
So when a vendor says "92 percent on estimates," ask which kind. Ask whether the same figure holds on the other kind. In our experience the two numbers are never the same, and a vendor quoting one number for both has not separated them.
Recall or precision, and can I see the misses?
There are two ways to be wrong. You can leave out work that should have been there, or you can add work that should not. They are not equally bad and they are not equally visible. Padding a file with plausible extra lines raises a naive accuracy score and gets caught by an adjuster later, which is your problem, not the vendor's.
The useful request is not the score, it is the miss list. Ask what the software habitually leaves out. Any vendor who has actually measured this can answer immediately and specifically. Then run one of your own closed files through it — a job you already wrote, already submitted, already got paid on — and compare line by line. Ten minutes of that will tell you more than any published percentage.
