An invoice extraction evaluation that accepts a €1,000 subtotal as the €1,200 amount due has missed a field definition. The response can be valid JSON, the copied number can appear on the page, and the record can still be wrong.
That is the failure this worksheet is built around. It asks what each field means, how the source supports it, and what happens when the result reaches the next step.
Invoice extraction evaluation: the field contract
Start with the workflow: extracting a draft for an accounts payable reviewer is a different task from preparing data for an unattended action. Write the intended use above the field specification.
| Field | Definition to settle | Check |
|---|---|---|
| Supplier | Legal entity issuing the invoice | Match the source identity |
| Invoice number | Supplier's invoice identifier | Preserve meaningful zeros and characters |
| Invoice date | Issue date | Distinguish it from due date and service dates |
| Currency | Currency of the billed amount | Follow the explicit-evidence or inference policy |
| Amount due | Amount currently owed | Check value, currency, and field meaning |
| Line items | Individual billed entries | Match the required rows and attributes |
| Payment details | Details permitted in the draft | Verify source and review handling |
Flag optional fields. Define nulls and conflicts. If a value may be calculated, record the permitted calculation. An inferred due date should have a different basis from a date printed beside “due.”
A test record someone else can review
Retain the original file, not only the extracted text. A reviewer may need the table layout to tell which amount belongs to which label.
A case record can be small:
- •Case ID and exact source-file version.
- •Document type, supplier-layout family, language, and scan condition.
- •Expected fields and their source locations.
- •Rules for normalization and allowed alternatives.
- •Expected handling of absent, unreadable, or conflicting information.
- •Reviewer and unresolved label questions.
Use a golden dataset to preserve the reviewed cases. Keep unfamiliar supplier layouts in a separate evaluation set when that is the kind of variation the product needs to handle.
If OCR is part of processing, save the OCR result used in the run. A wrong digit introduced there calls for a different fix from a correctly read figure assigned to the wrong field.
Why line-item counts are not enough
Three output rows do not establish that three source rows were extracted correctly. A price can move to the next row; a continuation line can become a separate charge.
For each required line item, define how it is matched to the source. A stable item code may be enough. Repeated descriptions may require quantity, source position, or reviewer judgment.
Then check the attributes on the matched row. For a quantity and unit-price calculation, state any permitted rounding rule. An arithmetic tolerance should not excuse choosing another item's price.
Give discounts and continuation lines explicit treatment. Otherwise, the evaluation may call a different representation wrong before the team has agreed which representation it needs.
How should invoice extraction accuracy be reported?
Use both a field result and a document result.
Take ten invoices with six required header fields each. If 57 fields are correct, the exact-field pass rate is 57/60, or 95%. If the three errors occur on different invoices, seven invoices have all six fields correct: a 70% complete-header pass rate.
That example says nothing about line items unless they were included in the rule. Name the scope precisely.
A complete-document acceptance rule might require all mandatory header values, correctly matched required line items, and no unsupported payment detail. Report the number meeting that whole rule. Do not average away a wrong currency with several correct optional notes.
The field-level metrics guide covers the matching and denominator choices behind those numbers.
Business rules are another layer
Check the rules your product uses after extraction. Can a credit note follow this workflow? How is a duplicate invoice handled? What happens when line-item arithmetic and the printed total disagree?
These checks can catch an unusable record without identifying the cause. A total mismatch may reflect an extraction mistake, a rule the parser does not support, or an inconsistency in the source. Keep the diagnosis open until someone inspects the evidence.
A business-rule pass also cannot prove that every copied field is correct. Test the source values and the downstream rules separately, then run the whole workflow.
The release comparison
Run the baseline and candidate on the same reviewed batch with the same matching rules. Save prompt, model configuration, parser, and dataset versions.
Compare the failed cases as well as the scores. Did the candidate repair repeated-total errors but lose supplier identifiers? Did unfamiliar layouts improve while scanned pages became worse?
Add review load and processing time to that discussion. A configuration that returns more records for human correction may change the operating cost even if its model-call price falls. Use the existing document extraction cost study for the model-cost discussion and your own measurements for the workflow decision.
Which records should reach a reviewer?
Decide before the batch runs. Missing optional information may be acceptable; unresolved supplier identity or unsupported payment details may require a stop.
The review screen should show the value, its source, and the reason for the flag. Measure whether reviewers can resolve it, and preserve corrected cases for later testing.
Bring this worksheet into the broader document automation evaluation plan. It gives the team a shared definition of a usable invoice record, from the first copied field to the point where someone relies on it.
Enjoyed reading? Add us as a preferred source in Google to support us.
Add Lovelaice as a preferred source
