Document extraction turns information in a document into fields another system can use. A key requirement in this guide is preserving meaning: which number belongs to which item, whether a quantity is in cases or individual units, and whether a handwritten change supersedes the printed value.
A successful extraction should preserve the link between the proposed value and its source. A successful workflow also validates the value, resolves uncertainty, and writes it to the destination correctly.
This chapter supports the purchase-order intake and catalog enrichment workflows, which work through complete examples. Stage descriptions draw on the sources cited; the schema, validation, and staging practices are this guide's proposed design.
Start from the input you actually receive
Before choosing any extraction technology, establish what arrives and in what form. The answer can make extraction unnecessary.
| Input | What it looks like | First approach to consider |
|---|---|---|
| Structured electronic data | A defined message format exchanged between systems | Map the fields directly; no document interpretation needed |
| Stable forms | The same layout from a known sender | Templates and rules, checked against the source |
| Digital documents with variable layouts | PDFs or spreadsheets that differ by sender | Document models or multimodal language models, with validation |
| Scans, photos, handwriting | Images of paper, with marks and amendments | OCR and layout analysis first; expect more exceptions. Images of products rather than documents belong to computer vision |
| Free text | An email body describing what the customer wants | Extraction with a high escalation rate for anything ambiguous |
Structured exchange already exists for some of these documents. X12 publishes a Purchase Order transaction set (850) that defines the format and data content of a purchase order for use in an EDI environment. X12 transaction sets Where a trading partner can send orders that way, interpreting a PDF of the same order adds cost and risk without adding information. The extraction question is then which partners cannot, and whether they could be offered a structured route such as a portal or form.
Separate the stages
For your technical lead
Optical character recognition identifies text in an image. Layout interpretation identifies relationships such as headings, tables, and labels. Field extraction maps information into the desired schema. Normalization represents values consistently. Validation checks them against rules and reference data.
Microsoft Research separately describes OCR, layout analysis, and information extraction. Our normalization, validation, and write stages extend those distinctions into a proposed business workflow. OCR and Document Understanding Correctly reading “12” does not tell you whether it means units, cases, a price, or a delivery date.
| Stage | Example output | Important check |
|---|---|---|
| Read | Text and page location | Was the character sequence captured correctly? |
| Interpret | Line-item row | Were columns and continuation pages understood? |
| Extract | Quantity and unit | Are the fields attached to the correct product? |
| Normalize | Consistent unit representation | Is the conversion explicit and correct? |
| Validate | Approved or flagged record | Does it satisfy business and reference-data rules? |
| Write | Destination transaction | Was it created once with the approved values? |
Errors compound across stages. A misread digit becomes a wrong quantity; a misread table boundary attaches a correct quantity to the wrong item; a correct extraction normalized with the wrong pack size becomes a wrong order. Evaluating only the final record hides which stage failed, and fixing the wrong stage wastes effort. Keep stage-level outputs in the evaluation log.
Compare approaches on your documents
A structured supplier feed may remove the need for document interpretation. Stable forms may be handled by templates and rules. Variable layouts may justify a document model or multimodal language model. Compare the full process on representative inputs rather than assuming the most flexible model is the best choice.
Include scanned and digital documents, multipage tables, amendments, duplicate attachments, and similar product variants where those occur in practice. Keep near-duplicate forms together when constructing development and held-out sets so familiar layouts do not inflate apparent generalization.
Flexible models handle new layouts without a template, but they can also produce plausible values that are not on the page. Templates tend to fail loudly when a layout no longer matches, though a small shift can still make them read the wrong zone; a flexible model may fail quietly. That difference matters more than average accuracy when an unnoticed error is expensive, and it is one reason the schema below insists on a source reference for every value.
Design a schema that preserves uncertainty
For your technical lead
Define each field's type, unit, required status, permitted values, and source reference. Distinguish missing, unreadable, conflicting, and not applicable. Turning all four into an empty string hides information the reviewer needs.
Preserve the original value alongside the normalized one. If “2 cases” becomes “24 units,” the conversion should be visible and supported by an approved mapping. Do not let a model invent the pack size.
Give repeated line items stable identities within the document. Otherwise a correction or retry may update the wrong row. Record the source document version so an amended purchase order does not silently merge with an earlier one.
A proposed structure for each extracted field:
| Element | Example | Purpose |
|---|---|---|
| Value as written | “2 cs” | Shows exactly what the source said |
| Normalized value | 24 each | What the destination receives, after conversion |
| Conversion basis | Pack size 12, item master | Makes the conversion checkable |
| Source reference | Page 1, line 3, quantity column | Lets a reviewer verify in seconds |
| Status | Present, missing, unreadable, conflicting, not applicable | Keeps uncertainty visible |
| Flags | Below threshold, outside permitted values, conflicts with another field | Routes the field to review |
The source reference is the most valuable element. A reviewer who can jump to the exact span checks a value quickly; one who has to search the document re-does the extraction.
Normalize with code, not with the model
For your technical lead
Unit conversions, date formats, identifier formats, and permitted-value mappings are deterministic. Do them in ordinary code, using approved reference data, and keep the source value. NIST publishes conversion factors such as pounds to kilograms and pounds per square inch to kilopascals, which the catalog chapter uses in its worked example. NIST Guide to the SI, Appendix B.8
Three cases need particular care:
- Pack and case quantities. The conversion between cases and units belongs to the item master, not to the document or the model. If the document states a pack size that disagrees with the master, flag it.
- Designations that look like measurements. A nominal pipe size or a model number containing digits is a name, not a length. Converting it produces a confident, wrong value.
- Values outside the controlled vocabulary. Propose a mapping for review rather than forcing the nearest permitted value.
Use confidence carefully
For your analyst
Some extraction products return confidence estimates for fields or other elements. Microsoft's documentation describes several such scores and notes that not all fields return one. These are product-specific signals to evaluate, not a universal definition of correctness. Microsoft accuracy and confidence guidance
Before using a threshold to route work, compare scores with actual errors on your documents. Examine critical fields separately. A high-scoring extraction can still refer to the wrong item or use an obsolete document. A language model's verbal statement that it is confident is not interchangeable with a validated extraction score.
Research on neural classifiers shows why local checking is needed. Guo and colleagues found that the modern neural networks they studied were poorly calibrated, meaning their probability estimates did not match how often they were correct, and that a simple post-processing step improved calibration on most of the image and document classification datasets they tested. Guo et al., arXiv 1706.04599 That work predates current extraction products and concerns classification, not field extraction. It supports a practice rather than a prediction: build a reliability table before trusting a threshold. Group evaluated fields by score band, record the observed error rate in each band, and set the review threshold where the observed error rate becomes acceptable for that field. Repeat it for each critical field and after any model or document change.
Evaluate fields and completed documents
For your analyst
Report critical-field errors, unsupported additions, missing values, and correct complete records. A large number of easy fields can dominate a field-level average while a consequential quantity error remains hidden.
Document-level correctness also compounds: a record is right only if every field in it is right, so a high field accuracy can still leave a noticeable share of documents with at least one error. The figure below illustrates this under the assumption that field errors are independent; real errors cluster by document quality and layout.
High field accuracy, lower document accuracy
- Each field 98.0% correct
- Each field 99.5% correct
Show the data
| Field accuracy | All 20 fields correct |
|---|---|
| 98.0% | 66.8% |
| 99.5% | 90.5% |
Measure reviewer effort and downstream correction. Record how often the system escalates and whether those escalations are useful. A low error rate achieved by sending almost everything back to employees may be acceptable for some workflows but offers a different economic case from broad automation.
A proposed evaluation layout:
| Level | Measure | Why |
|---|---|---|
| Field | Errors by field, with critical fields reported separately | Finds the fields that need rules or review |
| Field | Unsupported values: present in the output, absent from the source | Catches invented values that look plausible |
| Document | Share of documents with every critical field correct | Matches what the destination actually needs |
| Document | Documents correctly held for review | Rewards escalation where it is warranted |
| Workflow | Reviewer minutes per document, including held ones | Captures the real cost of the exceptions |
| Workflow | Errors found after writing | The errors that reached customers or ledgers |
Exceptions are the economics
The cost of an extraction workflow sits largely in the documents that do not pass straight through. A document that needs review can take longer than keying it by hand, because the reviewer must read both the source and the proposed values. As the exception share rises, the time saved on clean documents can be consumed by the rest.
The order-intake chapter works through this with illustrative numbers: assisted effort rises with the exception share and crosses the manual baseline at a point the chapter's figure marks. Before a pilot, estimate the exception share by document source. Some senders may produce clean documents and others mostly exceptions; the second group may be better served by a structured channel.
Keep validation and authorization explicit
For your technical lead
Use deterministic checks for required fields, permitted units, arithmetic relationships, known customer identities, and duplicate references. A valid schema does not establish business correctness: an incorrect product code can still be a perfectly valid string.
The pilot should stage proposed changes for an authorized reviewer. Check the write path independently, including retries and partial failure. A reliable extractor connected to an unreliable integration can create expensive errors. Writes need idempotent submission, because an uncertain response does not establish that the original write failed; the order-intake chapter shows the sequence.
Plan for new variants
Document sources change. A supplier redesigns its form, a customer starts sending scans instead of PDFs, an amended document arrives with handwritten corrections. A system evaluated on last quarter's documents may meet this quarter's without warning.
Proposed controls: track exception and correction rates by sender, flag a sender whose rates move, keep a sample of each new layout for the evaluation set, and assign an owner for reference data such as pack sizes and customer item codes. A rising exception rate for one sender is a signal to inspect that sender's documents before it shows up as downstream errors.
Questions to ask before buying or building
The answers to these questions change the decision more than a demonstration on clean samples does:
- Source references: does every extracted value come with the page and location it was read from, and can a reviewer jump to it?
- Missing versus empty: can the output distinguish a missing value from an unreadable, conflicting, or not-applicable one?
- Confidence: which fields return a confidence signal, how was it validated, and can it be checked against errors on our documents?
- New layouts: what happens when a new layout arrives: a loud failure, a template request, or a silent best guess?
- Amendments: how are amended or duplicate documents identified, and linked to the earlier version?
- Data handling: where are documents processed and stored, for how long, and are they used for any other purpose?
- Write path: what does the integration do on a timeout, and how is a duplicate write prevented?
- Monitoring: what will we see each week: exception rate by sender, correction rate after review, errors found after writing?
Ask for an evaluation on a sample of your own documents, including difficult ones, scored by field and by complete document. A vendor's aggregate accuracy figure measured on other documents does not establish performance on yours.
Decide whether to proceed
Proceed when the system improves the complete task while meeting the critical-field requirements and keeping exceptions manageable. Defer when source identity is unclear, product mappings are inconsistent, or structured intake solves the problem more directly.
Budget for new document variants and reference-data maintenance. Document processing is an operating capability with continuing input changes, not a one-time exercise in reading a clean sample.
Photographs of products, shelves, and other non-document images are covered in computer vision, and recorded calls in speech systems; each needs its own evaluation on the inputs you actually receive.
Related chapters: order intake, catalog enrichment, model selection, retrieval, human review, pilot design.