Complex production documents fail in relationships, not only characters. Structure-aware document parsing preserves relationships between headings, tables, fields, values, and pages. Developers should evaluate the returned data structure and its evidence. Do not select a system from a generic vendor list or a single accuracy number. Use the AI-agent parser evaluation guide for product selection. Use this guide to test whether a chosen system can preserve the relationships your application needs.
How to evaluate structure-aware document parsing
Use a real document with known values and difficult structure. This example uses the public multi-page bank statement in Extend's API quickstart. The same file is available in the site's public document bank, with a saved Parse 2.0 output for inspection.
Evaluate these signals independently:
- Reading order: Headings, account summaries, and transaction tables must appear in the order a person reads them.
- Nested objects and arrays: Each account must own its balances and transactions.
- Parent-child relationships: Every transaction must reference the correct parent account.
- Repeated records: The transaction count must match the ground truth across all pages.
- Cross-page references: A continued table must retain the correct header and account context.
- Source citations: Each material field must point to the correct page and region.
- Field confidence: Confidence or review signals must be available at the field path.
- Schema validation: The final value must conform to the declared schema and business rules.
Failure modes and required verification
| Failure mode | Required output signal | Verification method |
|---|---|---|
| Column or section order changes | Ordered chunks or blocks with page metadata | Compare the parsed sequence with human reading order |
| A child field is attached to the wrong parent | Nested object or explicit stable parent key | Assert that every child key matches its enclosing object |
| Repeated rows are omitted or duplicated | Complete array with stable cardinality | Compare array length and row identifiers with ground truth |
| A table continues on another page | Page-aware rows with retained table and parent context | Test records before and after each page boundary |
| A plausible value comes from the wrong region | Field-level citation with page and bounding polygon | Render the citation over the source PDF and inspect the region |
| A low-quality value passes automatically | Field-level review or confidence signal | Apply a fixed threshold and measure review precision and recall |
| The response shape drifts | Versioned JSON Schema and schema-valid output | Validate every response before the next workflow step |
| Totals do not reconcile | Explicit business-rule result | Recalculate totals from the returned rows and compare balances |
Complete schema and API call
The schema below represents one statement with a parent account and repeated transactions. Each transaction repeats the parent account number. That deliberate redundancy makes a wrong parent-child link testable.
from extend_ai import Extend
client = Extend()
schema = {
"type": "object",
"properties": {
"statement_period": {
"type": "object",
"properties": {
"start_date": {"type": ["string", "null"], "extend:type": "date"},
"end_date": {"type": ["string", "null"], "extend:type": "date"},
},
"required": ["start_date", "end_date"],
},
"account": {
"type": "object",
"properties": {
"account_number": {"type": ["string", "null"]},
"beginning_balance": {"type": ["number", "null"]},
"ending_balance": {"type": ["number", "null"]},
"transactions": {
"type": "array",
"items": {
"type": "object",
"properties": {
"posted_date": {
"type": ["string", "null"],
"extend:type": "date",
},
"description": {"type": ["string", "null"]},
"amount": {"type": ["number", "null"]},
"running_balance": {"type": ["number", "null"]},
"parent_account_number": {
"type": ["string", "null"],
"description": "The account number that owns this transaction",
},
},
"required": [
"posted_date",
"description",
"amount",
"running_balance",
"parent_account_number",
],
},
},
},
"required": [
"account_number",
"beginning_balance",
"ending_balance",
"transactions",
],
},
},
"required": ["statement_period", "account"],
}
run = client.extract(
file={
"url": "https://extend-public-files.s3.us-east-2.amazonaws.com/bank_statement_example.pdf"
},
config={
"schema": schema,
"advancedOptions": {"citationsEnabled": True},
},
)The schema documentation defines supported nested objects, arrays, nullable fields, and typed dates. The extraction overview explains how Extract uses parsed content. The synchronous call is suitable for testing. Use asynchronous runs and webhooks for large files and production volume.
Abbreviated structure-aware response
The API returns values and field metadata under matching paths. This abbreviated shape uses values visible in the public sample. Exact confidence values and polygons vary by run.
{
"value": {
"statement_period": {
"start_date": "2020-12-22",
"end_date": "2021-01-25"
},
"account": {
"account_number": "000000861006133",
"beginning_balance": 438.59,
"ending_balance": 459.25,
"transactions": [
{
"posted_date": "2020-12-24",
"description": "Discover E-Payment 6930",
"amount": -27.90,
"running_balance": 410.69,
"parent_account_number": "000000861006133"
},
{
"posted_date": "2020-12-29",
"description": "ATM Cash Deposit",
"amount": 101.00,
"running_balance": 438.60,
"parent_account_number": "000000861006133"
}
]
}
},
"metadata": {
"account.account_number": {
"citations": [{"page": {"number": 1}, "referenceText": "000000861006133"}]
},
"account.transactions[0].amount": {
"citations": [{"page": {"number": 1}, "referenceText": "-27.90"}]
}
}
}The nested object proves ownership. The repeated parent_account_number makes each row independently testable. The metadata path binds a field to its source. See the current response format for the complete citation, confidence, and review shape.
Validation rule before the next agent action
Do not send extracted data to an agent because the response is valid JSON. Validate the relationships and the business rule first.
from decimal import Decimal
value = run.output.value
metadata = run.output.metadata
account = value["account"]
assert all(
row["parent_account_number"] == account["account_number"]
for row in account["transactions"]
)
assert all(
metadata.get(f"account.transactions[{index}].amount", {}).get("citations")
for index in range(len(account["transactions"]))
)
net_change = sum(
Decimal(str(row["amount"])) for row in account["transactions"]
)
assert Decimal(str(account["beginning_balance"])) + net_change == Decimal(
str(account["ending_balance"])
)
next_agent_action = "post_statement" # Run only after all assertions pass.In a production workflow, a failed assertion should route the document to review. The workflow must not silently discard a repeated row or accept a value from the wrong page.
Which benchmark answers which question
RealDoc-Bench measures parsing layout with Adjusted F1 and document Q&A with field-level accuracy. The public corpus includes 1,500 layout samples and 1,359 prompts across 581 documents in financial services, real estate, logistics, and healthcare. It does not measure business-schema extraction or every private document type.
LongArray-Extract measures complete repeated-record extraction on 45 financial, clinical, and legal PDFs. Failed and timed-out runs score zero. It tests array cardinality and field accuracy, not generic RAG retrieval or PDF-to-Markdown quality.
Use the benchmark hub for methodology and public data. For product selection, return to the AI-agent parser evaluation guide. For implementation, use the extraction docs, evaluation docs, and workflow docs.
A private-corpus test plan
- Define the exact schema, required relationships, and business rules.
- Select documents with page breaks, repeated records, nested tables, and known scan defects.
- Label every expected value and parent-child link before the test.
- Run the same product mode and configuration across the full corpus.
- Count missing rows, duplicates, wrong parents, failures, and timeouts.
- Render citations for every material error.
- Measure review volume at the threshold your workflow will use.
- Ship only after the schema and business-rule checks pass on held-out documents.
FAQ
What is structure-aware document parsing?
Structure-aware document parsing preserves relationships between headings, tables, fields, values, and pages. Structured extraction can then keep each value attached to its parent object, repeated record, and source region.
How do I test nested and repeated fields?
Define the target schema before the run. Score parent-child links, array cardinality, field values, source citations, and schema validity separately.
