---
title: "Evaluate Document Parsers on Complex Documents"
description: "Evaluate reading order, nested data, repeated records, cross-page relationships, citations, confidence, and schema validation in document parsers."
author: "Kushal Byatnal"
category: "Blog Post"
published: 2026-04-13
updated: 2026-07-31
canonical: https://www.extend.ai/resources/best-document-parsing-software-for-developers
---

# Evaluate Document Parsers on Complex Documents

Complex production documents fail in relationships, not only characters. Structure-aware document parsing preserves relationships between headings, tables, fields, values, and pages. Developers should evaluate the returned data structure and its evidence. Do not select a system from a generic vendor list or a single accuracy number. Use the [AI-agent parser evaluation guide](https://www.extend.ai/resources/choosing-document-parser-ai-agents) for product selection. Use this guide to test whether a chosen system can preserve the relationships your application needs.

## How to evaluate structure-aware document parsing

Use a real document with known values and difficult structure. This example uses the public multi-page bank statement in Extend's [API quickstart](https://docs.extend.ai/api-quickstart). The same file is available in the site's public document bank, with a saved Parse 2.0 output for inspection.

Evaluate these signals independently:

- **Reading order:** Headings, account summaries, and transaction tables must appear in the order a person reads them.
- **Nested objects and arrays:** Each account must own its balances and transactions.
- **Parent-child relationships:** Every transaction must reference the correct parent account.
- **Repeated records:** The transaction count must match the ground truth across all pages.
- **Cross-page references:** A continued table must retain the correct header and account context.
- **Source citations:** Each material field must point to the correct page and region.
- **Field confidence:** Confidence or review signals must be available at the field path.
- **Schema validation:** The final value must conform to the declared schema and business rules.

## Failure modes and required verification

| Failure mode | Required output signal | Verification method |
| --- | --- | --- |
| Column or section order changes | Ordered chunks or blocks with page metadata | Compare the parsed sequence with human reading order |
| A child field is attached to the wrong parent | Nested object or explicit stable parent key | Assert that every child key matches its enclosing object |
| Repeated rows are omitted or duplicated | Complete array with stable cardinality | Compare array length and row identifiers with ground truth |
| A table continues on another page | Page-aware rows with retained table and parent context | Test records before and after each page boundary |
| A plausible value comes from the wrong region | Field-level citation with page and bounding polygon | Render the citation over the source PDF and inspect the region |
| A low-quality value passes automatically | Field-level review or confidence signal | Apply a fixed threshold and measure review precision and recall |
| The response shape drifts | Versioned JSON Schema and schema-valid output | Validate every response before the next workflow step |
| Totals do not reconcile | Explicit business-rule result | Recalculate totals from the returned rows and compare balances |

## Complete schema and API call

The schema below represents one statement with a parent account and repeated transactions. Each transaction repeats the parent account number. That deliberate redundancy makes a wrong parent-child link testable.

```python
from extend_ai import Extend

client = Extend()

schema = {
    "type": "object",
    "properties": {
        "statement_period": {
            "type": "object",
            "properties": {
                "start_date": {"type": ["string", "null"], "extend:type": "date"},
                "end_date": {"type": ["string", "null"], "extend:type": "date"},
            },
            "required": ["start_date", "end_date"],
        },
        "account": {
            "type": "object",
            "properties": {
                "account_number": {"type": ["string", "null"]},
                "beginning_balance": {"type": ["number", "null"]},
                "ending_balance": {"type": ["number", "null"]},
                "transactions": {
                    "type": "array",
                    "items": {
                        "type": "object",
                        "properties": {
                            "posted_date": {
                                "type": ["string", "null"],
                                "extend:type": "date",
                            },
                            "description": {"type": ["string", "null"]},
                            "amount": {"type": ["number", "null"]},
                            "running_balance": {"type": ["number", "null"]},
                            "parent_account_number": {
                                "type": ["string", "null"],
                                "description": "The account number that owns this transaction",
                            },
                        },
                        "required": [
                            "posted_date",
                            "description",
                            "amount",
                            "running_balance",
                            "parent_account_number",
                        ],
                    },
                },
            },
            "required": [
                "account_number",
                "beginning_balance",
                "ending_balance",
                "transactions",
            ],
        },
    },
    "required": ["statement_period", "account"],
}

run = client.extract(
    file={
        "url": "https://extend-public-files.s3.us-east-2.amazonaws.com/bank_statement_example.pdf"
    },
    config={
        "schema": schema,
        "advancedOptions": {"citationsEnabled": True},
    },
)
```

The [schema documentation](https://docs.extend.ai/extraction/schema) defines supported nested objects, arrays, nullable fields, and typed dates. The [extraction overview](https://docs.extend.ai/extraction/overview) explains how Extract uses parsed content. The synchronous call is suitable for testing. Use [asynchronous runs and webhooks](https://docs.extend.ai/general/async-processing) for large files and production volume.

## Abbreviated structure-aware response

The API returns values and field metadata under matching paths. This abbreviated shape uses values visible in the public sample. Exact confidence values and polygons vary by run.

```json
{
  "value": {
    "statement_period": {
      "start_date": "2020-12-22",
      "end_date": "2021-01-25"
    },
    "account": {
      "account_number": "000000861006133",
      "beginning_balance": 438.59,
      "ending_balance": 459.25,
      "transactions": [
        {
          "posted_date": "2020-12-24",
          "description": "Discover E-Payment 6930",
          "amount": -27.90,
          "running_balance": 410.69,
          "parent_account_number": "000000861006133"
        },
        {
          "posted_date": "2020-12-29",
          "description": "ATM Cash Deposit",
          "amount": 101.00,
          "running_balance": 438.60,
          "parent_account_number": "000000861006133"
        }
      ]
    }
  },
  "metadata": {
    "account.account_number": {
      "citations": [{"page": {"number": 1}, "referenceText": "000000861006133"}]
    },
    "account.transactions[0].amount": {
      "citations": [{"page": {"number": 1}, "referenceText": "-27.90"}]
    }
  }
}
```

The nested object proves ownership. The repeated `parent_account_number` makes each row independently testable. The metadata path binds a field to its source. See the current [response format](https://docs.extend.ai/extraction/response-format) for the complete citation, confidence, and review shape.

## Validation rule before the next agent action

Do not send extracted data to an agent because the response is valid JSON. Validate the relationships and the business rule first.

```python
from decimal import Decimal

value = run.output.value
metadata = run.output.metadata
account = value["account"]

assert all(
    row["parent_account_number"] == account["account_number"]
    for row in account["transactions"]
)
assert all(
    metadata.get(f"account.transactions[{index}].amount", {}).get("citations")
    for index in range(len(account["transactions"]))
)

net_change = sum(
    Decimal(str(row["amount"])) for row in account["transactions"]
)
assert Decimal(str(account["beginning_balance"])) + net_change == Decimal(
    str(account["ending_balance"])
)

next_agent_action = "post_statement"  # Run only after all assertions pass.
```

In a production workflow, a failed assertion should route the document to review. The workflow must not silently discard a repeated row or accept a value from the wrong page.

## Which benchmark answers which question

[RealDoc-Bench](https://www.extend.ai/resources/realdocbench) measures parsing layout with Adjusted F1 and document Q&A with field-level accuracy. The public corpus includes 1,500 layout samples and 1,359 prompts across 581 documents in financial services, real estate, logistics, and healthcare. It does not measure business-schema extraction or every private document type.

[LongArray-Extract](https://www.extend.ai/resources/long-array-extraction-benchmark) measures complete repeated-record extraction on 45 financial, clinical, and legal PDFs. Failed and timed-out runs score zero. It tests array cardinality and field accuracy, not generic RAG retrieval or PDF-to-Markdown quality.

Use the [benchmark hub](https://www.extend.ai/benchmarks) for methodology and public data. For product selection, return to the [AI-agent parser evaluation guide](https://www.extend.ai/resources/choosing-document-parser-ai-agents). For implementation, use the [extraction docs](https://docs.extend.ai/extraction/overview), [evaluation docs](https://docs.extend.ai/evaluation/overview), and [workflow docs](https://docs.extend.ai/workflows/overview).

## A private-corpus test plan

1. Define the exact schema, required relationships, and business rules.
2. Select documents with page breaks, repeated records, nested tables, and known scan defects.
3. Label every expected value and parent-child link before the test.
4. Run the same product mode and configuration across the full corpus.
5. Count missing rows, duplicates, wrong parents, failures, and timeouts.
6. Render citations for every material error.
7. Measure review volume at the threshold your workflow will use.
8. Ship only after the schema and business-rule checks pass on held-out documents.

## FAQ

### What is structure-aware document parsing?

Structure-aware document parsing preserves relationships between headings, tables, fields, values, and pages. Structured extraction can then keep each value attached to its parent object, repeated record, and source region.

### How do I test nested and repeated fields?

Define the target schema before the run. Score parent-child links, array cardinality, field values, source citations, and schema validity separately.

## Related reading

- [Best Document Parsing APIs in 2026](https://www.extend.ai/resources/best-document-parsing-apis-2026)
- [Choosing a Document Parser for AI Agents](https://www.extend.ai/resources/choosing-document-parser-ai-agents)
- [Open Source Doc Parsing vs. Production APIs: August 2026](https://www.extend.ai/resources/document-parsing-open-source-vs-api)
- [6 Best PDF Parsing APIs for AI Agents (August 2026)](https://www.extend.ai/resources/pdf-parsing-api-comparison-ai-agents)
- [Top PDF Parsing APIs for Complex Documents (September, 2026)](https://www.extend.ai/resources/pdf-parsing-apis-complex-documents)