Commercial & Residential Lease Extractor

Parse

Extract

{
  "name": "Commercial / residential leases Processing Pipeline",
  "steps": [
    {
      "name": "startTrigger1",
      "type": "TRIGGER",
      "next": [
        {
          "step": "parse1"
        }
      ]
    },
    {
      "name": "parse1",
      "type": "PARSE",
      "config": {
        "parseConfig": {
          "blockOptions": {
            "text": {
              "agentic": {
                "enabled": true
              }
            }
          },
          "chunkingStrategy": {
            "type": "document"
          }
        }
      },
      "next": [
        {
          "step": "extraction2"
        }
      ]
    },
    {
      "name": "extraction2",
      "type": "EXTRACT"
    }
  ]
}

# Commercial / Residential Leases Processing — Extend AI Skill

## What this pipeline does

Automatically extracts critical lease agreement data from commercial and residential lease documents by first parsing the full document to markdown, then structurally extracting 17+ key fields (parties, dates, rent terms, insurance requirements, renewal options) into a normalized JSON schema. Handles complex multi-section legal documents with varied clause structures, OCR-scanned originals, and typed handwriting via agentic OCR.

## When to use this

- **Lease portfolio ingestion**: Mass upload 100+ lease agreements and extract normalized records for database import or risk analysis
- **Due diligence & M&A**: Extract rental terms, expiration dates, renewal clauses, and insurance requirements across target properties in minutes
- **Renewal pipeline**: Identify all leases expiring within 12 months by filtering on `lease_end_date` and extract renewal terms automatically
- **Tenant/landlord compliance**: Validate insurance requirements, permitted use, and maintenance responsibilities across a portfolio
- **Workflow automation**: Feed extracted data directly into CRMs, property management systems, or legal review workflows without manual data entry

## Processor pipeline

### Step 1: Parse (parse_performance + agentic OCR)
**Purpose**: Convert lease document to structured markdown for context and citation generation.

**Config**:
- `engine: "parse_performance"` — Extend AI's highest-accuracy parser, tuned for legal documents with complex layouts, multi-column text, signature blocks, and embedded tables
- `blockOptions.text.agentic.enabled: true` — Activates agentic OCR fallback for scanned/handwritten leases; automatically detects low-confidence regions and re-processes with vision models
- `chunkingStrategy.type: "document"` — Keeps entire lease as one chunk, preserving clause relationships and context across pages

**Why**: Leases are legal contracts where clause ordering and cross-references matter. Document-level chunking prevents extraction of terms in isolation. Agentic OCR handles real-world documents (faxes, scans, mixed handwriting) that basic OCR fails on.

### Step 2: Extract (extraction_performance + Zod schema)
**Purpose**: Pull 17 structured fields into strongly-typed JSON, with citations pointing to source text and bounding boxes.

**Config**:
- Schema includes date fields (`lease_start_date`, `lease_end_date`, `signing_date`), currency fields (`base_rent`, `late_fee`, `security_deposit`), text fields (`tenant_name`, `landlord_name`, `permitted_use`), and complex nested objects (`base_rent: { amount, iso_4217_currency_code }`)
- `baseProcessor: "extraction_performance"` — Production-grade extractor with highest accuracy for multi-page legal documents
- Field descriptions are explicit and legally precise (e.g., "Total annual rent due, excluding taxes and additional charges") to drive accuracy

**Why**: Leases contain dollar amounts, dates, and legal terms that require structured extraction. Zod schema enforces type safety and nullable fields where information may be absent (e.g., not all leases have a late fee). Citations allow downstream human review to validate extracted values against source.

---

## TypeScript implementation



---

## CLI equivalent

```bash
# Step 1: Parse lease to markdown
extend parse lease.pdf --config '{
  "blockOptions": {
    "text": { "agentic": { "enabled": true } }
  },
  "chunkingStrategy": { "type": "document" }
}' > lease.md

# Step 2: Extract fields using schema file
extend extract lease.pdf --schema lease-schema.json
```

**lease-schema.json** (Zod schema converted to JSON):
```json
{
  "type": "object",
  "properties": {
    "lease_title": { "type": ["string", "null"], "description": "Title of the lease agreement" },
    "tenant_name": { "type": ["string", "null"], "description": "Full legal name of the tenant" },
    "landlord_name": { "type": ["string", "null"], "description": "Full legal name of the landlord" },
    "premises_address": { "type": ["string", "null"], "description": "Full street address of leased property" },
    "lease_start_date": { "type": ["string", "null"], "description": "Lease start date (ISO yyyy-mm-dd)" },
    "lease_end_date": { "type": ["string", "null"], "description": "Lease end date (ISO yyyy-mm-dd)" },
    "lease_term_months": { "type": ["number", "null"], "description": "Lease term length in months" },
    "base_rent": {
      "type": "object",
      "properties": {
        "amount": { "type": ["number", "null"], "description": "Rent amount" },
        "iso_4217_currency_code": { "type": ["string", "null"], "description": "Currency code (USD, CAD, etc.)" }
      }
    },
    "security_deposit": {
      "type": "object",
      "properties": {
        "amount": { "type": ["number", "null"], "description": "Deposit amount" },
        "iso_4217_currency_code": { "type": ["string", "null"], "description": "Currency code" }
      }
    },
    "permitted_use": { "type": ["string", "null"], "description": "Permitted use of premises (office, retail, residential, etc.)" },
    "insurance_requirements": { "type": ["string", "null"], "description": "Insurance requirements and liability limits" },
    "renewal_option": { "type": ["string", "null"], "description": "Renewal or extension terms and notice requirements" },
    "termination_clause": { "type": ["string", "null"], "description": "Early termination conditions and penalties" },
    "payment_due_date": { "type": ["string", "null"], "description": "When rent is due (e.g., 1st of each month)" },
    "base_rent_frequency": { "type": ["string", "null"], "description": "Frequency of rent payments (monthly, annually, etc.)" },
    "lease_reference_number": { "type": ["string", "null"], "description": "Lease reference or deal ID" },
    "signing_date": { "type": ["string", "null"], "description": "Date lease was signed (ISO yyyy-mm-dd)" },
    "premises_description": { "type": ["string", "null"], "description": "Physical description, improvements, square footage" },
    "late_fee": {
      "type": "object",
      "properties": {
        "amount": { "type": ["number", "null"], "description": "Late fee amount" },
        "iso_4217_currency_code": { "type": ["string", "null"], "description": "Currency code" }
      }
    },
    "tenant_signatory": { "type": ["string", "null"], "description": "Name and title of tenant signatory" },
    "landlord_signatory": { "type": ["string", "null"], "description": "Name and title of landlord signatory" },
    "maintenance_responsibilities": { "type": ["string", "null"], "description": "Maintenance and repair obligations" }
  }
}
```

---

## Schema

The extraction schema captures 17 core lease fields in a single flat object:

```json
{
  "type": "object",
  "properties": {

import fs from "fs";
import { ExtendClient, extendDate, extendCurrency } from "extend-ai";
import { z } from "zod";

const client = new ExtendClient({ token: process.env.EXTEND_API_KEY });

/**
 * Extract key fields from a commercial or residential lease agreement.
 * Pipeline: Parse to markdown → Extract 17 structured fields with citations.
 */
export async function processCommercialresidentialleases(
  filePath: string
): Promise<void> {
  console.log(`\n📄 Processing lease: ${filePath}`);

  // Step 1: Read file and convert to data URL for SDK (local file support)
  const fileBuffer = fs.readFileSync(filePath);
  const dataUrl = `data:application/octet-stream;base64,${fileBuffer.toString("base64")}`;

  // Step 2: Parse to markdown with agentic OCR enabled
  console.log("🔍 Parsing lease document...");
  const parseRun = await client.parseRuns.createAndPoll({
    file: { url: dataUrl },
    config: {
      blockOptions: {
        text: {
          agentic: {
            enabled: true, // Handles scans, handwriting, complex layouts
          },
        },
      },
      chunkingStrategy: {
        type: "document", // Keep full lease as one chunk
      },
    },
  });

  if (parseRun.status !== "PROCESSED") {
    console.error(`❌ Parse failed with status: ${parseRun.status}`);
    return;
  }

  const markdown = parseRun.output.chunks.map((c) => c.content).join("\n\n");
  console.log(`✅ Parsed ${markdown.length} characters of markdown`);

  // Step 3: Extract structured lease fields
  console.log("📋 Extracting lease fields...");

  const leaseSchema = z.object({
    lease_title: z
      .string()
      .nullable()
      .describe(
        "Title or name of the lease agreement, e.g. 'Commercial Lease Agreement' or 'Residential Lease'"
      ),
    tenant_name: z
      .string()
      .nullable()
      .describe(
        "Full legal name of the tenant (lessee), individual or business entity"
      ),
    landlord_name: z
      .string()
      .nullable()
      .describe(
        "Full legal name of the landlord (lessor), individual or business entity"
      ),
    lease_reference_number: z
      .string()
      .nullable()
      .describe(
        "Lease reference number, agreement number, or deal ID assigned by landlord or broker"
      ),
    premises_address: z
      .string()
      .nullable()
      .describe(
        "Full street address, city, state, ZIP of the leased property, e.g. '123 Main St, San Francisco, CA 94105'"
      ),
    premises_description: z
      .string()
      .nullable()
      .describe(
        "Description of improvements, fixtures, or physical condition of premises, e.g. 'office space, newly painted, 5,000 sq ft'"
      ),
    lease_start_date: extendDate().describe(
      "Lease commencement date (ISO yyyy-mm-dd)"
    ),
    lease_end_date: extendDate().describe(
      "Lease expiration date (ISO yyyy-mm-dd)"
    ),
    lease_term_months: z
      .number()
      .nullable()
      .describe("Primary lease term length in months, e.g. 12, 24, 36"),
    signing_date: extendDate().describe(
      "Date both parties signed the lease (ISO yyyy-mm-dd)"
    ),
    permitted_use: z
      .string()
      .nullable()
      .describe(
        "Permitted use of premises, e.g. 'office space', 'retail', 'residential apartment'"
      ),
    base_rent: extendCurrency().describe(
      "Base annual or monthly rent amount with currency code"
    ),
    base_rent_frequency: z
      .string()
      .nullable()
      .describe(
        "Frequency of rent payments, e.g. 'monthly', 'quarterly', 'annually'"
      ),
    payment_due_date: z
      .string()
      .nullable()
      .describe(
        "Due date for rent payment, e.g. 'the 1st of each month' or 'March 1 of each year'"
      ),
    late_fee: extendCurrency().describe(
      "Late fee or penalty amount if rent is overdue, with currency code"
    ),
    security_deposit: extendCurrency().describe(
      "Security deposit amount held by landlord, with currency code"
    ),
    renewal_option: z
      .string()
      .nullable()
      .describe(
        "Description of renewal or extension option, including notice requirements and new term length"
      ),
    termination_clause: z
      .string()
      .nullable()
      .describe(
        "Early termination conditions, including notice period and any penalties or fees"
      ),
    insurance_requirements: z
      .string()
      .nullable()
      .describe(
        "Insurance requirements for tenant and/or landlord, including coverage types, limits, and notice periods for cancellation"
      ),
    maintenance_responsibilities: z
      .string()
      .nullable()
      .describe(
        "Description of repairs, maintenance, and upkeep responsibilities, e.g. who pays for repairs, utilities, property taxes"
      ),
    tenant_signatory: z
      .string()
      .nullable()
      .describe("Name and title of person who signed lease on behalf of tenant"),
    landlord_signatory: z
      .string()
      .nullable()
      .describe(
        "Name and title of person who signed lease on behalf of landlord"
      ),
  });

  const extractRun = await client.extractRuns.createAndPoll({
    file: { url: dataUrl },
    config: {
      schema: leaseSchema,
    },
  });

  if (extractRun.status !== "PROCESSED") {
    console.error(`❌ Extraction failed with status: ${extractRun.status}`);
    return;
  }

  const extractedData = extractRun.output.value;

  // Step 4: Display results
  console.log("\n✅ Extraction complete!\n");
  console.log("=== EXTRACTED LEASE DATA ===\n");
  console.log(`📌 Title: ${extractedData.lease_title || "(not found)"}`);
  console.log(`🏢 Landlord: ${extractedData.landlord_name || "(not found)"}`);
  console.log(`👤 Tenant: ${extractedData.tenant_name || "(not found)"}`);
  console.log(`📍 Address: ${extractedData.premises_address || "(not found)"}`);
  console.log(
    `📅 Start: ${extractedData.lease_start_date || "(not found)"} | End: ${extractedData.lease_end_date || "(not found)"}`
  );
  console.log(
    `⏱️  Term: ${extractedData.lease_term_months ? `${extractedData.lease_term_months} months` : "(not found)"}`
  );
  console.log(
    `💰 Base Rent: ${extractedData.base_rent.amount ? `${extractedData.base_rent.amount} ${extractedData.base_rent.iso_4217_currency_code || "USD"}` : "(not found)"} / ${extractedData.base_rent_frequency || "?"}`
  );
  console.log(
    `🔐 Security Deposit: ${extractedData.security_deposit.amount ? `${extractedData.security_deposit.amount} ${extractedData.security_deposit.iso_4217_currency_code || "USD"}` : "(not found)"}`
  );
  console.log(
    `⚖️  Permitted Use: ${extractedData.permitted_use || "(not found)"}`
  );
  console.log(
    `🔄 Renewal: ${extractedData.renewal_option ? extractedData.renewal_option.substring(0, 80) + "..." : "(not found)"}`
  );
  console.log(
    `🛡️  Insurance: ${extractedData.insurance_requirements ? extractedData.insurance_requirements.substring(0, 80) + "..." : "(not found)"}`
  );

  // Step 5: Output full JSON for downstream processing
  console.log("\n=== FULL EXTRACTED JSON ===\n");
  console.log(JSON.stringify(extractedData, null, 2));
}

// Invoke with command-line file argument
const filePath = process.argv[2];
if (!filePath) {
  console.error("Usage: npx ts-node solution.ts <path-to-lease.pdf>");
  process.exit(1);
}
processCommercialresidentialleases(filePath).catch((err) => {
  console.error("Pipeline error:", err);
  process.exit(1);
});

import os
import sys
import base64
from extend_ai import Extend

client = Extend(token=os.environ["EXTEND_API_KEY"])


def process_commercial_residential_leases(file_path: str) -> None:
    """
    Extract key fields from a commercial or residential lease agreement.
    Pipeline: Parse to markdown → Extract 17 structured fields with citations.
    """
    print(f"\n📄 Processing lease: {file_path}")

    # Step 1: Read file and convert to data URL for SDK (local file support)
    with open(file_path, "rb") as f:
        file_buffer = f.read()
    data_url = f"data:application/octet-stream;base64,{base64.b64encode(file_buffer).decode('utf-8')}"

    # Step 2: Parse to markdown with agentic OCR enabled
    print("🔍 Parsing lease document...")
    parse_run = client.parse_runs.create_and_poll(
        file={"url": data_url},
        config={
            "block_options": {
                "text": {
                    "agentic": {
                        "enabled": True,  # Handles scans, handwriting, complex layouts
                    },
                },
            },
            "chunking_strategy": {
                "type": "document",  # Keep full lease as one chunk
            },
        },
    )

    if parse_run.status != "PROCESSED":
        print(f"❌ Parse failed with status: {parse_run.status}")
        return

    markdown = "\n\n".join([c["content"] for c in parse_run.output["chunks"]])
    print(f"✅ Parsed {len(markdown)} characters of markdown")

    # Step 3: Extract structured lease fields
    print("📋 Extracting lease fields...")

    lease_schema = {
        "type": "object",
        "properties": {
            "lease_type": {
                "type": ["string", "null"],
                "description": "Type of lease (Commercial or Residential)",
            },
            "lessor_name": {
                "type": ["string", "null"],
                "description": "Full legal name of the lessor/landlord",
            },
            "lessee_name": {
                "type": ["string", "null"],
                "description": "Full legal name of the lessee/tenant",
            },
            "execution_date": {
                "type": ["string", "null"],
                "description": "Date the lease agreement was executed (YYYY-MM-DD format)",
            },
            "property_address": {
                "type": ["string", "null"],
                "description": "Full address of the leased property",
            },
            "property_improvements": {
                "type": "array",
                "description": "List of improvements or modifications to the property",
                "items": {"type": "string"},
            },
            "lease_term_months": {
                "type": ["string", "null"],
                "description": "Primary lease term duration in months",
            },
            "commencement_date": {
                "type": ["string", "null"],
                "description": "Date lease begins (YYYY-MM-DD format)",
            },
            "expiration_date": {
                "type": ["string", "null"],
                "description": "Date lease expires (YYYY-MM-DD format)",
            },
            "annual_rent": {
                "type": ["string", "null"],
                "description": "Annual rental amount in dollars",
            },
            "property_use": {
                "type": ["string", "null"],
                "description": "Permitted use of the leased property",
            },
            "renewal_term_years": {
                "type": ["string", "null"],
                "description": "Optional renewal term duration in years",
            },
        },
    }

    extract_run = client.extract_runs.create_and_poll(
        file={"url": data_url},
        config={"schema": lease_schema},
    )

    if extract_run.status != "PROCESSED":
        print(f"❌ Extraction failed with status: {extract_run.status}")
        return

    extracted_data = extract_run.output["value"]

    # Step 4: Display results
    print("\n✅ Extraction complete!\n")
    print("=== EXTRACTED LEASE DATA ===\n")
    print(f"📌 Type: {extracted_data.get('lease_type') or '(not found)'}")
    print(f"🏢 Lessor: {extracted_data.get('lessor_name') or '(not found)'}")
    print(f"👤 Lessee: {extracted_data.get('lessee_name') or '(not found)'}")
    print(f"📍 Address: {extracted_data.get('property_address') or '(not found)'}")
    print(
        f"📅 Commencement: {extracted_data.get('commencement_date') or '(not found)'} | Expiration: {extracted_data.get('expiration_date') or '(not found)'}"
    )
    print(
        f"⏱️  Term: {extracted_data.get('lease_term_months') or '(not found)'} months"
    )
    print(f"💰 Annual Rent: {extracted_data.get('annual_rent') or '(not found)'}")
    print(
        f"⚖️  Property Use: {extracted_data.get('property_use') or '(not found)'}"
    )
    print(
        f"🔄 Renewal Term: {extracted_data.get('renewal_term_years') or '(not found)'} years"
    )
    improvements = extracted_data.get("property_improvements") or []
    print(
        f"🏗️  Improvements: {', '.join(improvements) if improvements else '(not found)'}"
    )

    # Step 5: Output full JSON for downstream processing
    print("\n=== FULL EXTRACTED JSON ===\n")
    import json

    print(json.dumps(extracted_data, indent=2))


# Invoke with command-line file argument
if __name__ == "__main__":
    if len(sys.argv) < 2:
        print("Usage: python solution.py <path-to-lease.pdf>")
        sys.exit(1)

    file_path = sys.argv[1]

    try:
        process_commercial_residential_leases(file_path)
    except Exception as err:
        print(f"Pipeline error: {err}")
        sys.exit(1)

import java.io.IOException;
import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.file.Files;
import java.nio.file.Paths;
import java.util.*;
import java.util.stream.Collectors;

/**
 * Extend AI REST API client in pure Java (no official SDK exists yet).
 * This code calls https://api.extend.ai directly using java.net.http.HttpClient
 * with only standard library imports for zero external dependencies.
 */
public class CommercialResidentialLeaseProcessor {

    private static final String API_BASE = "https://api.extend.ai";
    private static final String API_KEY = System.getenv("EXTEND_API_KEY");
    private final HttpClient httpClient;

    public CommercialResidentialLeaseProcessor() {
        this.httpClient = HttpClient.newHttpClient();
    }

    /**
     * Extract key fields from a commercial or residential lease agreement.
     * Pipeline: Parse to markdown → Extract 17 structured fields with citations.
     */
    public void processCommercialResidentialLeases(String filePath) throws IOException, InterruptedException {
        System.out.println("\n📄 Processing lease: " + filePath);

        // Step 1: Read file and convert to data URL for API
        byte[] fileBuffer = Files.readAllBytes(Paths.get(filePath));
        String base64 = Base64.getEncoder().encodeToString(fileBuffer);
        String dataUrl = "data:application/octet-stream;base64," + base64;

        // Step 2: Parse to markdown with agentic OCR enabled
        System.out.println("🔍 Parsing lease document...");
        Map<String, Object> parseRunData = createAndPollParseRun(dataUrl);

        if (!"PROCESSED".equals(parseRunData.get("status"))) {
            System.err.println("❌ Parse failed with status: " + parseRunData.get("status"));
            return;
        }

        String markdown = extractMarkdownFromParseRun(parseRunData);
        System.out.println("✅ Parsed " + markdown.length() + " characters of markdown");

        // Step 3: Extract structured lease fields
        System.out.println("📋 Extracting lease fields...");

        Map<String, Object> extractRunData = createAndPollExtractRun(dataUrl);

        if (!"PROCESSED".equals(extractRunData.get("status"))) {
            System.err.println("❌ Extraction failed with status: " + extractRunData.get("status"));
            return;
        }

        @SuppressWarnings("unchecked")
        Map<String, Object> extractedData = (Map<String, Object>) extractRunData.get("output");

        // Step 4: Display results
        System.out.println("\n✅ Extraction complete!\n");
        System.out.println("=== EXTRACTED LEASE DATA ===\n");
        System.out.println("📌 Title: " + getOrDefault(extractedData, "lease_type", "(not found)"));
        System.out.println("🏢 Landlord: " + getOrDefault(extractedData, "lessor_name", "(not found)"));
        System.out.println("👤 Tenant: " + getOrDefault(extractedData, "lessee_name", "(not found)"));
        System.out.println("📍 Address: " + getOrDefault(extractedData, "property_address", "(not found)"));
        System.out.println("📅 Start: " + getOrDefault(extractedData, "commencement_date", "(not found)")
                + " | End: " + getOrDefault(extractedData, "expiration_date", "(not found)"));
        System.out.println("⏱️  Term: " + getOrDefault(extractedData, "lease_term_months", "(not found)") + " months");
        System.out.println("💰 Annual Rent: " + getOrDefault(extractedData, "annual_rent", "(not found)"));
        System.out.println("⚖️  Permitted Use: " + getOrDefault(extractedData, "property_use", "(not found)"));
        System.out.println("🔄 Renewal: " + truncate(getOrDefault(extractedData, "renewal_term_years", "(not found)"), 80));

        // Step 5: Output full JSON for downstream processing
        System.out.println("\n=== FULL EXTRACTED JSON ===\n");
        System.out.println(toJsonString(extractedData));
    }

    /**
     * Create and poll a parse run via REST API.
     */
    private Map<String, Object> createAndPollParseRun(String dataUrl) throws IOException, InterruptedException {
        String payload = buildParseRunPayload(dataUrl);
        String runId = null;

        // Create parse run
        HttpRequest request = HttpRequest.newBuilder()
                .uri(URI.create(API_BASE + "/api/v1/parse-runs"))
                .header("Authorization", "Bearer " + API_KEY)
                .header("Content-Type", "application/json")
                .POST(HttpRequest.BodyPublishers.ofString(payload))
                .build();

        HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());
        Map<String, Object> parseResponse = parseJsonObject(response.body());
        runId = (String) parseResponse.get("id");

        // Poll until completion
        return pollRunCompletion("/api/v1/parse-runs/" + runId);
    }

    /**
     * Create and poll an extract run via REST API.
     */
    private Map<String, Object> createAndPollExtractRun(String dataUrl) throws IOException, InterruptedException {
        String payload = buildExtractRunPayload(dataUrl);

        HttpRequest request = HttpRequest.newBuilder()
                .uri(URI.create(API_BASE + "/api/v1/extract-runs"))
                .header("Authorization", "Bearer " + API_KEY)
                .header("Content-Type", "application/json")
                .POST(HttpRequest.BodyPublishers.ofString(payload))
                .build();

        HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());
        Map<String, Object> extractResponse = parseJsonObject(response.body());
        String runId = (String) extractResponse.get("id");

        return pollRunCompletion("/api/v1/extract-runs/" + runId);
    }

    /**
     * Poll run completion endpoint until status is PROCESSED or failed.
     */
    private Map<String, Object> pollRunCompletion(String endpoint) throws IOException, InterruptedException {
        long maxWaitMs = 300000; // 5 minutes
        long startTime = System.currentTimeMillis();

        while (System.currentTimeMillis() - startTime < maxWaitMs) {
            HttpRequest request = HttpRequest.newBuilder()
                    .uri(URI.create(API_BASE + endpoint))
                    .header("Authorization", "Bearer " + API_KEY)
                    .GET()
                    .build();

            HttpResponse<String> response = httpClient.send(request, HttpResponse.BodyHandlers.ofString());
            Map<String, Object> runData = parseJsonObject(response.body());
            String status = (String) runData.get("status");

            if ("PROCESSED".equals(status)) {
                return runData;
            }

            Thread.sleep(2000);
        }

        Map<String, Object> timeoutResult = new HashMap<>();
        timeoutResult.put("status", "TIMEOUT");
        return timeoutResult;
    }

    /**
     * Build parse run request payload.
     */
    private String buildParseRunPayload(String dataUrl) {
        return "{"
                + "\"file\":{\"url\":\"" + escapeJson(dataUrl) + "\"},"
                + "\"config\":{"
                + "\"blockOptions\":{"
                + "\"text\":{"
                + "\"agentic\":{\"enabled\":true}"
                + "}"
                + "},"
                + "\"chunkingStrategy\":{\"type\":\"document\"}"
                + "}"
                + "}";
    }

    /**
     * Build extract run request payload with lease schema.
     */
    private String buildExtractRunPayload(String dataUrl) {
        String schema = buildLeaseSchema();
        return "{"
                + "\"file\":{\"url\":\"" + escapeJson(dataUrl) + "\"},"
                + "\"config\":{"
                + "\"schema\":" + schema
                + "}"
                + "}";
    }

    /**
     * Build the lease extraction schema JSON.
     */
    private String buildLeaseSchema() {
        return "{"
                + "\"type\":\"object\","
                + "\"properties\":{"
                + "\"lease_type\":{\"type\":[\"string\",\"null\"],\"description\":\"Type of lease (Commercial or Residential)\"},"
                + "\"lessor_name\":{\"type\":[\"string\",\"null\"],\"description\":\"Full legal name of the lessor/landlord\"},"
                + "\"lessee_name\":{\"type\":[\"string\",\"null\"],\"description\":\"Full legal name of the lessee/tenant\"},"
                + "\"execution_date\":{\"type\":[\"string\",\"null\"],\"description\":\"Date the lease agreement was executed (YYYY-MM-DD format)\"},"
                + "\"property_address\":{\"type\":[\"string\",\"null\"],\"description\":\"Full address of the leased property\"},"
                + "\"property_improvements\":{\"type\":\"array\",\"description\":\"List of improvements or modifications to the property\",\"items\":{\"type\":\"string\"}},"
                + "\"lease_term_months\":{\"type\":[\"string\",\"null\"],\"description\":\"Primary lease term duration in months\"},"
                + "\"commencement_date\":{\"type\":[\"string\",\"null\"],\"description\":\"Date lease begins (YYYY-MM-DD format)\"},"
                + "\"expiration_date\":{\"type\":[\"string\",\"null\"],\"description\":\"Date lease expires (YYYY-MM-DD format)\"},"
                + "\"annual_rent\":{\"type\":[\"string\",\"null\"],\"description\":\"Annual rental amount in dollars\"},"
                + "\"property_use\":{\"type\":[\"string\",\"null\"],\"description\":\"Permitted use of the leased property\"},"
                + "\"renewal_term_years\":{\"type\":[\"string\",\"null\"],\"description\":\"Optional renewal term duration in years\"}"
                + "}"
                + "}";
    }

    /**
     * Extract markdown from parse run output.
     */
    private String extractMarkdownFromParseRun(Map<String, Object> parseRun) {
        @SuppressWarnings("unchecked")
        Map<String, Object> output = (Map<String, Object>) parseRun.get("output");
        if (output == null) return "";

        @SuppressWarnings("unchecked")
        List<Map<String, Object>> chunks = (List<Map<String, Object>>) output.get("chunks");
        if (chunks == null) return "";

        return chunks.stream()
                .map(c -> (String) c.get("content"))
                .filter(Objects::nonNull)
                .collect(Collectors.joining("\n\n"));
    }

    /**
     * Simple JSON object parser (sufficient for this use case).
     */
    @SuppressWarnings("unchecked")
    private Map<String, Object> parseJsonObject(String json) {
        Map<String, Object> result = new LinkedHashMap<>();
        json = json.trim();
        if (!json.startsWith("{") || !json.endsWith("}")) {
            return result;
        }

        String content = json.substring(1, json.length() - 1).trim();
        int depth = 0;
        int start = 0;
        String currentKey = null;

        for (int i = 0; i < content.length(); i++) {
            char c = content.charAt(i);

            if ((c == '{' || c == '[') && depth == 0 && currentKey != null) {
                depth++;
            } else if ((c == '}' || c == ']') && depth > 0) {
                depth--;
            }

            if (c == ':' && depth == 0 && currentKey == null) {
                currentKey = content.substring(start, i).trim();
                currentKey = unquote(currentKey);
                start = i + 1;
            } else if ((c == ',' || i == content.length() - 1) && depth == 0) {
                if (currentKey != null) {
                    int end = (c == ',') ? i : i + 1;
                    String valueStr = content.substring(start, end).trim();
                    result.put(currentKey, parseValue(valueStr));
                    currentKey = null;
                    start = i + 1;
                }
            }
        }

        return result;
    }

    /**
     * Parse a JSON value string.
     */
    @SuppressWarnings("unchecked")
    private Object parseValue(String value) {
        value = value.trim();
        if (value.equals("null")) {
            return null;
        }
        if (value.equals("true")) {
            return true;
        }
        if (value.equals("false")) {
            return false;
        }
        if (value.startsWith("\"") && value.endsWith("\"")) {
            return unquote(value);
        }
        if (value.startsWith("[") && value.endsWith("]")) {
            List<Object> list = new ArrayList<>();
            String content = value.substring(1, value.length() - 1).trim();
            if (!content.isEmpty()) {
                for (String item : content.split(",")) {
                    list.add(parseValue(item.trim()));
                }
            }
            return list;
        }
        if (value.startsWith("{")) {
            return parseJsonObject(value);
        }
        try {
            if (value.contains(".")) {
                return Double.parseDouble(value);
            }
            return Long.parseLong(value);
        } catch (NumberFormatException e) {
            return value;
        }
    }

    /**
     * Remove surrounding quotes from a JSON string.
     */
    private String unquote(String s) {
        if (s.startsWith("\"") && s.endsWith("\"") && s.length() >= 2) {
            return s.substring(1, s.length() - 1);
        }
        return s;
    }

    /**
     * Escape a string for JSON.
     */
    private String escapeJson(String s) {
        return s.replace("\\", "\\\\")
                .replace("\"", "\\\"")
                .replace("\n", "\\n")
                .replace("\r", "\\r")
                .replace("\t", "\\t");
    }

    /**
     * Get value from map with default.
     */
    private String getOrDefault(Map<String, Object> map, String key, String defaultValue) {
        Object value = map.get(key);
        return value != null ? value.toString() : defaultValue;
    }

    /**
     * Truncate string to max length with ellipsis.
     */
    private String truncate(String s, int maxLength) {
        if (s.length() > maxLength) {
            return s.substring(0, maxLength) + "...";
        }
        return s;
    }

    /**
     * Convert map to simple JSON string.
     */
    @SuppressWarnings("unchecked")
    private String toJsonString(Map<String, Object> map) {
        StringBuilder sb = new StringBuilder("{");
        boolean first = true;
        for (Map.Entry<String, Object> entry : map.entrySet()) {
            if (!first) sb.append(",");
            sb.append("\"").append(entry.getKey()).append("\":");
            Object value = entry.getValue();
            if (value == null) {
                sb.append("null");
            } else if (value instanceof String) {
                sb.append("\"").append(escapeJson((String) value)).append("\"");
            } else if (value instanceof Map) {
                sb.append(toJsonString((Map<String, Object>) value));
            } else if (value instanceof List) {
                List<?> list = (List<?>) value;
                sb.append("[");
                for (int i = 0; i < list.size(); i++) {
                    if (i > 0) sb.append(",");
                    Object item = list.get(i);
                    if (item instanceof String) {
                        sb.append("\"").append(escapeJson((String) item)).append("\"");
                    } else {
                        sb.append(item);
                    }
                }
                sb.append("]");
            } else {
                sb.append(value);
            }
            first = false;
        }
        sb.append("}");
        return sb.toString();
    }

    public static void main(String[] args) {
        if (args.length < 1) {
            System.err.println("Usage: java CommercialResidentialLeaseProcessor <path-to-lease.pdf>");
            System.exit(1);
        }

        try {
            CommercialResidentialLeaseProcessor processor = new CommercialResidentialLeaseProcessor();
            processor.processCommercialResidentialLeases(args[0]);
        } catch (IOException | InterruptedException e) {
            System.err.println("Pipeline error: " + e.getMessage());
            e.printStackTrace();
            System.exit(1);
        }
    }
}

// This Go code uses the Extend REST API directly because Extend has no official Go SDK.
// It mirrors the same endpoints and request/response shapes as the TypeScript SDK.

package main

import (
	"bytes"
	"encoding/base64"
	"encoding/json"
	"flag"
	"fmt"
	"io/ioutil"
	"log"
	"net/http"
	"os"
	"time"
)

const extendAPIBase = "https://api.extend.ai"

type ParseConfig struct {
	BlockOptions struct {
		Text struct {
			Agentic struct {
				Enabled bool `json:"enabled"`
			} `json:"agentic"`
		} `json:"text"`
	} `json:"blockOptions"`
	ChunkingStrategy struct {
		Type string `json:"type"`
	} `json:"chunkingStrategy"`
}

type ParseRequest struct {
	File struct {
		URL string `json:"url"`
	} `json:"file"`
	Config ParseConfig `json:"config"`
}

type ParseChunk struct {
	Content string `json:"content"`
}

type ParseOutput struct {
	Chunks []ParseChunk `json:"chunks"`
}

type ParseRun struct {
	Status string      `json:"status"`
	Output ParseOutput `json:"output"`
}

type ExtractSchema struct {
	LeaseType           interface{} `json:"lease_type,omitempty"`
	LessorName          interface{} `json:"lessor_name,omitempty"`
	LesseeName          interface{} `json:"lessee_name,omitempty"`
	ExecutionDate       interface{} `json:"execution_date,omitempty"`
	PropertyAddress     interface{} `json:"property_address,omitempty"`
	PropertyImprovements interface{} `json:"property_improvements,omitempty"`
	LeaseTermMonths     interface{} `json:"lease_term_months,omitempty"`
	CommencementDate    interface{} `json:"commencement_date,omitempty"`
	ExpirationDate      interface{} `json:"expiration_date,omitempty"`
	AnnualRent          interface{} `json:"annual_rent,omitempty"`
	PropertyUse         interface{} `json:"property_use,omitempty"`
	RenewalTermYears    interface{} `json:"renewal_term_years,omitempty"`
}

type ExtractConfig struct {
	Schema map[string]interface{} `json:"schema"`
}

type ExtractRequest struct {
	File struct {
		URL string `json:"url"`
	} `json:"file"`
	Config ExtractConfig `json:"config"`
}

type ExtractOutput struct {
	Value map[string]interface{} `json:"value"`
}

type ExtractRun struct {
	Status string       `json:"status"`
	Output ExtractOutput `json:"output"`
}

func makeRequest(method, endpoint string, token string, body interface{}) ([]byte, error) {
	url := extendAPIBase + endpoint
	
	var reqBody []byte
	var err error
	if body != nil {
		reqBody, err = json.Marshal(body)
		if err != nil {
			return nil, err
		}
	}

	req, err := http.NewRequest(method, url, bytes.NewReader(reqBody))
	if err != nil {
		return nil, err
	}

	req.Header.Set("Authorization", fmt.Sprintf("Bearer %s", token))
	req.Header.Set("Content-Type", "application/json")

	client := &http.Client{Timeout: 300 * time.Second}
	resp, err := client.Do(req)
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()

	respBody, err := ioutil.ReadAll(resp.Body)
	if err != nil {
		return nil, err
	}

	if resp.StatusCode < 200 || resp.StatusCode >= 300 {
		return nil, fmt.Errorf("API error: status %d, body: %s", resp.StatusCode, string(respBody))
	}

	return respBody, nil
}

func processLeaseDocument(filePath string, token string) error {
	fmt.Printf("\n📄 Processing lease: %s\n", filePath)

	// Step 1: Read file and convert to data URL
	fileBuffer, err := ioutil.ReadFile(filePath)
	if err != nil {
		return fmt.Errorf("failed to read file: %w", err)
	}
	dataURL := fmt.Sprintf("data:application/octet-stream;base64,%s", base64.StdEncoding.EncodeToString(fileBuffer))

	// Step 2: Parse to markdown with agentic OCR enabled
	fmt.Println("🔍 Parsing lease document...")
	parseReq := ParseRequest{
		Config: ParseConfig{},
	}
	parseReq.File.URL = dataURL
	parseReq.Config.BlockOptions.Text.Agentic.Enabled = true
	parseReq.Config.ChunkingStrategy.Type = "document"

	respBody, err := makeRequest("POST", "/v1/parse-runs/create-and-poll", token, parseReq)
	if err != nil {
		return fmt.Errorf("parse request failed: %w", err)
	}

	var parseRun ParseRun
	if err := json.Unmarshal(respBody, &parseRun); err != nil {
		return fmt.Errorf("failed to unmarshal parse response: %w", err)
	}

	if parseRun.Status != "PROCESSED" {
		return fmt.Errorf("❌ Parse failed with status: %s", parseRun.Status)
	}

	var markdown string
	for _, chunk := range parseRun.Output.Chunks {
		markdown += chunk.Content + "\n\n"
	}
	fmt.Printf("✅ Parsed %d characters of markdown\n", len(markdown))

	// Step 3: Extract structured lease fields
	fmt.Println("📋 Extracting lease fields...")

	schema := map[string]interface{}{
		"type": "object",
		"properties": map[string]interface{}{
			"lease_type": map[string]interface{}{
				"type":        []string{"string", "null"},
				"description": "Type of lease (Commercial or Residential)",
			},
			"lessor_name": map[string]interface{}{
				"type":        []string{"string", "null"},
				"description": "Full legal name of the lessor/landlord",
			},
			"lessee_name": map[string]interface{}{
				"type":        []string{"string", "null"},
				"description": "Full legal name of the lessee/tenant",
			},
			"execution_date": map[string]interface{}{
				"type":        []string{"string", "null"},
				"description": "Date the lease agreement was executed (YYYY-MM-DD format)",
			},
			"property_address": map[string]interface{}{
				"type":        []string{"string", "null"},
				"description": "Full address of the leased property",
			},
			"property_improvements": map[string]interface{}{
				"type":        "array",
				"description": "List of improvements or modifications to the property",
				"items": map[string]interface{}{
					"type": "string",
				},
			},
			"lease_term_months": map[string]interface{}{
				"type":        []string{"string", "null"},
				"description": "Primary lease term duration in months",
			},
			"commencement_date": map[string]interface{}{
				"type":        []string{"string", "null"},
				"description": "Date lease begins (YYYY-MM-DD format)",
			},
			"expiration_date": map[string]interface{}{
				"type":        []string{"string", "null"},
				"description": "Date lease expires (YYYY-MM-DD format)",
			},
			"annual_rent": map[string]interface{}{
				"type":        []string{"string", "null"},
				"description": "Annual rental amount in dollars",
			},
			"property_use": map[string]interface{}{
				"type":        []string{"string", "null"},
				"description": "Permitted use of the leased property",
			},
			"renewal_term_years": map[string]interface{}{
				"type":        []string{"string", "null"},
				"description": "Optional renewal term duration in years",
			},
		},
	}

	extractReq := ExtractRequest{
		Config: ExtractConfig{
			Schema: schema,
		},
	}
	extractReq.File.URL = dataURL

	respBody, err = makeRequest("POST", "/v1/extract-runs/create-and-poll", token, extractReq)
	if err != nil {
		return fmt.Errorf("extract request failed: %w", err)
	}

	var extractRun ExtractRun
	if err := json.Unmarshal(respBody, &extractRun); err != nil {
		return fmt.Errorf("failed to unmarshal extract response: %w", err)
	}

	if extractRun.Status != "PROCESSED" {
		return fmt.Errorf("❌ Extraction failed with status: %s", extractRun.Status)
	}

	extractedData := extractRun.Output.Value

	// Step 4: Display results
	fmt.Println("\n✅ Extraction complete!\n")
	fmt.Println("=== EXTRACTED LEASE DATA ===\n")
	fmt.Printf("📌 Type: %v\n", getOrDefault(extractedData, "lease_type"))
	fmt.Printf("🏢 Lessor: %v\n", getOrDefault(extractedData, "lessor_name"))
	fmt.Printf("👤 Lessee: %v\n", getOrDefault(extractedData, "lessee_name"))
	fmt.Printf("📍 Address: %v\n", getOrDefault(extractedData, "property_address"))
	fmt.Printf("📅 Start: %v | End: %v\n", getOrDefault(extractedData, "commencement_date"), getOrDefault(extractedData, "expiration_date"))
	fmt.Printf("⏱️  Term: %v months\n", getOrDefault(extractedData, "lease_term_months"))
	fmt.Printf("💰 Annual Rent: %v\n", getOrDefault(extractedData, "annual_rent"))
	fmt.Printf("⚖️  Property Use: %v\n", getOrDefault(extractedData, "property_use"))
	fmt.Printf("🔄 Renewal Term: %v years\n", getOrDefault(extractedData, "renewal_term_years"))

	// Step 5: Output full JSON
	fmt.Println("\n=== FULL EXTRACTED JSON ===\n")
	jsonOut, _ := json.MarshalIndent(extractedData, "", "  ")
	fmt.Println(string(jsonOut))

	return nil
}

func getOrDefault(data map[string]interface{}, key string) interface{} {
	if val, ok := data[key]; ok && val != nil {
		return val
	}
	return "(not found)"
}

func main() {
	flag.Parse()
	args := flag.Args()

	if len(args) == 0 {
		fmt.Fprintf(os.Stderr, "Usage: %s <path-to-lease.pdf>\n", os.Args[0])
		os.Exit(1)
	}

	filePath := args[0]
	token := os.Getenv("EXTEND_API_KEY")
	if token == "" {
		log.Fatal("EXTEND_API_KEY environment variable not set")
	}

	if err := processLeaseDocument(filePath, token); err != nil {
		log.Fatalf("Pipeline error: %v", err)
	}
}

// Deploy the "Commercial / residential leases" pipeline to YOUR Extend account.
//
// The workflow below is fully self-contained — every EXTRACT/CLASSIFY/SPLIT
// step carries its extractor/classifier/splitter config INLINE, so this is a
// single API call. No processors to create or wire up beforehand.
// Idempotent: the created workflow id is cached in .extend/commercial-residential-leases.json,
// so re-running updates the existing workflow instead of duplicating it.
//
// Usage:
//   export EXTEND_API_KEY=sk_...   (from https://dashboard.extend.ai → API Keys)
//   npx tsx provision.ts
//
// Generated by doc1 (template: commercial-residential-leases).

import fs from "node:fs";
import path from "node:path";

const API = "https://api.extend.ai";
const VERSION = "2026-02-09";
const API_KEY = process.env.EXTEND_API_KEY;
if (!API_KEY) { console.error("Set EXTEND_API_KEY first."); process.exit(1); }

const STATE_DIR = path.join(process.cwd(), ".extend");
const STATE_FILE = path.join(STATE_DIR, "commercial-residential-leases.json");

type State = { workflowId?: string };
const state: State = fs.existsSync(STATE_FILE)
  ? JSON.parse(fs.readFileSync(STATE_FILE, "utf8"))
  : {};
function saveState() {
  fs.mkdirSync(STATE_DIR, { recursive: true });
  fs.writeFileSync(STATE_FILE, JSON.stringify(state, null, 2));
}

async function api(method: string, pathName: string, body?: unknown) {
  const res = await fetch(API + pathName, {
    method,
    headers: {
      Authorization: `Bearer ${API_KEY}`,
      "x-extend-api-version": VERSION,
      ...(body ? { "Content-Type": "application/json" } : {}),
    },
    body: body ? JSON.stringify(body) : undefined,
  });
  const data = await res.json().catch(() => ({}));
  if (!res.ok) throw new Error(`${method} ${pathName} failed (${res.status}): ${JSON.stringify(data).slice(0, 300)}`);
  return data;
}

// ── Workflow definition — extractor/classifier/splitter configs inline ──────
const WORKFLOW = {
  "name": "Commercial / residential leases Processing Pipeline",
  "steps": [
    {
      "name": "startTrigger1",
      "type": "TRIGGER",
      "next": [
        {
          "step": "parse1"
        }
      ]
    },
    {
      "name": "parse1",
      "type": "PARSE",
      "config": {
        "parseConfig": {
          "blockOptions": {
            "text": {
              "agentic": {
                "enabled": true
              }
            }
          },
          "chunkingStrategy": {
            "type": "document"
          }
        }
      },
      "next": [
        {
          "step": "extraction2"
        }
      ]
    },
    {
      "name": "extraction2",
      "type": "EXTRACT",
      "config": {
        "extractorConfig": {
          "schema": {
            "type": "object",
            "properties": {
              "lease_type": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Type of lease (Commercial or Residential)"
              },
              "lessor_name": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Full legal name of the lessor/landlord"
              },
              "lessee_name": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Full legal name of the lessee/tenant"
              },
              "execution_date": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Date the lease agreement was executed (YYYY-MM-DD format)"
              },
              "property_address": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Full address of the leased property"
              },
              "property_improvements": {
                "type": "array",
                "description": "List of improvements or modifications to the property",
                "items": {
                  "type": "string"
                }
              },
              "lease_term_months": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Primary lease term duration in months"
              },
              "commencement_date": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Date lease begins (YYYY-MM-DD format)"
              },
              "expiration_date": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Date lease expires (YYYY-MM-DD format)"
              },
              "annual_rent": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Annual rental amount in dollars"
              },
              "property_use": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Permitted use of the leased property"
              },
              "renewal_term_years": {
                "type": [
                  "string",
                  "null"
                ],
                "description": "Optional renewal term duration in years"
              }
            }
          }
        }
      }
    }
  ]
};

async function main() {
  console.log(`Deploying "${WORKFLOW.name}"…`);

  if (state.workflowId) {
    console.log(`✓ workflow already provisioned (${state.workflowId}) — updating steps`);
    await api("POST", `/workflows/${state.workflowId}`, { steps: WORKFLOW.steps });
  } else {
    // Reuse an existing workflow with the same name if one exists (e.g. a
    // previous run's state file was lost) instead of creating a duplicate.
    try {
      const list = await api("GET", `/workflows?name=${encodeURIComponent(WORKFLOW.name)}`);
      const items = (list.data ?? list.items ?? []) as Array<{ name?: string; id?: string }>;
      const existing = items.find((x) => x.name === WORKFLOW.name);
      if (existing?.id) {
        state.workflowId = existing.id; saveState();
        console.log(`✓ workflow "${WORKFLOW.name}" found in your account (${existing.id}) — updating steps`);
        await api("POST", `/workflows/${existing.id}`, { steps: WORKFLOW.steps });
      }
    } catch { /* lookup is best-effort; fall through to create */ }

    if (!state.workflowId) {
      const created = await api("POST", "/workflows", WORKFLOW);
      const wfId = created.id ?? created.workflow?.id;
      if (!wfId) throw new Error("Could not read created workflow id from response");
      state.workflowId = wfId; saveState();
      console.log(`+ created workflow (${wfId})`);
    }
  }

  // Deploy the current draft as a new version so the workflow is runnable —
  // best-effort: some accounts/plans may not require this explicit step.
  await api("POST", `/workflows/${state.workflowId}/versions`, {}).catch(() => {});

  console.log("\nDone. Run documents through it with:");
  console.log(`  POST ${API}/workflow_runs  { workflow: { id: "${state.workflowId}" }, file: { url: "https://…" } }`);
  console.log("Or open the workflow in the Extend dashboard to review and deploy it.");
}

main().catch((e) => { console.error(e.message ?? e); process.exit(1); });

import json
import os
from pathlib import Path

from extend_ai import Extend

API_KEY = os.environ.get("EXTEND_API_KEY")
if not API_KEY:
    print("Set EXTEND_API_KEY first.")
    exit(1)

STATE_DIR = Path.cwd() / ".extend"
STATE_FILE = STATE_DIR / "commercial-residential-leases.json"

state: dict = {}
if STATE_FILE.exists():
    state = json.loads(STATE_FILE.read_text())


def save_state():
    STATE_DIR.mkdir(parents=True, exist_ok=True)
    STATE_FILE.write_text(json.dumps(state, indent=2))


WORKFLOW = {
    "name": "Commercial / residential leases Processing Pipeline",
    "steps": [
        {
            "name": "startTrigger1",
            "type": "TRIGGER",
            "next": [{"step": "parse1"}],
        },
        {
            "name": "parse1",
            "type": "PARSE",
            "config": {
                "parseConfig": {
                    "blockOptions": {
                        "text": {"agentic": {"enabled": True}}
                    },
                    "chunkingStrategy": {"type": "document"},
                }
            },
            "next": [{"step": "extraction2"}],
        },
        {
            "name": "extraction2",
            "type": "EXTRACT",
            "config": {
                "extractorConfig": {
                    "schema": {
                        "type": "object",
                        "properties": {
                            "lease_type": {
                                "type": ["string", "null"],
                                "description": "Type of lease (Commercial or Residential)",
                            },
                            "lessor_name": {
                                "type": ["string", "null"],
                                "description": "Full legal name of the lessor/landlord",
                            },
                            "lessee_name": {
                                "type": ["string", "null"],
                                "description": "Full legal name of the lessee/tenant",
                            },
                            "execution_date": {
                                "type": ["string", "null"],
                                "description": "Date the lease agreement was executed (YYYY-MM-DD format)",
                            },
                            "property_address": {
                                "type": ["string", "null"],
                                "description": "Full address of the leased property",
                            },
                            "property_improvements": {
                                "type": "array",
                                "description": "List of improvements or modifications to the property",
                                "items": {"type": "string"},
                            },
                            "lease_term_months": {
                                "type": ["string", "null"],
                                "description": "Primary lease term duration in months",
                            },
                            "commencement_date": {
                                "type": ["string", "null"],
                                "description": "Date lease begins (YYYY-MM-DD format)",
                            },
                            "expiration_date": {
                                "type": ["string", "null"],
                                "description": "Date lease expires (YYYY-MM-DD format)",
                            },
                            "annual_rent": {
                                "type": ["string", "null"],
                                "description": "Annual rental amount in dollars",
                            },
                            "property_use": {
                                "type": ["string", "null"],
                                "description": "Permitted use of the leased property",
                            },
                            "renewal_term_years": {
                                "type": ["string", "null"],
                                "description": "Optional renewal term duration in years",
                            },
                        },
                    }
                }
            },
        },
    ],
}


def main():
    client = Extend(token=API_KEY)
    
    print(f'Deploying "{WORKFLOW["name"]}…"')

    if state.get("workflowId"):
        workflow_id = state["workflowId"]
        print(f"✓ workflow already provisioned ({workflow_id}) — updating steps")
        client.workflows.update(workflow_id, steps=WORKFLOW["steps"])
    else:
        # Reuse an existing workflow with the same name if one exists.
        existing_id = None
        try:
            workflows_list = client.workflows.list(name=WORKFLOW["name"])
            items = workflows_list.data if hasattr(workflows_list, "data") else (workflows_list.items if hasattr(workflows_list, "items") else [])
            for item in items:
                if getattr(item, "name", None) == WORKFLOW["name"]:
                    existing_id = getattr(item, "id", None)
                    break
            
            if existing_id:
                state["workflowId"] = existing_id
                save_state()
                print(
                    f'✓ workflow "{WORKFLOW["name"]}" found in your account ({existing_id}) — updating steps'
                )
                client.workflows.update(existing_id, steps=WORKFLOW["steps"])
        except Exception:
            # lookup is best-effort; fall through to create
            pass

        if not state.get("workflowId"):
            created = client.workflows.create(**WORKFLOW)
            workflow_id = created.id
            if not workflow_id:
                raise ValueError("Could not read created workflow id from response")
            state["workflowId"] = workflow_id
            save_state()
            print(f"+ created workflow ({workflow_id})")

    # Deploy the current draft as a new version so the workflow is runnable —
    # best-effort: some accounts/plans may not require this explicit step.
    try:
        client.workflows.create_version(state["workflowId"])
    except Exception:
        pass

    print("\nDone. Run documents through it with:")
    print(
        f'  POST https://api.extend.ai/workflow_runs  {{ "workflow": {{ "id": "{state["workflowId"]}" }}, "file": {{ "url": "https://…" }} }}'
    )
    print("Or open the workflow in the Extend dashboard to review and deploy it.")


if __name__ == "__main__":
    try:
        main()
    except Exception as e:
        print(str(e))
        exit(1)

// This code uses the Extend REST API directly because Extend has no official Java SDK yet.
// For details on the API, see https://api.extend.ai/docs

import java.io.IOException;
import java.net.URI;
import java.net.URLEncoder;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

public class ProvisionCommercialResidentialLeases {
    private static final String API = "https://api.extend.ai";
    private static final String VERSION = "2026-02-09";
    private static final String API_KEY = System.getenv("EXTEND_API_KEY");
    private static final Path STATE_DIR = Paths.get(System.getProperty("user.dir"), ".extend");
    private static final Path STATE_FILE = STATE_DIR.resolve("commercial-residential-leases.json");
    private static final HttpClient HTTP = HttpClient.newHttpClient();

    static {
        if (API_KEY == null || API_KEY.isEmpty()) {
            System.err.println("Set EXTEND_API_KEY first.");
            System.exit(1);
        }
    }

    static class State {
        public String workflowId;
    }

    private static State state = new State();

    public static void main(String[] args) throws Exception {
        loadState();
        main();
    }

    private static void loadState() throws IOException {
        if (Files.exists(STATE_FILE)) {
            String json = Files.readString(STATE_FILE);
            state = parseState(json);
        }
    }

    private static void saveState() throws IOException {
        Files.createDirectories(STATE_DIR);
        String json = stateToJson(state);
        Files.writeString(STATE_FILE, json);
    }

    private static State parseState(String json) {
        State s = new State();
        if (json.contains("\"workflowId\"")) {
            int start = json.indexOf("\"workflowId\"") + 13;
            int end = json.indexOf("\"", start + 1);
            s.workflowId = json.substring(start + 1, end);
        }
        return s;
    }

    private static String stateToJson(State s) {
        if (s.workflowId == null) {
            return "{}";
        }
        return "{\n  \"workflowId\": \"" + s.workflowId + "\"\n}";
    }

    private static String objectToJson(Object obj) {
        if (obj instanceof String) {
            return "\"" + ((String) obj).replace("\"", "\\\"") + "\"";
        } else if (obj instanceof Map) {
            Map<String, Object> map = (Map<String, Object>) obj;
            StringBuilder sb = new StringBuilder("{");
            boolean first = true;
            for (Map.Entry<String, Object> entry : map.entrySet()) {
                if (!first) sb.append(",");
                sb.append("\"").append(entry.getKey()).append("\":");
                sb.append(objectToJson(entry.getValue()));
                first = false;
            }
            sb.append("}");
            return sb.toString();
        } else if (obj instanceof List) {
            List<Object> list = (List<Object>) obj;
            StringBuilder sb = new StringBuilder("[");
            boolean first = true;
            for (Object item : list) {
                if (!first) sb.append(",");
                sb.append(objectToJson(item));
                first = false;
            }
            sb.append("]");
            return sb.toString();
        }
        return obj == null ? "null" : obj.toString();
    }

    private static Object apiCall(String method, String pathName, Object body) throws Exception {
        HttpRequest.Builder reqBuilder = HttpRequest.newBuilder()
                .uri(new URI(API + pathName))
                .method(method, body != null ? 
                        HttpRequest.BodyPublishers.ofString(objectToJson(body)) : 
                        HttpRequest.BodyPublishers.noBody())
                .header("Authorization", "Bearer " + API_KEY)
                .header("x-extend-api-version", VERSION);

        if (body != null) {
            reqBuilder.header("Content-Type", "application/json");
        }

        HttpRequest req = reqBuilder.build();
        HttpResponse<String> res = HTTP.send(req, HttpResponse.BodyHandlers.ofString());

        if (res.statusCode() < 200 || res.statusCode() >= 300) {
            String errorBody = res.body().length() > 300 ? res.body().substring(0, 300) : res.body();
            throw new Exception(method + " " + pathName + " failed (" + res.statusCode() + "): " + errorBody);
        }

        return parseJson(res.body());
    }

    private static Object parseJson(String json) {
        json = json.trim();
        if (json.startsWith("{")) {
            return parseJsonObject(json);
        } else if (json.startsWith("[")) {
            return parseJsonArray(json);
        }
        return json;
    }

    private static Map<String, Object> parseJsonObject(String json) {
        Map<String, Object> map = new HashMap<>();
        json = json.substring(1, json.length() - 1).trim();
        if (json.isEmpty()) return map;

        int i = 0;
        while (i < json.length()) {
            int keyStart = json.indexOf("\"", i) + 1;
            int keyEnd = json.indexOf("\"", keyStart);
            String key = json.substring(keyStart, keyEnd);
            int colonIdx = json.indexOf(":", keyEnd);
            int valueStart = colonIdx + 1;
            while (valueStart < json.length() && Character.isWhitespace(json.charAt(valueStart))) {
                valueStart++;
            }

            int valueEnd = findJsonValueEnd(json, valueStart);
            String valueStr = json.substring(valueStart, valueEnd).trim();
            map.put(key, parseJson(valueStr));

            i = valueEnd;
            if (i < json.length() && json.charAt(i) == ',') i++;
            while (i < json.length() && Character.isWhitespace(json.charAt(i))) i++;
        }
        return map;
    }

    private static List<Object> parseJsonArray(String json) {
        List<Object> list = new ArrayList<>();
        json = json.substring(1, json.length() - 1).trim();
        if (json.isEmpty()) return list;

        int i = 0;
        while (i < json.length()) {
            while (i < json.length() && Character.isWhitespace(json.charAt(i))) i++;
            int valueEnd = findJsonValueEnd(json, i);
            String valueStr = json.substring(i, valueEnd).trim();
            list.add(parseJson(valueStr));
            i = valueEnd;
            if (i < json.length() && json.charAt(i) == ',') i++;
            while (i < json.length() && Character.isWhitespace(json.charAt(i))) i++;
        }
        return list;
    }

    private static int findJsonValueEnd(String json, int start) {
        char c = json.charAt(start);
        if (c == '"') {
            int i = start + 1;
            while (i < json.length() && json.charAt(i) != '"') {
                if (json.charAt(i) == '\\') i++;
                i++;
            }
            return i + 1;
        } else if (c == '{') {
            int depth = 1, i = start + 1;
            while (i < json.length() && depth > 0) {
                if (json.charAt(i) == '{') depth++;
                else if (json.charAt(i) == '}') depth--;
                i++;
            }
            return i;
        } else if (c == '[') {
            int depth = 1, i = start + 1;
            while (i < json.length() && depth > 0) {
                if (json.charAt(i) == '[') depth++;
                else if (json.charAt(i) == ']') depth--;
                i++;
            }
            return i;
        } else {
            int i = start;
            while (i < json.length() && json.charAt(i) != ',' && json.charAt(i) != '}' && json.charAt(i) != ']') {
                i++;
            }
            return i;
        }
    }

    private static void main() throws Exception {
        String workflowName = "Commercial / residential leases Processing Pipeline";
        System.out.println("Deploying \"" + workflowName + "\"…");

        Map<String, Object> workflow = buildWorkflow(workflowName);

        if (state.workflowId != null && !state.workflowId.isEmpty()) {
            System.out.println("✓ workflow already provisioned (" + state.workflowId + ") — updating steps");
            Map<String, Object> updateBody = new HashMap<>();
            updateBody.put("steps", workflow.get("steps"));
            apiCall("POST", "/workflows/" + state.workflowId, updateBody);
        } else {
            boolean foundExisting = false;
            try {
                String encoded = URLEncoder.encode(workflowName, StandardCharsets.UTF_8);
                Object listResp = apiCall("GET", "/workflows?name=" + encoded, null);
                if (listResp instanceof Map) {
                    Map<String, Object> listMap = (Map<String, Object>) listResp;
                    List<Object> items = (List<Object>) (listMap.getOrDefault("data", listMap.getOrDefault("items", new ArrayList<>())));
                    for (Object item : items) {
                        if (item instanceof Map) {
                            Map<String, Object> itemMap = (Map<String, Object>) item;
                            if (workflowName.equals(itemMap.get("name")) && itemMap.containsKey("id")) {
                                state.workflowId = (String) itemMap.get("id");
                                saveState();
                                System.out.println("✓ workflow \"" + workflowName + "\" found in your account (" + state.workflowId + ") — updating steps");
                                Map<String, Object> updateBody = new HashMap<>();
                                updateBody.put("steps", workflow.get("steps"));
                                apiCall("POST", "/workflows/" + state.workflowId, updateBody);
                                foundExisting = true;
                                break;
                            }
                        }
                    }
                }
            } catch (Exception e) {
                // lookup is best-effort; fall through to create
            }

            if (!foundExisting) {
                Object created = apiCall("POST", "/workflows", workflow);
                String wfId = null;
                if (created instanceof Map) {
                    Map<String, Object> createdMap = (Map<String, Object>) created;
                    wfId = (String) createdMap.getOrDefault("id", null);
                    if (wfId == null && createdMap.containsKey("workflow")) {
                        Map<String, Object> workflowMap = (Map<String, Object>) createdMap.get("workflow");
                        wfId = (String) workflowMap.get("id");
                    }
                }
                if (wfId == null) {
                    throw new Exception("Could not read created workflow id from response");
                }
                state.workflowId = wfId;
                saveState();
                System.out.println("+ created workflow (" + wfId + ")");
            }
        }

        try {
            apiCall("POST", "/workflows/" + state.workflowId + "/versions", new HashMap<>());
        } catch (Exception e) {
            // best-effort: some accounts/plans may not require this explicit step
        }

        System.out.println("\nDone. Run documents through it with:");
        System.out.println("  POST " + API + "/workflow_runs  { workflow: { id: \"" + state.workflowId + "\" }, file: { url: \"https://…\" } }");
        System.out.println("Or open the workflow in the Extend dashboard to review and deploy it.");
    }

    private static Map<String, Object> buildWorkflow(String name) {
        Map<String, Object> workflow = new HashMap<>();
        workflow.put("name", name);
        
        List<Map<String, Object>> steps = new ArrayList<>();
        
        // Step 1: Trigger
        Map<String, Object> trigger = new HashMap<>();
        trigger.put("name", "startTrigger1");
        trigger.put("type", "TRIGGER");
        List<Map<String, Object>> triggerNext = new ArrayList<>();
        Map<String, Object> triggerNextStep = new HashMap<>();
        triggerNextStep.put("step", "parse1");
        triggerNext.add(triggerNextStep);
        trigger.put("next", triggerNext);
        steps.add(trigger);
        
        // Step 2: Parse
        Map<String, Object> parse = new HashMap<>();
        parse.put("name", "parse1");
        parse.put("type", "PARSE");
        Map<String, Object> parseConfig = new HashMap<>();
        Map<String, Object> blockOptions = new HashMap<>();
        Map<String, Object> text = new HashMap<>();
        Map<String, Object> agentic = new HashMap<>();
        agentic.put("enabled", true);
        text.put("agentic", agentic);
        blockOptions.put("text", text);
        Map<String, Object> innerParseConfig = new HashMap<>();
        innerParseConfig.put("blockOptions", blockOptions);
        Map<String, Object> chunkingStrategy = new HashMap<>();
        chunkingStrategy.put("type", "document");
        innerParseConfig.put("chunkingStrategy", chunkingStrategy);
        parseConfig.put("parseConfig", innerParseConfig);
        parse.put("config", parseConfig);
        List<Map<String, Object>> parseNext = new ArrayList<>();
        Map<String, Object> parseNextStep = new HashMap<>();
        parseNextStep.put("step", "extraction2");
        parseNext.add(parseNextStep);
        parse.put("next", parseNext);
        steps.add(parse);
        
        // Step 3: Extract
        Map<String, Object> extract = new HashMap<>();
        extract.put("name", "extraction2");
        extract.put("type", "EXTRACT");
        Map<String, Object> extractConfig = new HashMap<>();
        Map<String, Object> extractorConfig = new HashMap<>();
        Map<String, Object> schema = buildSchema();
        extractorConfig.put("schema", schema);
        extractConfig.put("extractorConfig", extractorConfig);
        extract.put("config", extractConfig);
        steps.add(extract);
        
        workflow.put("steps", steps);
        return workflow;
    }

    private static Map<String, Object> buildSchema() {
        Map<String, Object> schema = new HashMap<>();
        schema.put("type", "object");
        
        Map<String, Object> properties = new HashMap<>();
        
        properties.put("lease_type", buildProperty("string", "Type of lease (Commercial or Residential)"));
        properties.put("lessor_name", buildProperty("string", "Full legal name of the lessor/landlord"));
        properties.put("lessee_name", buildProperty("string", "Full legal name of the lessee/tenant"));
        properties.put("execution_date", buildProperty("string", "Date the lease agreement was executed (YYYY-MM-DD format)"));
        properties.put("property_address", buildProperty("string", "Full address of the leased property"));
        
        Map<String, Object> propertyImprovements = new HashMap<>();
        propertyImprovements.put("type", "array");
        propertyImprovements.put("description", "List of improvements or modifications to the property");
        Map<String, Object> items = new HashMap<>();
        items.put("type", "string");
        propertyImprovements.put("items", items);
        properties.put("property_improvements", propertyImprovements);
        
        properties.put("lease_term_months", buildProperty("string", "Primary lease term duration in months"));
        properties.put("commencement_date", buildProperty("string", "Date lease begins (YYYY-MM-DD format)"));
        properties.put("expiration_date", buildProperty("string", "Date lease expires (YYYY-MM-DD format)"));
        properties.put("annual_rent", buildProperty("string", "Annual rental amount in dollars"));
        properties.put("property_use", buildProperty("string", "Permitted use of the leased property"));
        properties.put("renewal_term_years", buildProperty("string", "Optional renewal term duration in years"));
        
        schema.put("properties", properties);
        return schema;
    }

    private static Map<String, Object> buildProperty(String type, String description) {
        Map<String, Object> prop = new HashMap<>();
        List<String> types = new ArrayList<>();
        types.add(type);
        types.add("null");
        prop.put("type", types);
        prop.put("description", description);
        return prop;
    }
}

// This Go program calls the Extend REST API directly (base URL https://api.extend.ai)
// because Extend has no official Go SDK yet. It uses only the standard library
// (net/http and encoding/json) with zero external dependencies.
//
// Deploy the "Commercial / residential leases" pipeline to YOUR Extend account.
//
// The workflow below is fully self-contained — every EXTRACT/CLASSIFY/SPLIT
// step carries its extractor/classifier/splitter config INLINE, so this is a
// single API call. No processors to create or wire up beforehand.
// Idempotent: the created workflow id is cached in .extend/commercial-residential-leases.json,
// so re-running updates the existing workflow instead of duplicating it.
//
// Usage:
//   export EXTEND_API_KEY=sk_...   (from https://dashboard.extend.ai → API Keys)
//   go run provision.go
//
// Generated by doc1 (template: commercial-residential-leases).

package main

import (
	"bytes"
	"encoding/json"
	"fmt"
	"io"
	"net/http"
	"os"
	"path/filepath"
)

const (
	API     = "https://api.extend.ai"
	VERSION = "2026-02-09"
)

var (
	apiKey  string
	stateDir string
	stateFile string
)

type State struct {
	WorkflowID string `json:"workflowId,omitempty"`
}

type PropertySchema struct {
	Type        string                 `json:"type"`
	Properties  map[string]interface{} `json:"properties"`
	Description string                 `json:"description,omitempty"`
}

type ExtractorConfig struct {
	Schema map[string]interface{} `json:"schema"`
}

type ExtractConfig struct {
	ExtractorConfig ExtractorConfig `json:"extractorConfig"`
}

type ParseConfig struct {
	BlockOptions struct {
		Text struct {
			Agentic struct {
				Enabled bool `json:"enabled"`
			} `json:"agentic"`
		} `json:"text"`
	} `json:"blockOptions"`
	ChunkingStrategy struct {
		Type string `json:"type"`
	} `json:"chunkingStrategy"`
}

type Config struct {
	ParseConfig interface{} `json:"parseConfig,omitempty"`
	ExtractorConfig interface{} `json:"extractorConfig,omitempty"`
}

type NextStep struct {
	Step string `json:"step"`
}

type WorkflowStep struct {
	Name   string      `json:"name"`
	Type   string      `json:"type"`
	Config interface{} `json:"config,omitempty"`
	Next   []NextStep  `json:"next,omitempty"`
}

type Workflow struct {
	Name  string         `json:"name"`
	Steps []WorkflowStep `json:"steps"`
}

type APIResponse struct {
	ID       string        `json:"id,omitempty"`
	Workflow map[string]interface{} `json:"workflow,omitempty"`
	Data     []map[string]interface{} `json:"data,omitempty"`
	Items    []map[string]interface{} `json:"items,omitempty"`
}

type WorkflowItem struct {
	Name string `json:"name"`
	ID   string `json:"id"`
}

var state State

func loadState() error {
	if _, err := os.Stat(stateFile); os.IsNotExist(err) {
		state = State{}
		return nil
	}
	data, err := os.ReadFile(stateFile)
	if err != nil {
		return err
	}
	return json.Unmarshal(data, &state)
}

func saveState() error {
	if err := os.MkdirAll(stateDir, 0755); err != nil {
		return err
	}
	data, err := json.MarshalIndent(state, "", "  ")
	if err != nil {
		return err
	}
	return os.WriteFile(stateFile, data, 0644)
}

func apiCall(method, pathName string, body interface{}) (map[string]interface{}, error) {
	url := API + pathName
	var reqBody io.Reader
	if body != nil {
		jsonData, err := json.Marshal(body)
		if err != nil {
			return nil, err
		}
		reqBody = bytes.NewReader(jsonData)
	}

	req, err := http.NewRequest(method, url, reqBody)
	if err != nil {
		return nil, err
	}

	req.Header.Set("Authorization", "Bearer "+apiKey)
	req.Header.Set("x-extend-api-version", VERSION)
	if body != nil {
		req.Header.Set("Content-Type", "application/json")
	}

	client := &http.Client{}
	resp, err := client.Do(req)
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()

	respBody, err := io.ReadAll(resp.Body)
	if err != nil {
		return nil, err
	}

	var data map[string]interface{}
	json.Unmarshal(respBody, &data)

	if resp.StatusCode < 200 || resp.StatusCode >= 300 {
		errMsg := string(respBody)
		if len(errMsg) > 300 {
			errMsg = errMsg[:300]
		}
		return nil, fmt.Errorf("%s %s failed (%d): %s", method, pathName, resp.StatusCode, errMsg)
	}

	return data, nil
}

func buildWorkflow() Workflow {
	schema := map[string]interface{}{
		"type": "object",
		"properties": map[string]interface{}{
			"lease_type": map[string]interface{}{
				"type":        []string{"string", "null"},
				"description": "Type of lease (Commercial or Residential)",
			},
			"lessor_name": map[string]interface{}{
				"type":        []string{"string", "null"},
				"description": "Full legal name of the lessor/landlord",
			},
			"lessee_name": map[string]interface{}{
				"type":        []string{"string", "null"},
				"description": "Full legal name of the lessee/tenant",
			},
			"execution_date": map[string]interface{}{
				"type":        []string{"string", "null"},
				"description": "Date the lease agreement was executed (YYYY-MM-DD format)",
			},
			"property_address": map[string]interface{}{
				"type":        []string{"string", "null"},
				"description": "Full address of the leased property",
			},
			"property_improvements": map[string]interface{}{
				"type":        "array",
				"description": "List of improvements or modifications to the property",
				"items": map[string]interface{}{
					"type": "string",
				},
			},
			"lease_term_months": map[string]interface{}{
				"type":        []string{"string", "null"},
				"description": "Primary lease term duration in months",
			},
			"commencement_date": map[string]interface{}{
				"type":        []string{"string", "null"},
				"description": "Date lease begins (YYYY-MM-DD format)",
			},
			"expiration_date": map[string]interface{}{
				"type":        []string{"string", "null"},
				"description": "Date lease expires (YYYY-MM-DD format)",
			},
			"annual_rent": map[string]interface{}{
				"type":        []string{"string", "null"},
				"description": "Annual rental amount in dollars",
			},
			"property_use": map[string]interface{}{
				"type":        []string{"string", "null"},
				"description": "Permitted use of the leased property",
			},
			"renewal_term_years": map[string]interface{}{
				"type":        []string{"string", "null"},
				"description": "Optional renewal term duration in years",
			},
		},
	}

	parseConfigObj := map[string]interface{}{
		"blockOptions": map[string]interface{}{
			"text": map[string]interface{}{
				"agentic": map[string]interface{}{
					"enabled": true,
				},
			},
		},
		"chunkingStrategy": map[string]interface{}{
			"type": "document",
		},
	}

	extractConfigObj := map[string]interface{}{
		"extractorConfig": map[string]interface{}{
			"schema": schema,
		},
	}

	return Workflow{
		Name: "Commercial / residential leases Processing Pipeline",
		Steps: []WorkflowStep{
			{
				Name: "startTrigger1",
				Type: "TRIGGER",
				Next: []NextStep{
					{Step: "parse1"},
				},
			},
			{
				Name: "parse1",
				Type: "PARSE",
				Config: map[string]interface{}{
					"parseConfig": parseConfigObj,
				},
				Next: []NextStep{
					{Step: "extraction2"},
				},
			},
			{
				Name: "extraction2",
				Type: "EXTRACT",
				Config: extractConfigObj,
			},
		},
	}
}

func main() {
	var err error
	apiKey = os.Getenv("EXTEND_API_KEY")
	if apiKey == "" {
		fmt.Fprintf(os.Stderr, "Set EXTEND_API_KEY first.\n")
		os.Exit(1)
	}

	wd, err := os.Getwd()
	if err != nil {
		fmt.Fprintf(os.Stderr, "Error getting working directory: %v\n", err)
		os.Exit(1)
	}
	stateDir = filepath.Join(wd, ".extend")
	stateFile = filepath.Join(stateDir, "commercial-residential-leases.json")

	if err := loadState(); err != nil {
		fmt.Fprintf(os.Stderr, "Error loading state: %v\n", err)
		os.Exit(1)
	}

	workflow := buildWorkflow()
	fmt.Printf("Deploying \"%s\"…\n", workflow.Name)

	if state.WorkflowID != "" {
		fmt.Printf("✓ workflow already provisioned (%s) — updating steps\n", state.WorkflowID)
		updateBody := map[string]interface{}{
			"steps": workflow.Steps,
		}
		_, err := apiCall("POST", "/workflows/"+state.WorkflowID, updateBody)
		if err != nil {
			fmt.Fprintf(os.Stderr, "%v\n", err)
			os.Exit(1)
		}
	} else {
		// Try to find an existing workflow with the same name
		listResp, err := apiCall("GET", "/workflows?name="+encodeURLComponent(workflow.Name), nil)
		if err == nil {
			var items []WorkflowItem
			if dataItems, ok := listResp["data"].([]interface{}); ok {
				for _, item := range dataItems {
					if itemMap, ok := item.(map[string]interface{}); ok {
						if name, ok := itemMap["name"].(string); ok && name == workflow.Name {
							if id, ok := itemMap["id"].(string); ok {
								state.WorkflowID = id
								_ = saveState()
								fmt.Printf("✓ workflow \"%s\" found in your account (%s) — updating steps\n", workflow.Name, id)
								updateBody := map[string]interface{}{
									"steps": workflow.Steps,
								}
								_, err := apiCall("POST", "/workflows/"+id, updateBody)
								if err != nil {
									fmt.Fprintf(os.Stderr, "%v\n", err)
									os.Exit(1)
								}
								goto done
							}
						}
					}
				}
			}
			if itemsData, ok := listResp["items"].([]interface{}); ok {
				for _, item := range itemsData {
					if itemMap, ok := item.(map[string]interface{}); ok {
						if name, ok := itemMap["name"].(string); ok && name == workflow.Name {
							if id, ok := itemMap["id"].(string); ok {
								state.WorkflowID = id
								_ = saveState()
								fmt.Printf("✓ workflow \"%s\" found in your account (%s) — updating steps\n", workflow.Name, id)
								updateBody := map[string]interface{}{
									"steps": workflow.Steps,
								}
								_, err := apiCall("POST", "/workflows/"+id, updateBody)
								if err != nil {
									fmt.Fprintf(os.Stderr, "%v\n", err)
									os.Exit(1)
								}
								goto done
							}
						}
					}
				}
			}
		}

		if state.WorkflowID == "" {
			created, err := apiCall("POST", "/workflows", workflow)
			if err != nil {
				fmt.Fprintf(os.Stderr, "%v\n", err)
				os.Exit(1)
			}

			wfID := ""
			if id, ok := created["id"].(string); ok && id != "" {
				wfID = id
			} else if wfData, ok := created["workflow"].(map[string]interface{}); ok {
				if id, ok := wfData["id"].(string); ok {
					wfID = id
				}
			}

			if wfID == "" {
				fmt.Fprintf(os.Stderr, "Could not read created workflow id from response\n")
				os.Exit(1)
			}

			state.WorkflowID = wfID
			if err := saveState(); err != nil {
				fmt.Fprintf(os.Stderr, "Error saving state: %v\n", err)
				os.Exit(1)
			}
			fmt.Printf("+ created workflow (%s)\n", wfID)
		}
	}

done:
	// Deploy the current draft as a new version so the workflow is runnable —
	// best-effort: some accounts/plans may not require this explicit step.
	_, _ = apiCall("POST", "/workflows/"+state.WorkflowID+"/versions", map[string]interface{}{})

	fmt.Printf("\nDone. Run documents through it with:\n")
	fmt.Printf("  POST %s/workflow_runs  { \"workflow\": { \"id\": \"%s\" }, \"file\": { \"url\": \"https://…\" } }\n", API, state.WorkflowID)
	fmt.Printf("Or open the workflow in the Extend dashboard to review and deploy it.\n")
}

func encodeURLComponent(s string) string {
	result := ""
	for _, ch := range s {
		if (ch >= 'A' && ch <= 'Z') || (ch >= 'a' && ch <= 'z') || (ch >= '0' && ch <= '9') || ch == '-' || ch == '_' || ch == '.' || ch == '~' {
			result += string(ch)
		} else {
			result += fmt.Sprintf("%%%02X", ch)
		}
	}
	return result
}