Paperboy Current Process

This document describes the end-to-end journey a PDF takes through Paperboy, from upload to final data export.

Pipeline Overview

┌─────────────┐     ┌──────────────────────┐     ┌────────────────────┐     ┌────────────────────┐     ┌───────────────┐
│   1. Upload │───-▶│ 2. Classification    │────▶│    3. Splitting    │─-──▶│   4. Extraction    │────▶│   5. Export   │
│             │     │                      │     │                    │     │                    │     │               │
│  PDF → Blob │     │ → classification.json│     │ → child PDFs       │     │ → extraction.json  │     │ → CSV         │
│  status:    │     │   status: classified │     │   status: classified     │   status: extracted│     │               │
│  uploaded   │     │                      │     │   (per child)      │     │                    │     │               │
└─────────────┘     └──────────────────────┘     └────────────────────┘     └────────────────────┘     └───────────────┘

All artifacts are stored in Azure Blob Storage under the path:

/{userEmail}/{clientName}/{step}/

Step 1 — Upload

Trigger: User uploads a PDF via the UI or API.

Endpoints:

• POST /api/v1/files/upload — single file
• POST /api/v1/files/upload-multiple — batch upload

What happens:

1. The file is received as a multipart form upload and its MIME type is validated (PDF only).
2. The PDF buffer is uploaded to Azure Blob Storage at:

{clientName}/files/{timestamp}-{filename}.pdf

3. A new FileData record is inserted into the database with:

status = 'uploaded'
blob_path = <blob path above>

4. If an Agentic workflow was previously paused waiting for this upload (status = 'awaiting_upload'), it is automatically resumed.

FileData record at this point:

{
  "id": "<guid>",
  "original_file_name": "tax2024.pdf",
  "blob_path": "user@email.com/clientName/files/1711234567890-tax2024.pdf",
  "status": "uploaded",
  "classification_json_path": null,
  "extraction_json_path": null,
  "parent_file_id": null
}

Step 2 — Classification

Trigger:

• Direct API call: POST /api/v1/files/:id/classify
• Agentic chat: the classify_file tool is invoked by the Agentic Loop when the LLM decides to classify

What happens:

2a. Text Extraction

The full PDF is downloaded from Blob Storage and all text is extracted page-by-page using pdf-parse via extractPDFText().

2b. Chunked Classification (LLM)

Because documents can be large, the text is processed in 10-page chunks. For each chunk:

1. A prompt is built with the page range (e.g. "Pages 1–10 of 45") and the chunk’s text.
2. The prompt is sent to Azure OpenAI (temperature: 0.3) using the pdf_classification skill loaded from the database.
3. The LLM returns a structured JSON response identifying the form types present in those pages.
4. If parsing fails for a chunk, it is labelled Unidentified Form Data and processing continues.

2c. Merge & Normalise

After all chunks are processed:

• All per-chunk forms[] arrays are concatenated.
• Consecutive forms of the same type are merged into a single entry.
• Any Unidentified Form Data entries are renamed to Extra form data.
• Non-contiguous unidentified page ranges are expressed using a page_ranges[] array instead of page_start / page_end.

2d. Produce classification.json

The merged result is serialised and uploaded to Blob Storage:

{userEmail}/{clientName}/classifications/{name}_classification.json

classification.json schema:

{
  "is_multi_form": true,
  "num_pages": 12,
  "classified_in_chunks": true,
  "num_chunks": 2,
  "forms": [
    {
      "form_type": "Form 1040",
      "page_start": 1,
      "page_end": 4,
      "confidence": 0.92
    },
    {
      "form_type": "Schedule C",
      "page_start": 5,
      "page_end": 8,
      "confidence": 0.88
    },
    {
      "form_type": "Extra form data",
      "page_ranges": [
        { "start": 9, "end": 10 },
        { "start": 11, "end": 12 }
      ],
      "confidence": 0.5
    }
  ]
}

Key fields:

2e. Update Database

The FileData record is updated:

classification_json_path = "…/classifications/{name}_classification.json"
status = 'classified'

Step 3 — Document Splitting

Trigger: Happens automatically at the end of Step 2 when is_multi_form === true OR forms.length > 1.

There is no separate API call — splitting is embedded inside the classification step.

What happens:

3a. Build Page Ranges

The forms[] array from classification.json is walked to build a list of page ranges:

[
  { start: 1, end: 4 },   // Form 1040
  { start: 5, end: 8 },   // Schedule C
  { start: 9, end: 12 }   // Extra form data
]

Both the page_start/page_end and page_ranges[] variants are handled.

3b. Validate Ranges

All page ranges are validated to be within document bounds. Ranges outside the document’s page count are discarded.

3c. Split the PDF

splitPDFByPageRanges() in pdfProcessor.ts uses pdf-lib to create one new PDFDocument per form:

1. For each page range, a new empty PDF is created.
2. The relevant pages are copied from the source document (converting from 1-indexed to 0-indexed internally).
3. The result is serialised to a Buffer.

3d. Upload Child PDFs & Create Child Records

For each split document:

1. The child PDF is uploaded to Blob Storage at:

{userEmail}/{clientName}/files/{baseName}_{sanitizedFormType}.pdf

e.g. tax2024_Form_1040.pdf, tax2024_Schedule_C.pdf

2. A new child FileData record is inserted, linked to the parent:

{
  "id": "<new-guid>",
  "parent_file_id": "<parent-guid>",
  "form_index": 0,
  "form_type": "Form 1040",
  "blob_path": "…/files/tax2024_Form_1040.pdf",
  "status": "classified"
}

After this step the original PDF has been divided into N child PDFs, each covering exactly one form type.

Step 4 — Extraction

Trigger:

• Direct API call: POST /api/v1/files/:id/extract
• Agentic chat: extract_file tool invoked by the Agentic Loop

Scope: Extraction operates on each child file independently. If the file has no children (single-form document), it is extracted directly.

What happens:

4a. Determine Form Type

The form type is resolved in this order:

1. Read file.form_type directly from the FileData row (set during splitting).
2. Fall back to downloading classification.json from Blob and indexing forms[file.form_index].

4b. Select Extraction Skill

The form type string is mapped to a specific LLM skill stored in the Skills database table:

4c. LLM Extraction

1. The child PDF buffer is downloaded from Blob Storage.
2. Text is extracted via extractPDFText().
3. The skill’s system and user prompts are loaded from the DB, with {formType} and {pdfText} substituted.
4. The prompt is sent to Azure OpenAI (temperature: 0.1 for precision).
5. The response is parsed as structured JSON.

4d. Produce extraction.json

The parsed extraction result is uploaded to Blob Storage:

{userEmail}/{clientName}/extractions/{name}_extraction.json

Example extraction.json (Form 1040):

{
  "form_type": "Form 1040",
  "tax_year": "2024",
  "taxpayer_name": "John Smith",
  "ssn": "XXX-XX-1234",
  "filing_status": "Single",
  "total_income": 85000,
  "adjusted_gross_income": 82000,
  "taxable_income": 68000,
  "total_tax": 12500,
  "total_payments": 13000,
  "refund_amount": 500
}

4e. Update Database

extraction_json_path = "…/extractions/{name}_extraction.json"
status = 'extracted'

Step 5 — Export (CSV)

Trigger:

• Direct API call: POST /api/v1/files/:id/export
• Agentic chat: export_file tool

What happens:

1. The extraction.json is downloaded from Blob Storage.
2. The nested JSON object is flattened into key-value rows.
3. A CSV is produced using csv-writer and uploaded to:

{userEmail}/{clientName}/exports/{name}.csv

4. For multi-form documents, each child’s extraction is exported to its own CSV.
5. FileData.export_csv_path is updated.

Agentic Orchestration

All steps above can be triggered individually via REST or driven automatically by the Agentic Loop — a ReAct-style LLM agent.

How it works:

1. A user sends a chat message (e.g. "Process tax2024.pdf for John Smith").
2. The Agentic Loop iterates up to 10 rounds:
   • Asks the LLM what tool to run next.
   • Executes the tool (single or in parallel batches via Promise.allSettled).
   • Feeds the result back to the LLM.
   • Repeats until the LLM signals complete or needs_info.
3. If a required file has not been uploaded yet, the loop pauses (awaiting_upload) and resumes automatically when the file arrives.
4. Each step is recorded in AgentExecutions.execution_steps for audit and replay.

Typical automated sequence:

classify_file  →  extract_file  →  export_file

Blob Storage Layout

{containerName}/
  {userEmail}/
    {clientName}/
      files/
        1711234567890-tax2024.pdf          ← original upload
        tax2024_Form_1040.pdf              ← split child
        tax2024_Schedule_C.pdf             ← split child
        tax2024_Extra_form_data.pdf        ← split child
      classifications/
        tax2024_classification.json        ← produced in Step 2
      extractions/
        tax2024_Form_1040_extraction.json  ← produced in Step 4
        tax2024_Schedule_C_extraction.json
      exports/
        tax2024_Form_1040.csv              ← produced in Step 5
        tax2024_Schedule_C.csv

FileData Status Lifecycle

uploaded  ──▶  classified  ──▶  extracted
                   ▲
                   │
         (child rows also set to 'classified'
          immediately after splitting)

Key Source Files