Skip to navigation
Research and Spikes

Classifier Service:-Architecture Design

1. Background & Problem Statement

Applications frequently receive multi-document uploads that contain mixed document types, for example a single PDF containing multiple tax forms, invoices, or supporting records. Before extraction can occur, each logical document segment must be identified before being routed to the appropriate extract. A centralized Classifier service is required to solve this problem in a consistent, extensible way. Today this responsibility sits inside ClassifyFileTool.ts in Paperboy. Form classification and splitting are tightly coupled in a service. This creates several challenges:

  • The classify logic cannot be scaled or replaced independently
  • Incorrect classification results to incorrect processes downstream
ServiceRepositoryResponsibility
fluxx-classifierfluxx-classifier (new repo)Consume a PDF, identify document types and page boundaries, return a ClassificationResult
fluxx-document-splitterfluxx-document-splitter (new repo)Consume a ClassificationResult + original PDF reference, produce discrete split documents stored in accessible blob storage, return a SplitManifest

2. Current State (As-Is) in Paperboy

2.1 Where Classification Lives Today

FileResponsibility
backend/src/api/v1/agent/tools/ClassifyFileTool.tsClassifies AND splits does both in one execute() call
backend/src/utils/pdfProcessor.tssplitPDFByPageRanges() low-level PDF page copy using pdf-lib
backend/src/api/v1/files/model.tsPersists split child file records (parent_file_id FK)
local-deploy/db/migrations/V3__create_filedata.sqlFileData table with parent_file_id, form_index columns

2.2 Current Classification and Split Data Flow

  ClassifyFileTool.execute()
         │
         ├─ 1. Download PDF from Azure Blob Storage
         ├─ 2. Extract text via pdf-parse
         ├─ 3. Chunk text (10 pages/chunk)
         ├─ 4. Call Azure OpenAI per chunk → classification JSON
         ├─ 5. Merge chunk results → master classification JSON
         ├─ 6. Upload classification JSON to Blob (classifications/)
         ├─ 7. Update FileData.status = 'classified'
         │
         └─ IF is_multi_form = true OR forms.length > 1:
              ├─ 8. Build page ranges from forms[]
              ├─ 9. Call splitPDFByPageRanges() → Buffer[]
              ├─ 10. Upload each split PDF to Blob (files/)
              └─ 11. Insert child FileData rows (parent_file_id = parent.id)

2.3 Classification JSON Schema (Current Output)

{
  "is_multi_form": true,
  "num_pages": 12,
  "classified_in_chunks": true,
  "num_chunks": 2,
  "forms": [
    {
      "form_type": "Form 1040",
      "page_start": 1,
      "page_end": 4,
      "confidence": 0.92
    },
    {
      "form_type": "Schedule C",
      "page_start": 5,
      "page_end": 8,
      "confidence": 0.88
    },
    {
      "form_type": "Extra form data",
      "page_ranges": [
        { "start": 9, "end": 10 },
        { "start": 11, "end": 12 }
      ],
      "confidence": 0.5
    }
  ]
}

2.4 FileData Table (Split Records)

FileData
  id                       (GUID, PK)
  parent_file_id           (GUID, FK → FileData.id)    ← NULL for originals (uploaded file)
  form_index               (INT)                       ← 0-based position in split
  form_type                (NVARCHAR)                  ← e.g. "Form 1040"
  blob_path                (NVARCHAR)                  ← files/{splitFileName}.pdf
  classification_json_path (NVARCHAR)                  ← shared with parent
  status                   (NVARCHAR)                  ← 'classified' after split

3. Challenges with the Current Architecture

#ProblemImpact
P1Classify + Split are coupled in one toolCannot split without re-classifying; cannot replace split logic independently

4.1 Design Principles

  1. Single Responsibility - The Classifier receives a message and emits addressable forms classification result.
  2. Contract-Driven - Input is a message containing internal url. Output is a ClassificationResult DTO persisted to the database and/or blob storage.
  3. Fault Isolated - Classification failures must not corrupt the parent file record, this must raise an explicit error that the orchestrator (agentic loop) can surface.
  4. Independently Scalable - The classifier must be callable independently of classification.

4.2 Proposed Service Boundary

┌─────────────────────────────────────────────────────────────────┐
│                     Agentic Loop (Orchestrator)                 │
│                                                                 │
│   fluxx-classifier ──► output ClassificationResult JSON in Blob │
│         │                                                       │
│         │  published a message                                  │
│         │                                                       │
│         ▼  (if multiple form type)                              │
│   fluxx-document-splitter ──► output SplitResult                │
│         │                                                       │
│         │  published a message                                  │
│         │                                                       │
│         ▼                                                       │
│   ExtractFileTool (operates on child FileData rows)             │
└─────────────────────────────────────────────────────────────────┘

Classification and Document Splitter become independent with a well-defined handoff.

4.3 New Service: fluxx-classifier

Location: [CLADevOps/fluxx-classifier: Created by Terraform](https://github.com/CLADevOps/fluxx-classifier)

Published message in accepted-file topic:

{
  fileId: string;
  internalUrl: string;
  clientApplicationId: string;
  timestamp: string
}

High level instruction:

  1. Download the file using internalUrl props from the received message.
  2. Split the file into chunks of 10 (dafault).
  3. Classifiy each chunk using OpenAI.
  4. Filter each form type classification.
  5. Save filtered classification result and upload to blob storage as json file
  6. Save classification result to classifier.File
  7. Publish a message to classified-topic.

SplitResult (as output):

{
  "fileId": "c41abfc2-f1ca-4613-bf34-55fca178c884";
  "internalUrl": "www.blobstorage.com.net/container/test.pdf";
  "clientApplicationId": "a29382d0-2b78-47fe-8864-7c93fa816278'";
  "timestamp": "2026-01-12T12:12:00",
  "is_multi_form": true,
  "num_pages": 12,
  "classified_in_chunks": true,
  "num_chunks": 2,
  "forms": [
    {
      "form_type": "Form 1040",
      "page_start": 1,
      "page_end": 4,
      "confidence": 0.92
    },
    {
      "form_type": "Schedule C",
      "page_start": 5,
      "page_end": 8,
      "confidence": 0.88
    },
    {
      "form_type": "Extra form data",
      "page_ranges": [
        { "start": 9, "end": 10 },
        { "start": 11, "end": 12 }
      ],
      "confidence": 0.5
    }
  ]
}

4.4 Updated fluxx-document-splitter

fluxx-document-splitter is simplified to document splitter only:

  • Removes all classification logic
  • Sets FileData.status = 'spltted' on success
  • Returns the splitted document result for extraction.

This makes the agentic loop’s workflow explicit:

classify_file → (if multi_form) → split_document → extract_file

4.5 Blob Storage Layout

{containerName}/
  {clientName}/
    files/
      original.pdf                          ← original upload
    files_splitted/                         ← NEW: this is just to separate the splitted files from parent file
      original_Form_1040.pdf                ← splitted child
      original_Schedule_C.pdf               ← splitted child
    classifications/
      original_classification.json          ← classification result
    splits/
      original_split_manifest.json          ← NEW: SplitResult JSON
    extractions/
      original_Form_1040_extraction.json
    exports/
      original_Form_1040_export.csv

A dedicated splits/ folder separates split manifests from classification and extraction artifacts.

4.6 End-to-End Data Flow (To-Be)

1. User: "Classify and process the Johnson tax return"
         │
         ▼
2. AgenticLoop → classify_file(fileName="johnson_tax_return.pdf")
         │
         ▼
3. ClassifyFileTool (fluxx-classifier)
   ├─ Download PDF from Blob
   ├─ Chunk text → LLM per chunk → merge → ClassificationResult
   ├─ Upload classification JSON → classifications/
   ├─ Update FileData.status = 'classified'
   └─ Publish message: { parentFileId, parentFileName, parentBlobPath, classification_json_path }
         │
         ▼  (published message consumed if is_multi_form=true)
4. fluxx-document-splitter receives message
         │
         ▼
5. SplitDocumentTool (fluxx-document-splitter)
   ├─ Download and parse ClassificationResult from Blob
   ├─ Assert is_multi_form === true; exit gracefully if false (no split needed)
   ├─ Build validated PageRange[] from forms[]
   ├─ splitPDFByPageRanges() → Buffer[]
   ├─ Upload split PDFs → files_splitted/
   ├─ Upload SplitManifest → splits/
   ├─ Insert child FileData rows (status='split')
   ├─ Update parent FileData.status = 'split'
   └─ Publish SplitResult: { parentFileId, parentFileName, totalForms, splitFiles[] }
         │
         ▼
6. To Be Determined(TBD)