> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://cla-demo.postmansolutions.com/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://cla-demo.postmansolutions.com/_mcp/server. # Classifier Service:-Architecture Design ## 1. Background & Problem Statement Applications frequently receive multi-document uploads that contain mixed document types, for example a single PDF containing multiple tax forms, invoices, or supporting records. Before extraction can occur, each logical document segment must be identified before being routed to the appropriate extract. A centralized Classifier service is required to solve this problem in a consistent, extensible way. Today this responsibility sits **inside `ClassifyFileTool.ts`** in Paperboy. Form classification and splitting are tightly coupled in a service. This creates several challenges: * The classify logic cannot be scaled or replaced independently * Incorrect classification results to incorrect processes downstream | Service | Repository | Responsibility | | --------------------------- | ------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------- | | **fluxx-classifier** | `fluxx-classifier` (new repo) | Consume a PDF, identify document types and page boundaries, return a `ClassificationResult` | | **fluxx-document-splitter** | `fluxx-document-splitter` (new repo) | Consume a `ClassificationResult` + original PDF reference, produce discrete split documents stored in accessible blob storage, return a `SplitManifest` | ## 2. Current State (As-Is) in Paperboy ### 2.1 Where Classification Lives Today | File | Responsibility | | ---------------------------------------------------- | ----------------------------------------------------------------- | | `backend/src/api/v1/agent/tools/ClassifyFileTool.ts` | **Classifies AND splits** does both in one `execute()` call | | `backend/src/utils/pdfProcessor.ts` | `splitPDFByPageRanges()` low-level PDF page copy using `pdf-lib` | | `backend/src/api/v1/files/model.ts` | Persists split child file records (`parent_file_id` FK) | | `local-deploy/db/migrations/V3__create_filedata.sql` | `FileData` table with `parent_file_id`, `form_index` columns | ### 2.2 Current Classification and Split Data Flow ``` ClassifyFileTool.execute() │ ├─ 1. Download PDF from Azure Blob Storage ├─ 2. Extract text via pdf-parse ├─ 3. Chunk text (10 pages/chunk) ├─ 4. Call Azure OpenAI per chunk → classification JSON ├─ 5. Merge chunk results → master classification JSON ├─ 6. Upload classification JSON to Blob (classifications/) ├─ 7. Update FileData.status = 'classified' │ └─ IF is_multi_form = true OR forms.length > 1: ├─ 8. Build page ranges from forms[] ├─ 9. Call splitPDFByPageRanges() → Buffer[] ├─ 10. Upload each split PDF to Blob (files/) └─ 11. Insert child FileData rows (parent_file_id = parent.id) ``` ### 2.3 Classification JSON Schema (Current Output) ```json { "is_multi_form": true, "num_pages": 12, "classified_in_chunks": true, "num_chunks": 2, "forms": [ { "form_type": "Form 1040", "page_start": 1, "page_end": 4, "confidence": 0.92 }, { "form_type": "Schedule C", "page_start": 5, "page_end": 8, "confidence": 0.88 }, { "form_type": "Extra form data", "page_ranges": [ { "start": 9, "end": 10 }, { "start": 11, "end": 12 } ], "confidence": 0.5 } ] } ``` ### 2.4 FileData Table (Split Records) ``` FileData id (GUID, PK) parent_file_id (GUID, FK → FileData.id) ← NULL for originals (uploaded file) form_index (INT) ← 0-based position in split form_type (NVARCHAR) ← e.g. "Form 1040" blob_path (NVARCHAR) ← files/{splitFileName}.pdf classification_json_path (NVARCHAR) ← shared with parent status (NVARCHAR) ← 'classified' after split ``` --- ## 3. Challenges with the Current Architecture | # | Problem | Impact | | -- | ---------------------------------------- | ----------------------------------------------------------------------------- | | P1 | Classify + Split are coupled in one tool | Cannot split without re-classifying; cannot replace split logic independently | --- ## 4. Recommended Architecture ### 4.1 Design Principles 1. **Single Responsibility** - The Classifier receives a message and emits addressable forms classification result. 2. **Contract-Driven** - Input is a message containing internal url. Output is a `ClassificationResult` DTO persisted to the database and/or blob storage. 3. **Fault Isolated** - Classification failures must not corrupt the parent file record, this must raise an explicit error that the orchestrator (agentic loop) can surface. 4. **Independently Scalable** - The classifier must be callable independently of classification. ### 4.2 Proposed Service Boundary ``` ┌─────────────────────────────────────────────────────────────────┐ │ Agentic Loop (Orchestrator) │ │ │ │ fluxx-classifier ──► output ClassificationResult JSON in Blob │ │ │ │ │ │ published a message │ │ │ │ │ ▼ (if multiple form type) │ │ fluxx-document-splitter ──► output SplitResult │ │ │ │ │ │ published a message │ │ │ │ │ ▼ │ │ ExtractFileTool (operates on child FileData rows) │ └─────────────────────────────────────────────────────────────────┘ ``` Classification and Document Splitter become **independent** with a well-defined handoff. ### 4.3 New Service: `fluxx-classifier` **Location:** `[CLADevOps/fluxx-classifier: Created by Terraform](https://github.com/CLADevOps/fluxx-classifier)` **Published message in accepted-file topic:** ```typescript { fileId: string; internalUrl: string; clientApplicationId: string; timestamp: string } ``` **High level instruction:** 1. Download the file using internalUrl props from the received message. 2. Split the file into chunks of 10 (dafault). 3. Classifiy each chunk using OpenAI. 4. Filter each form type classification. 5. Save filtered classification result and upload to blob storage as json file 6. Save classification result to `classifier.File` 7. Publish a message to `classified-topic`. **SplitResult (as output):** ```typescript { "fileId": "c41abfc2-f1ca-4613-bf34-55fca178c884"; "internalUrl": "www.blobstorage.com.net/container/test.pdf"; "clientApplicationId": "a29382d0-2b78-47fe-8864-7c93fa816278'"; "timestamp": "2026-01-12T12:12:00", "is_multi_form": true, "num_pages": 12, "classified_in_chunks": true, "num_chunks": 2, "forms": [ { "form_type": "Form 1040", "page_start": 1, "page_end": 4, "confidence": 0.92 }, { "form_type": "Schedule C", "page_start": 5, "page_end": 8, "confidence": 0.88 }, { "form_type": "Extra form data", "page_ranges": [ { "start": 9, "end": 10 }, { "start": 11, "end": 12 } ], "confidence": 0.5 } ] } ``` ### 4.4 Updated `fluxx-document-splitter` fluxx-document-splitter is **simplified to document splitter only**: * Removes all classification logic * Sets `FileData.status = 'spltted'` on success * Returns the splitted document result for extraction. This makes the agentic loop's workflow explicit: ``` classify_file → (if multi_form) → split_document → extract_file ``` ### 4.5 Blob Storage Layout ``` {containerName}/ {clientName}/ files/ original.pdf ← original upload files_splitted/ ← NEW: this is just to separate the splitted files from parent file original_Form_1040.pdf ← splitted child original_Schedule_C.pdf ← splitted child classifications/ original_classification.json ← classification result splits/ original_split_manifest.json ← NEW: SplitResult JSON extractions/ original_Form_1040_extraction.json exports/ original_Form_1040_export.csv ``` A dedicated `splits/` folder separates split manifests from classification and extraction artifacts. ### 4.6 End-to-End Data Flow (To-Be) ``` 1. User: "Classify and process the Johnson tax return" │ ▼ 2. AgenticLoop → classify_file(fileName="johnson_tax_return.pdf") │ ▼ 3. ClassifyFileTool (fluxx-classifier) ├─ Download PDF from Blob ├─ Chunk text → LLM per chunk → merge → ClassificationResult ├─ Upload classification JSON → classifications/ ├─ Update FileData.status = 'classified' └─ Publish message: { parentFileId, parentFileName, parentBlobPath, classification_json_path } │ ▼ (published message consumed if is_multi_form=true) 4. fluxx-document-splitter receives message │ ▼ 5. SplitDocumentTool (fluxx-document-splitter) ├─ Download and parse ClassificationResult from Blob ├─ Assert is_multi_form === true; exit gracefully if false (no split needed) ├─ Build validated PageRange[] from forms[] ├─ splitPDFByPageRanges() → Buffer[] ├─ Upload split PDFs → files_splitted/ ├─ Upload SplitManifest → splits/ ├─ Insert child FileData rows (status='split') ├─ Update parent FileData.status = 'split' └─ Publish SplitResult: { parentFileId, parentFileName, totalForms, splitFiles[] } │ ▼ 6. To Be Determined(TBD) ```