> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://cla-demo.postmansolutions.com/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://cla-demo.postmansolutions.com/_mcp/server.

# High Level Design-(HLD)---fluxx general classifier

**Service Name:** fluxx-general-classifier

**Version:** 1.0.0

**Date:** March 27, 2026

**Status:** Draft

---

## Table of Contents

1. [Overview](#overview)

2. [System Architecture](#system-architecture)

3. [API Endpoints](#api-endpoints)

4. [Data Flow](#data-flow)

5. [External Dependencies](#external-dependencies)

6. [Technology Stack](#technology-stack)

7. [Non-Functional Requirements](#non-functional-requirements)

8. [Security Considerations](#security-considerations)

9. [Deployment Architecture](#deployment-architecture)

10. [Monitoring & Observability](#monitoring--observability)

---

## Overview

### Purpose

The fluxx-general-classifier service provides an asynchronous PDF classification system that processes uploaded PDF documents by splitting them into pages, converting pages to images, and using LLM (Large Language Model) to classify document content and forms. The service batches images into groups of 20 for efficient processing and returns a merged, collated JSON output of the classification results.

### Business Context

This service enables automated document classification and form recognition for tax returns and business documents. It supports identification of IRS tax forms (individual and corporate), common business documents (invoices, financial statements, receipts), and other tax-related documentation. The service supports downstream business processes that require structured document data extraction and categorization.

### Key Features

* **Asynchronous PDF Processing**: Non-blocking PDF upload and processing

* **Page-to-Image Conversion**: Converts PDF pages to images for LLM processing

* **Batch Processing**: Groups images into batches of 20 for optimal LLM performance

* **Tax Form Classification**: Identifies and classifies IRS tax forms, schedules, and supporting documents

* **Business Document Recognition**: Identifies invoices, financial statements, receipts, and purchase orders

* **Status Tracking**: Real-time status monitoring of classification jobs

* **Collated JSON Output**: Merged and structured classification results

---

## System Architecture

### High-Level Architecture Diagram

```

┌─────────────────────────────────────────────────────────────────┐

│                        API Gateway / LB                          │

└─────────────────────────────────────────────────────────────────┘

                              ▼

┌─────────────────────────────────────────────────────────────────┐

│                  fluxx-general-classifier API                    │

│  ┌────────────────────────────────────────────────────────────┐ │

│  │  Middleware (CORS, Auth, CSRF, Correlation ID)            │ │

│  └────────────────────────────────────────────────────────────┘ │

│  ┌────────────────────────────────────────────────────────────┐ │

│  │  Route Handlers                                            │ │

│  │  - POST /api/v1/Classifications                            │ │

│  │  - GET /api/v1/Classifications/{jobId}                     │ │

│  └────────────────────────────────────────────────────────────┘ │

│  ┌────────────────────────────────────────────────────────────┐ │

│  │  Service Layer                                             │ │

│  │  - ClassificationService                                   │ │

│  │  - JobStatusService                                        │ │

│  └────────────────────────────────────────────────────────────┘ │

│  ┌────────────────────────────────────────────────────────────┐ │

│  │  Processing Layer                                          │ │

│  │  - PDFSplitterService                                      │ │

│  │  - ImageConverterService                                   │ │

│  │  - BatchProcessorService                                   │ │

│  │  - LLMInvocationService                                    │ │

│  │  - ResultCollatorService                                   │ │

│  └────────────────────────────────────────────────────────────┘ │

│  ┌────────────────────────────────────────────────────────────┐ │

│  │  Repository Layer                                          │ │

│  │  - JobRepository                                           │ │

│  │  - ClassificationResultRepository                          │ │

│  └────────────────────────────────────────────────────────────┘ │

└─────────────────────────────────────────────────────────────────┘

                              ▼

┌─────────────────────────────────────────────────────────────────┐

│  External Services & Storage                                     │

│  ┌──────────────┐ ┌──────────────┐ ┌──────────────┐            │

│  │ Azure Blob   │ │ Azure Service│ │   SQL DB     │            │

│  │ Storage      │ │     Bus      │ │  (Job Store) │            │

│  └──────────────┘ └──────────────┘ └──────────────┘            │

│  ┌──────────────┐ ┌──────────────┐                             │

│  │  LLM API     │ │ App Insights │                             │

│  │ (OpenAI/     │ │              │                             │

│  │  Azure AI)   │ │              │                             │

│  └──────────────┘ └──────────────┘                             │

└─────────────────────────────────────────────────────────────────┘

```

### Component Overview

\| Component | Responsibility |

\|-----------|---------------|

\| **API Layer** | Handles HTTP requests, validation, and response formatting |

\| **Service Layer** | Orchestrates business logic and workflow coordination |

\| **Processing Layer** | Executes document processing, conversion, and LLM invocation |

\| **Repository Layer** | Manages data persistence and retrieval |

\| **Message Queue** | Enables asynchronous job processing |

\| **Blob Storage** | Stores uploaded PDFs, intermediate images, and results |

\| **Database** | Stores job metadata and status information |

\| **LLM Service** | External API for document classification |

---

## API Endpoints

### 1. POST /api/v1/Classifications

**Purpose**: Initiates asynchronous PDF classification job

**Request**:

```http

POST /api/v1/Classifications

Content-Type: multipart/form-data

  

{

  "file": <PDF binary>,

  "classificationPrompt": "Classify this document and identify all forms present"

}

```

**Request Parameters**:

\| Parameter | Type | Required | Description |

\|-----------|------|----------|-------------|

\| file | File (PDF) | Yes | PDF file to classify |

\| classificationPrompt | String | Yes | LLM prompt for classification instructions |

**Response** (201 Created):

```json

{

  "jobId": "550e8400-e29b-41d4-a716-446655440000",

  "status": "in-progress",

  "createdAt": "2026-03-27T12:00:00Z",

  "statusUrl": "/api/v1/Classifications/550e8400-e29b-41d4-a716-446655440000"

}

```

**Response Headers**:

* `Location: /api/v1/Classifications/{jobId}`

**Error Responses**:

* `400 Bad Request` - Invalid file format or missing parameters

* `401 Unauthorized` - Missing or invalid authentication

* `413 Payload Too Large` - PDF exceeds maximum file size

* `422 Unprocessable Entity` - Invalid PDF or corrupted file

* `500 Internal Server Error` - Unexpected server error

---

### 2. GET /api/v1/Classifications/\{jobId}

**Purpose**: Retrieves classification job status and results

**Request**:

```http

GET /api/v1/Classifications/550e8400-e29b-41d4-a716-446655440000

```

**Path Parameters**:

\| Parameter | Type | Required | Description |

\|-----------|------|----------|-------------|

\| jobId | UUID | Yes | Classification job identifier |

**Response - In Progress** (200 OK):

```json

{

  "jobId": "550e8400-e29b-41d4-a716-446655440000",

  "status": "in-progress",

  "progress": {

    "totalPages": 50,

    "processedPages": 30,

    "percentComplete": 60

  },

  "createdAt": "2026-03-27T12:00:00Z",

  "updatedAt": "2026-03-27T12:05:00Z"

}

```

**Response - Success** (200 OK):

```json

{

  "jobId": "550e8400-e29b-41d4-a716-446655440000",

  "status": "success",

  "createdAt": "2026-03-27T12:00:00Z",

  "completedAt": "2026-03-27T12:10:00Z",

  "result": {

    "is_multi_form": true,

    "num_pages": 50,

    "forms": [

      {

        "page_start": 1,

        "page_end": 2,

        "form_type": "Form 1040",

        "confidence": 0.95

      },

      {

        "page_start": 3,

        "page_end": 4,

        "form_type": "Schedule C",

        "confidence": 0.93

      },

      {

        "page_start": 5,

        "page_end": 6,

        "form_type": "Schedule SE",

        "confidence": 0.91

      },

      {

        "page_start": 7,

        "page_end": 10,

        "form_type": "Form W-2",

        "confidence": 0.97

      },

      {

        "page_start": 11,

        "page_end": 30,

        "form_type": "Form 1099-NEC",

        "confidence": 0.88

      },

      {

        "form_type": "Receipt",

        "confidence": 0.72,

        "page_ranges": [

          {

            "start": 31,

            "end": 35

          },

          {

            "start": 40,

            "end": 42

          },

          {

            "start": 46,

            "end": 50

          }

        ]

      }

    ],

    "classified_in_chunks": true,

    "num_chunks": 3

  }

}

```

**Response - Failed** (200 OK):

```json

{

  "jobId": "550e8400-e29b-41d4-a716-446655440000",

  "status": "failed",

  "createdAt": "2026-03-27T12:00:00Z",

  "failedAt": "2026-03-27T12:02:00Z",

  "error": {

    "code": "PDF_PROCESSING_ERROR",

    "message": "Failed to convert page 15 to image",

    "details": "Corrupted PDF page detected"

  }

}

```

**Error Responses**:

* `404 Not Found` - Job ID does not exist

* `401 Unauthorized` - Missing or invalid authentication

* `500 Internal Server Error` - Unexpected server error

---

### Supported Document Types

The fluxx-general-classifier service is designed to identify and classify the following tax forms and business documents:

#### Individual Tax Forms

* Individual tax return (Form 1040)

* Schedule A Itemized Deductions

* Schedule B Interest and Ordinary Dividends

* Schedule C Profit or Loss From Business

* Schedule D Capital Gains and Losses

* Schedule E Supplemental Income and Loss

* Schedule F Profit or Loss From Farming

* Schedule SE Self-Employment Tax

* Schedule 1 Additional Income and Adjustments

* Schedule 2 Additional Taxes

* Schedule 3 Credits and Payments

* Form 4868 Extension of Time to File

* Form 8829 Expenses for Business Use of Home

* Form 4562 Depreciation and Amortization

* Form 4797 Sale of Business Property

* Form 6251 Alternative Minimum Tax

* Form 8606 Nondeductible IRAs

* Form 8889 Health Savings Accounts

* Form 8962 Premium Tax Credit

* Form 1095-A Health Insurance Marketplace Statement

* Form 1098 Mortgage Interest Statement

* Form 1098-T Tuition Statement

* Form 5498 IRA Contribution Information

* Form W-2 Wage and Tax Statement

* Form W-2G Certain Gambling Winnings

* Form 1099-NEC Nonemployee Compensation

* Form 1099-MISC Miscellaneous Income

* Form 1099-INT Interest Income

* Form 1099-DIV Dividends and Distributions

* Form 1099-B Proceeds From Broker Transactions

* Form 1099-R Distributions From Pensions and IRAs

* Form 1099-K Payment Card and Third Party Network Transactions

* Form SSA-1099 Social Security Benefit Statement

* K-1 (Form 1065) Partner's Share of Income

* K-1 (Form 1120-S) Shareholder's Share of Income

#### Corporate/Business Tax Forms

* Corporate income tax return (Form 1120)

* S-Corporation income tax return (Form 1120-S)

* Partnership return of income (Form 1065)

* Form 5471 Information Return of U.S. Persons With Respect to Certain Foreign Corporations

* Form 5472 Information Return of a 25% Foreign-Owned U.S. Corporation

* Form 926 Return by a U.S. Transferor of Property to a Foreign Corporation

* Form 1118 Foreign Tax Credit Corporations

* Form 5713 International Boycott Report

* Form 8832 Entity Classification Election

* Form 2553 Election by a Small Business Corporation

* Form 1125-A Cost of Goods Sold

* Form 1125-E Compensation of Officers

* Form 851 Affiliations Schedule

* Form 966 Corporate Dissolution or Liquidation

* Form 1096 Annual Summary and Transmittal

* Form 941 Employer's Quarterly Federal Tax Return

* Form 940 Federal Unemployment Tax Return

* Form 944 Employer's Annual Federal Tax Return

* Form 945 Annual Return of Withheld Federal Income Tax

#### Business Documents

* Invoice (vendor invoices, bills, statements with amounts due)

* Financial Statement (balance sheets, income statements, cash flow statements)

* Receipt

* Purchase Order

* Bill of Sale

#### Unidentified Documents

* Extra form data (for documents that cannot be identified from the categories above)

---

### Example Classification Prompts

The `classificationPrompt` parameter should provide clear instructions to the LLM about what to classify. Here are examples:

**Tax Forms Classification**:

```

Identify all IRS tax forms in this document. List each form with its official name and the page range where it appears. Include schedules and supporting forms.

```

**Business Tax Returns**:

```

Analyze this business tax return and identify all forms, schedules, and supporting documents. Classify each form by its IRS designation (e.g., Form 1120, Schedule K-1, Form 1125-A) and specify the page ranges.

```

**Tax Documents with Supporting Forms**:

```

Identify the primary tax return form and all supporting forms in this document. Include W-2s, 1099s, receipts, and other tax-related documentation. Provide specific form names and page numbers.

```

**Business Documents**:

```

Classify all business documents in this PDF. Identify invoices, financial statements, receipts, purchase orders, and any other business records. Specify the document type and page ranges for each.

```

**General Classification**:

```

Classify this document by identifying all distinct forms, reports, or document types present. For each identified item, specify its type/category and the page numbers where it appears.

```

---

## Data Flow

### Classification Job Flow

```

1. Client Uploads PDF

   ↓

2. API validates request and creates job record

   ↓

3. PDF saved to Blob Storage

   ↓

4. Job message posted to Service Bus queue

   ↓

5. Background worker picks up message

   ↓

6. PDF Split into individual pages

   ↓

7. Each page converted to image (PNG/JPEG)

   ↓

8. Images batched into groups of 20

   ↓

9. Each batch sent to LLM with classification prompt

   ↓

10. LLM responses collected

   ↓

11. Results merged and collated

   ↓

12. JSON result saved to Blob Storage

   ↓

13. Job status updated to "success"

   ↓

14. Client polls GET /api/v1/Classifications/{jobId} endpoint to retrieve results

```

### Async Processing Pattern

The service implements an **async request-reply pattern**:

1. **Client Request**: POST /api/v1/Classifications returns immediately with jobId

2. **Background Processing**: Worker processes job asynchronously

3. **Status Polling**: Client polls GET /api/v1/Classifications/\{jobId} until complete

4. **Result Retrieval**: Final classification results returned in status response

### Batching Strategy

* **Batch Size**: 20 images per LLM request

* **Rationale**: Balances LLM token limits with processing efficiency

* **Parallel Processing**: Multiple batches can be processed concurrently

* **Error Handling**: Failed batches are retried independently

---

## External Dependencies

### Azure Blob Storage

**Purpose**: Store uploaded PDFs, intermediate images, and classification results

**Containers**:

* `uploaded-pdfs` - Original PDF files

* `page-images` - Converted page images

* `classification-results` - Final JSON outputs

**Configuration**:

* Managed Identity authentication

* Lifecycle policies for automatic cleanup (30 days)

* Encryption at rest enabled

### Azure Service Bus

**Purpose**: Queue job messages for asynchronous processing

**Queue**: `classification-jobs`

**Message Format**:

```json

{

  "jobId": "550e8400-e29b-41d4-a716-446655440000",

  "pdfBlobPath": "uploaded-pdfs/550e8400.pdf",

  "classificationPrompt": "...",

  "createdAt": "2026-03-27T12:00:00Z"

}

```

**Configuration**:

* Message TTL: 24 hours

* Max delivery count: 3

* Dead letter queue enabled

### SQL Database

**Purpose**: Store job metadata and status

**Tables**:

* `ClassificationJobs` - Job tracking and status

* `JobProgress` - Detailed progress information

**Configuration**:

* Managed Identity authentication

* Connection pooling enabled

* Automatic retry policies

### LLM API (Azure OpenAI / OpenAI)

**Purpose**: Perform document classification and form recognition

**Model**: GPT-4 Vision or similar multimodal model

**Request Pattern**:

```json

{

  "model": "gpt-4-vision-preview",

  "messages": [

    {

      "role": "system",

      "content": "You are a document classification expert. Analyze the provided document images and identify all forms present. Return a JSON object with 'forms' array, where each form has: page_start (number), page_end (number), form_type (string), and confidence (number between 0 and 1). For forms on non-contiguous pages, use page_ranges array with start/end objects instead of page_start/page_end."

    },

    {

      "role": "user",

      "content": [

        {

          "type": "text",

          "text": "{classificationPrompt}"

        },

        {

          "type": "image_url",

          "image_url": "data:image/png;base64,..."

        }

      ]

    }

  ],

  "response_format": { "type": "json_object" }

}

```

**Configuration**:

* API key authentication (or Managed Identity if Azure OpenAI)

* Circuit breaker pattern for fault tolerance

* Retry policy with exponential backoff

* Request timeout: 60 seconds

### Azure Application Insights

**Purpose**: Logging, monitoring, and distributed tracing

**Telemetry**:

* Request/response logging

* Custom metrics (jobs processed, success rate)

* Performance counters

* Exception tracking

---

## Technology Stack

### Core Framework

* **Runtime**: Node.js 20 LTS

* **Language**: TypeScript 5.x

* **Web Framework**: Fastify (via framework-nodejs-fastify)

* **Framework**: framework-nodejs-fastify with full feature integration

### Architecture Organization

The service follows **feature-based organization** as mandated by API\_DEVELOPMENT\_STANDARDS.md:

* **Feature Structure**: Code organized by business feature (`classifications`) rather than technical layer

* **Version Management**: Features own their versions (`v1/`, `v2/`), enabling independent version evolution

* **Modular Design**: Classification feature is self-contained with its own routes, services, repositories, and models

* **Shared Components**: Common utilities, configuration, and middleware in `shared/` directory

* **Scalability**: New features can be added as independent modules without affecting existing code

This structure enables:

* Clean separation of concerns

* Independent feature deployment (if microservices split needed)

* Clear API versioning boundaries

* Team autonomy per feature

### Key Libraries

\| Library | Purpose |

\|---------|---------|

\| `framework-nodejs-fastify` | Core web framework integration |

\| `framework-nodejs-appconfig` | Configuration management |

\| `framework-nodejs-logging` | Structured logging |

\| `framework-nodejs-storage` | Azure Blob Storage client |

\| `framework-nodejs-sql` | SQL Database connectivity |

\| `framework-nodejs-pubsub` | Azure Service Bus integration |

\| `framework-nodejs-service-invocation` | HTTP client for LLM API calls |

\| `pdf-lib` or `pdf2pic` | PDF processing and page extraction |

\| `sharp` | Image processing and optimization |

\| `@azure/identity` | Azure authentication |

### Development Tools

* **Testing**: Vitest

* **Linting**: ESLint + Prettier

* **API Spec**: OpenAPI 3.0.3

* **API Testing**: Postman collections

---

## Non-Functional Requirements

### Performance

\| Metric | Target |

\|--------|--------|

\| API Response Time (POST /api/v1/Classifications) | \< 500ms (job creation) |

\| API Response Time (GET /api/v1/Classifications/\{jobId}) | \< 200ms |

\| PDF Processing Throughput | 10-20 pages/minute per worker |

\| Concurrent Job Processing | 10+ jobs simultaneously |

\| Maximum PDF Size | 50 MB |

\| Maximum Page Count | 500 pages per PDF |

### Scalability

* **Horizontal Scaling**: Multiple worker instances via Kubernetes

* **Queue-Based**: Service Bus enables distributed processing

* **Stateless Workers**: Workers can be scaled independently

* **Database Connection Pooling**: Efficient resource utilization

### Reliability

* **Availability Target**: 99.9%

* **Error Recovery**: Automatic retry for transient failures

* **Circuit Breaker**: Prevents cascade failures to LLM API

* **Dead Letter Queue**: Failed messages captured for analysis

* **Job Timeout**: Maximum processing time of 2 hours

### Security

* **Authentication**: OAuth 2.0 / Azure AD

* **Authorization**: RBAC with role-based endpoint access

* **CSRF Protection**: Enabled for state-changing operations

* **Data Encryption**:

\- In transit: TLS 1.2+

\- At rest: Azure Storage encryption

* **Secrets Management**: Azure Key Vault for API keys

* **Input Validation**: File type, size, and content validation

---

## Security Considerations

### Authentication & Authorization

* All endpoints require valid OAuth 2.0 Bearer token

* Role-based access control:

\- `classifier.write` - POST /api/v1/Classifications permission

\- `classifier.read` - GET /api/v1/Classifications permission

### Data Protection

* **PII Handling**: Assume PDFs may contain sensitive data

* **Data Retention**: Automatic cleanup after 30 days

* **Access Logging**: All data access logged to Application Insights

* **Encryption**: All data encrypted at rest and in transit

### API Security

* **Rate Limiting**: 100 requests per minute per user

* **CORS**: Restricted to allowed origins

* **CSRF Tokens**: Required for POST requests

* **File Upload Validation**:

\- File type whitelist (PDF only)

\- Magic number validation

\- Maximum file size enforcement

\- Malware scanning integration (optional)

### Network Security

* **Private Endpoints**: Azure services connected via private endpoints

* **NSG Rules**: Network security groups restrict access

* **TLS Enforcement**: HTTPS only, TLS 1.2 minimum

---

## Deployment Architecture

### Kubernetes Deployment

```yaml

Architecture:

  - AKS Cluster (3+ nodes)

  - Ingress Controller (NGINX)

  - Horizontal Pod Autoscaler

  - Persistent Volume Claims for temporary storage

```

### Environment Configuration

\| Environment | Purpose | Replicas |

\|-------------|---------|----------|

\| Development | Local/dev testing | 1 |

\| Staging | Pre-production validation | 2 |

\| Production | Live service | 3+ (auto-scale) |

### Resource Requirements

**API Pods**:

* CPU: 200m (request) / 1000m (limit)

* Memory: 256Mi (request) / 512Mi (limit)

**Worker Pods**:

* CPU: 500m (request) / 2000m (limit)

* Memory: 1Gi (request) / 2Gi (limit)

### Health Checks

* **Liveness Probe**: `GET /health/live` (10s interval)

* **Readiness Probe**: `GET /health/ready` (5s interval)

* **Startup Probe**: Initial 30s delay

---

## Monitoring & Observability

### Key Metrics

**Application Metrics**:

* Total classification jobs submitted

* Jobs completed successfully

* Jobs failed

* Average processing time per page

* LLM API call success rate

* Queue depth (Service Bus)

**Performance Metrics**:

* API endpoint response times (P50, P95, P99)

* Background worker processing rate

* Database query performance

* Blob storage operation latency

**Business Metrics**:

* Documents classified per day

* Most common document types

* Average document size

* Peak processing hours

### Alerts

* Job failure rate > 10%

* Average processing time > 30 minutes

* Queue backlog > 100 messages

* API error rate > 5%

* Database connection failures

* LLM API circuit breaker open

### Logging Strategy

* **Structured JSON logs** to Application Insights

* **Log Levels**:

\- INFO: Job lifecycle events

\- WARN: Retries and recoverable errors

\- ERROR: Processing failures and exceptions

* **Correlation IDs**: Track requests across services

* **PII Redaction**: Sensitive data not logged

### Distributed Tracing

* OpenTelemetry integration

* Trace job flow from API request through completion

* Track LLM API calls and performance

* Identify bottlenecks and optimization opportunities

---

## Appendix

### Glossary

\| Term | Definition |

\|------|------------|

\| **Classification Job** | An asynchronous task to process and classify a PDF document |

\| **Batch** | Group of 20 page images sent to LLM for processing |

\| **Collation** | Merging of classification results from multiple batches |

\| **Job Status** | Current state of classification job (in-progress, success, failed) |

### Future Enhancements

1. **Real-time Notifications**: WebSocket support for live status updates

2. **Webhook Support**: Callback URLs for job completion

3. **Multi-format Support**: Word, Excel, image files

4. **Custom Models**: Support for fine-tuned domain-specific LLM models

5. **Batch Job Submission**: Upload multiple PDFs in single request

6. **Priority Queues**: Expedited processing for premium users

7. **Result Caching**: Cache classification results for identical documents

---

**Document Owner**: Architecture Team

**Reviewers**: Security Team, Engineering Team, Product Team

**Next Review Date**: June 27, 2026