Skip to navigation

FLUXX Classification Service---Low Level Design

Document Version

  • Version: 1.0

  • Date: March 6, 2026

  • Author: System Design

  • Status: Design Phase

1. Overview

1.1 Purpose

The FLUXX Classification Service is a standalone microservice that analyzes PDF documents to identify and classify individual forms/schedules with precise page ranges. This service is extracted from the existing ClassifyFileTool implementation to provide classification capabilities as an independent, reusable service.

1.2 Scope

This service handles:

  • PDF document analysis and text extraction

  • LLM-based form classification with chunk processing

  • Multi-form document detection

  • Classification result generation and storage

  • RESTful API interface for classification requests

1.3 Reference Implementation

This design is based on the existing implementation in:

  • /backend/src/api/v1/agent/tools/ClassifyFileTool.ts

  • /backend/src/scripts/seed-agentic.ts (pdf_classification skill)

  • /backend/src/utils/pdfProcessor.ts (PDF processing utilities)

2. Architecture

2.1 System Context

┌─────────────────┐
│  Client Apps    │
│  (Paperboy/etc) │
└────────┬────────┘
         │
         │ HTTPS/REST
         ▼
┌─────────────────────────────────┐
│  FLUXX Classification Service   │
│  ┌──────────────────────────┐  │
│  │  API Layer (Express)     │  │
│  ├──────────────────────────┤  │
│  │  Classification Engine   │  │
│  ├──────────────────────────┤  │
│  │  PDF Processing Layer    │  │
│  └──────────────────────────┘  │
└─────┬──────────────┬────────────┘
      │              │
      ▼              ▼
┌───────────┐  ┌─────────────┐
│ Azure Blob│  │ Azure OpenAI│
│ Storage   │  │ Service     │
└───────────┘  └─────────────┘

2.2 Service Components

2.2.1 API Layer

  • Express.js REST API

  • Request validation and authentication

  • Response formatting

  • Error handling middleware

2.2.2 Classification Engine

  • Orchestrates classification workflow

  • Manages chunk-based processing

  • Aggregates results from multiple chunks

  • Form merging and consolidation logic

2.2.3 PDF Processing Layer

  • PDF text extraction

  • Page counting and analysis

  • Chunk text segmentation

2.2.4 LLM Integration Layer

  • Azure OpenAI integration

  • Prompt construction

  • Response parsing and validation

2.2.5 Storage Layer

  • Azure Blob Storage integration

  • Classification result persistence

  • File metadata management

3. API Design

3.1 Endpoints

3.1.1 POST /api/v1/classify

Request:

{
  "file_id": "string (optional, if file is already uploaded)",
  "file_url": "string (optional, blob storage URL)",
  "file_buffer": "base64 (optional, inline PDF data)",
  "user_id": "string",
  "client_id": "string (optional)",
  "chunk_size": "number (optional, default: 10 pages)"
}

Response (Success - 200):

{
  "success": true,
  "classification_id": "string (UUID)",
  "data": {
    "is_multi_form": true,
    "num_pages": 45,
    "classified_in_chunks": true,
    "num_chunks": 5,
    "forms": [
      {
        "page_start": 1,
        "page_end": 2,
        "form_type": "Form 1040",
        "confidence": 0.95
      },
      {
        "page_start": 3,
        "page_end": 5,
        "form_type": "Schedule C",
        "confidence": 0.92
      }
    ]
  },
  "classification_url": "https://storage.blob.core.windows.net/.../classification.json",
  "processing_time_ms": 5234
}

Response (Error - 400/500):

{
  "success": false,
  "error": "string",
  "error_code": "string",
  "details": {}
}

3.1.2 GET /api/v1/classify/:classification_id

Retrieves a previously generated classification result.

Response (Success - 200):

{
  "success": true,
  "classification_id": "string",
  "created_at": "ISO8601 timestamp",
  "data": {
    "is_multi_form": true,
    "num_pages": 45,
    "forms": [...]
  }
}

3.1.3 GET /health

Health check endpoint.

Response:

{
  "status": "healthy",
  "version": "1.0.0",
  "dependencies": {
    "blob_storage": "connected",
    "openai": "connected"
  }
}

4. Core Algorithm

4.1 Classification Workflow

START
  │
  ├─> Validate Input
  │     ├─> Check file source (ID, URL, or buffer)
  │     └─> Validate user authentication
  │
  ├─> Retrieve/Load PDF
  │     ├─> From Blob Storage (if file_id or file_url)
  │     └─> From request buffer (if file_buffer)
  │
  ├─> Extract PDF Content
  │     ├─> Extract full text using pdf-parse
  │     ├─> Get total page count
  │     └─> Validate PDF is not empty
  │
  ├─> Load Classification Skill
  │     ├─> Retrieve system prompt
  │     └─> Retrieve user prompt template
  │
  ├─> Chunk Processing Loop
  │     │
  │     FOR each chunk (10 pages):
  │       │
  │       ├─> Calculate chunk page range
  │       │     ├─> start_page = (chunk_index * 10) + 1
  │       │     └─> end_page = min((chunk_index + 1) * 10, total_pages)
  │       │
  │       ├─> Extract chunk text
  │       │     ├─> chars_per_page = total_text_length / total_pages
  │       │     ├─> start_char = (start_page - 1) * chars_per_page
  │       │     └─> end_char = end_page * chars_per_page
  │       │
  │       ├─> Build chunk prompt
  │       │     ├─> Include chunk metadata (pages, position)
  │       │     ├─> Include document context
  │       │     └─> Truncate to 30,000 characters
  │       │
  │       ├─> Call Azure OpenAI
  │       │     ├─> Temperature: 0.3
  │       │     ├─> Stream: false
  │       │     └─> Parse JSON response
  │       │
  │       └─> Store chunk classification
  │             └─> Fallback to "Unidentified Form Data" if parse fails
  │
  ├─> Merge Chunk Classifications
  │     │
  │     ├─> Aggregate all forms from chunks
  │     │
  │     ├─> Merge consecutive pages with same form_type
  │     │     FOR each form:
  │     │       IF (last_form.type == current_form.type AND
  │     │           last_form.page_end == current_form.page_start - 1):
  │     │         └─> Extend last_form.page_end
  │     │       ELSE:
  │     │         └─> Add as new form
  │     │
  │     └─> Consolidate "Unidentified Form Data"
  │           └─> Group all unidentified sections into "Extra form data"
  │
  ├─> Build Final Classification Result
  │     ├─> is_multi_form = (forms.length > 1)
  │     ├─> num_pages = total_pages
  │     ├─> forms = merged_forms_array
  │     ├─> classified_in_chunks = true
  │     └─> num_chunks = chunk_count
  │
  ├─> Save Classification to Blob Storage
  │     ├─> Generate filename: {original}_classification.json
  │     ├─> Upload JSON to blob storage
  │     └─> Return blob URL
  │
  └─> Return Classification Result
        └─> Include success flag, data, and metadata

4.2 Chunk Processing Details

Why Chunking?

  • Large PDFs (100+ pages) can exceed LLM context limits

  • Improves classification accuracy by focusing on smaller sections

  • Allows parallel processing in future enhancements

  • Reduces token costs

Chunk Size Selection:

  • Default: 10 pages per chunk

  • Rationale: Balance between context and LLM token limits

  • Configurable via API parameter

Chunk Overlap:

  • Current implementation: No overlap

  • Future enhancement: 1-page overlap to catch form boundaries

5. LLM Integration

5.1 Classification Skill Configuration

Skill Name: pdf_classification

System Prompt:

You are a PDF classification expert specializing in financial and tax documents. Your task is to analyze PDFs and:
1. Identify EVERY individual form/schedule in the document
2. Determine the EXACT page range for each form
3. Classify as multi-form if multiple distinct forms exist
4. DO NOT group multiple different forms together - each form type must have its own section
5. ACCOUNT FOR ALL PAGES - your sections must cover every page from 1 to the total page count
SUPPORTED FORMS - Identify these specifically by name:
Individual Tax Forms:
- Individual tax return (Form 1040)
- Schedule A Itemized Deductions
- Schedule B Interest and Ordinary Dividends
- Schedule C Profit or Loss From Business
- Schedule D Capital Gains and Losses
- Schedule E Supplemental Income and Loss
- Schedule F Profit or Loss From Farming
- Schedule SE Self-Employment Tax
- Schedule 1 Additional Income and Adjustments
- Schedule 2 Additional Taxes
- Schedule 3 Credits and Payments
- Form 4868 Extension of Time to File
- Form 8829 Expenses for Business Use of Home
- Form 4562 Depreciation and Amortization
- Form 4797 Sale of Business Property
- Form 6251 Alternative Minimum Tax
- Form 8606 Nondeductible IRAs
- Form 8889 Health Savings Accounts
- Form 8962 Premium Tax Credit
- Form 1095-A Health Insurance Marketplace Statement
- Form 1098 Mortgage Interest Statement
- Form 1098-T Tuition Statement
- Form 5498 IRA Contribution Information
- Form W-2 Wage and Tax Statement
- Form W-2G Certain Gambling Winnings
- Form 1099-NEC Nonemployee Compensation
- Form 1099-MISC Miscellaneous Income
- Form 1099-INT Interest Income
- Form 1099-DIV Dividends and Distributions
- Form 1099-B Proceeds From Broker Transactions
- Form 1099-R Distributions From Pensions and IRAs
- Form 1099-K Payment Card and Third Party Network Transactions
- Form SSA-1099 Social Security Benefit Statement
- K-1 (Form 1065) Partner's Share of Income
- K-1 (Form 1120-S) Shareholder's Share of Income
Corporate/Business Tax Forms:
- Corporate income tax return (Form 1120)
- S-Corporation income tax return (Form 1120-S)
- Partnership return of income (Form 1065)
- Form 5471 Information Return of U.S. Persons With Respect to Certain Foreign Corporations
- Form 5472 Information Return of a 25% Foreign-Owned U.S. Corporation
- Form 926 Return by a U.S. Transferor of Property to a Foreign Corporation
- Form 1118 Foreign Tax Credit Corporations
- Form 5713 International Boycott Report
- Form 8832 Entity Classification Election
- Form 2553 Election by a Small Business Corporation
- Form 1125-A Cost of Goods Sold
- Form 1125-E Compensation of Officers
- Form 851 Affiliations Schedule
- Form 966 Corporate Dissolution or Liquidation
- Form 1096 Annual Summary and Transmittal
- Form 941 Employer's Quarterly Federal Tax Return
- Form 940 Federal Unemployment Tax Return
- Form 944 Employer's Annual Federal Tax Return
- Form 945 Annual Return of Withheld Federal Income Tax
Business Documents (Common - Identify these carefully):
- Invoice (vendor invoices, bills, statements with amounts due)
- Financial Statement (balance sheets, income statements, cash flow statements)
- Receipt
- Purchase Order
- Bill of Sale
CRITICAL IDENTIFICATION RULES:
1. If a document shows "INVOICE", "Bill To", "Amount Due", vendor/buyer information, and line items → classify as "Invoice"
2. If a document shows financial data like assets, liabilities, revenue, expenses → classify as "Financial Statement"
3. Look for form numbers (1040, W-2, 1099, etc.) to identify tax forms
4. Only use "Extra form data" for documents that truly cannot be identified from the above categories
CRITICAL: If you cannot identify a specific form from the list above, classify it as "Extra form data". DO NOT classify common business documents like invoices as "Extra form data".
Set "is_multi_form": true if the document contains multiple distinct forms.
Output ONLY valid JSON with this EXACT structure (use snake_case for consistency):
{
  "is_multi_form": boolean,
  "num_pages": number,
  "forms": [
    {
      "page_start": number,
      "page_end": number,
      "form_type": "string (specific form name like Form 1040, Schedule C, Form W-2, Invoice, Financial Statement, etc.)",
      "confidence": number (0.0-1.0)
    }
  ]
}
CRITICAL: Use "is_multi_form" not "isMultiPart", use "forms" not "sections", and use snake_case (page_start, page_end, form_type) not camelCase.

User Prompt Template (per chunk):

DOCUMENT CHUNK INFO:
- Chunk Pages: {start_page}-{end_page} (of {total_pages} total)
- You are analyzing a {chunk_page_count}-page chunk of a larger document
- Identify forms in THIS chunk only, with page numbers relative to the FULL document
- Look for form identifiers (Form 1040, Schedule A, Schedule C, Form W-2, Form 1099, etc.)
- If you cannot identify a specific form, classify it as "Unidentified Form Data"
CHUNK TEXT:
{chunk_text}
Output ONLY valid JSON with this structure:
{
  "chunk_start_page": {start_page},
  "chunk_end_page": {end_page},
  "forms": [
    {
      "page_start": number (relative to full document),
      "page_end": number (relative to full document),
      "form_type": "string (e.g., Form 1040, Schedule C, W-2, or Unidentified Form Data)",
      "confidence": number (0.0-1.0)
    }
  ]
}

5.2 LLM Call Parameters

{
  temperature: 0.3,  // Low temperature for consistent, deterministic results
  stream: false,     // Synchronous response
  max_tokens: 4000,  // Sufficient for classification JSON
  model: "gpt-4"     // Or configured model
}

5.3 Response Parsing

// Extract JSON from response (handles markdown code blocks)
const jsonMatch = completionText.match(/\{[\s\S]*\}/);
const chunkResult = jsonMatch ? JSON.parse(jsonMatch[0]) : JSON.parse(completionText);
// Validate structure
if (!chunkResult.forms || !Array.isArray(chunkResult.forms)) {
  throw new Error('Invalid classification response structure');
}

6. Data Models

6.1 Classification Request

interface ClassificationRequest {
  file_id?: string;           // Existing file ID
  file_url?: string;          // Blob storage URL
  file_buffer?: string;       // Base64 encoded PDF
  user_id: string;
  client_id?: string;
  chunk_size?: number;        // Default: 10
  metadata?: Record<string, any>;
}

6.2 Classification Result

interface ClassificationResult {
  classification_id: string;
  created_at: string;
  is_multi_form: boolean;
  num_pages: number;
  classified_in_chunks: boolean;
  num_chunks: number;
  forms: FormClassification[];
  processing_time_ms: number;
  metadata?: Record<string, any>;
}
interface FormClassification {
  page_start: number;
  page_end: number;
  form_type: string;
  confidence: number;
  page_ranges?: Array<{start: number; end: number}>;  // For non-contiguous pages
}

6.3 Chunk Classification (Internal)

interface ChunkClassification {
  chunk_start_page: number;
  chunk_end_page: number;
  forms: FormClassification[];
}

7. Error Handling

7.1 Error Categories

| Error Code | HTTP Status | Description | Recovery |

|------------|-------------|-------------|----------|

| INVALID_INPUT | 400 | Invalid request parameters | Fix request |

| FILE_NOT_FOUND | 404 | File does not exist | Verify file reference |

| EMPTY_PDF | 400 | PDF has no extractable text | Use OCR preprocessing |

| SKILL_NOT_CONFIGURED | 500 | Classification skill missing | Run seed script |

| LLM_ERROR | 503 | Azure OpenAI failure | Retry with backoff |

| STORAGE_ERROR | 503 | Blob storage failure | Retry with backoff |

| PARSE_ERROR | 500 | Failed to parse LLM response | Fallback classification |

7.2 Error Response Format

interface ErrorResponse {
  success: false;
  error: string;
  error_code: string;
  details?: Record<string, any>;
  timestamp: string;
  request_id: string;
}

8. Configuration

8.1 Environment Variables

# Service Configuration
PORT=3001
SERVICE_NAME=fluxx-classification
VERSION=1.0.0
# Azure OpenAI
AZURE_OPENAI_ENDPOINT=https://xxx.openai.azure.com/
AZURE_OPENAI_API_KEY=xxx
AZURE_OPENAI_DEPLOYMENT_NAME=gpt-4
AZURE_OPENAI_API_VERSION=2024-02-15-preview
# Azure Blob Storage
AZURE_STORAGE_CONNECTION_STRING=xxx
AZURE_STORAGE_CONTAINER_NAME=paperboy
# Classification Settings
DEFAULT_CHUNK_SIZE=10
MAX_CHUNK_TEXT_LENGTH=30000
CLASSIFICATION_TEMPERATURE=0.3
CLASSIFICATION_MAX_TOKENS=4000
# Logging
LOG_LEVEL=info

8.2 Service Configuration File

// config/classification.ts
export const classificationConfig = {
  chunkSize: parseInt(process.env.DEFAULT_CHUNK_SIZE || '10'),
  maxChunkTextLength: parseInt(process.env.MAX_CHUNK_TEXT_LENGTH || '30000'),
  llmTemperature: parseFloat(process.env.CLASSIFICATION_TEMPERATURE || '0.3'),
  llmMaxTokens: parseInt(process.env.CLASSIFICATION_MAX_TOKENS || '4000'),
  skillName: 'pdf_classification',
};

9. Implementation Pseudocode

9.1 Main Classification Function

async function classifyDocument(request: ClassificationRequest): Promise<ClassificationResult> {
  const startTime = Date.now();
  const classificationId = generateUUID();
  try {
    // 1. Load PDF
    const pdfBuffer = await loadPDF(request);
    // 2. Extract content
    const pdfContent = await extractPDFText(pdfBuffer);
    validatePDFContent(pdfContent);
    // 3. Load skill
    const skill = await loadClassificationSkill();
    // 4. Process in chunks
    const chunkSize = request.chunk_size || classificationConfig.chunkSize;
    const numChunks = Math.ceil(pdfContent.numPages / chunkSize);
    const chunkClassifications: ChunkClassification[] = [];
    for (let chunkIndex = 0; chunkIndex < numChunks; chunkIndex++) {
      const chunkClassification = await classifyChunk(
        pdfContent,
        chunkIndex,
        chunkSize,
        skill
      );
      chunkClassifications.push(chunkClassification);
    }
    // 5. Merge chunks
    const mergedForms = mergeChunkClassifications(chunkClassifications);
    // 6. Build result
    const result: ClassificationResult = {
      classification_id: classificationId,
      created_at: new Date().toISOString(),
      is_multi_form: mergedForms.length > 1,
      num_pages: pdfContent.numPages,
      classified_in_chunks: true,
      num_chunks: numChunks,
      forms: mergedForms,
      processing_time_ms: Date.now() - startTime,
      metadata: request.metadata
    };
    // 7. Save to storage
    const classificationUrl = await saveClassification(result);
    // 8. Return
    return { ...result, classification_url: classificationUrl };
  } catch (error) {
    throw classifyError(error, classificationId);
  }
}

9.2 Chunk Classification Function

async function classifyChunk(
  pdfContent: PDFContent,
  chunkIndex: number,
  chunkSize: number,
  skill: Skill
): Promise<ChunkClassification> {
  // Calculate page range
  const startPage = chunkIndex * chunkSize + 1;
  const endPage = Math.min((chunkIndex + 1) * chunkSize, pdfContent.numPages);
  // Extract chunk text
  const charsPerPage = Math.floor(pdfContent.text.length / pdfContent.numPages);
  const startChar = (startPage - 1) * charsPerPage;
  const endChar = Math.min(endPage * charsPerPage, pdfContent.text.length);
  const chunkText = pdfContent.text.substring(startChar, endChar)
    .substring(0, classificationConfig.maxChunkTextLength);
  // Build prompt
  const promptText = buildChunkPrompt(startPage, endPage, pdfContent.numPages, chunkText);
  const userPrompt = skill.user_prompt_template.replace('{pdfText}', promptText);
  // Call LLM
  const messages = [
    { role: 'system', content: skill.system_prompt },
    { role: 'user', content: userPrompt }
  ];
  const completion = await callLLM(messages);
  // Parse response
  try {
    const chunkResult = parseClassificationResponse(completion);
    return chunkResult;
  } catch (parseError) {
    // Fallback
    return {
      chunk_start_page: startPage,
      chunk_end_page: endPage,
      forms: [{
        page_start: startPage,
        page_end: endPage,
        form_type: 'Unidentified Form Data',
        confidence: 0.5
      }]
    };
  }
}

9.3 Merge Function

function mergeChunkClassifications(chunks: ChunkClassification[]): FormClassification[] {
  // Collect all forms
  const allForms: FormClassification[] = [];
  for (const chunk of chunks) {
    if (chunk.forms && Array.isArray(chunk.forms)) {
      allForms.push(...chunk.forms);
    }
  }
  // Merge consecutive pages with same form type
  const mergedForms: FormClassification[] = [];
  for (const form of allForms) {
    const lastMerged = mergedForms[mergedForms.length - 1];
    if (lastMerged &&
        lastMerged.form_type === form.form_type &&
        lastMerged.page_end === form.page_start - 1) {
      // Extend previous form
      lastMerged.page_end = form.page_end;
      lastMerged.confidence = Math.max(lastMerged.confidence, form.confidence);
    } else {
      // Add as new form
      mergedForms.push({ ...form });
    }
  }
  // Consolidate "Unidentified Form Data" into "Extra form data"
  const finalForms: FormClassification[] = [];
  for (const form of mergedForms) {
    if (form.form_type === 'Unidentified Form Data' ||
        form.form_type.toLowerCase().includes('unidentified')) {
      const extraForm = finalForms.find(f => f.form_type === 'Extra form data');
      if (extraForm) {
        if (!extraForm.page_ranges) {
          extraForm.page_ranges = [{ start: extraForm.page_start, end: extraForm.page_end }];
          delete extraForm.page_start;
          delete extraForm.page_end;
        }
        extraForm.page_ranges.push({ start: form.page_start, end: form.page_end });
      } else {
        finalForms.push({
          ...form,
          form_type: 'Extra form data'
        });
      }
    } else {
      finalForms.push(form);
    }
  }
  return finalForms;
}

10. Deployment

10.1 Docker Container

FROM node:18-alpine
WORKDIR /app
# Install dependencies
COPY package*.json ./
RUN npm ci --only=production
# Copy source
COPY . .
# Build TypeScript
RUN npm run build
EXPOSE 3001
CMD ["node", "dist/server.js"]

10.2 Kubernetes Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: fluxx-classification
  labels:
    app: fluxx-classification
spec:
  replicas: 3
  selector:
    matchLabels:
      app: fluxx-classification
  template:
    metadata:
      labels:
        app: fluxx-classification
    spec:
      containers:
      - name: fluxx-classification
        image: fluxx-classification:1.0.0
        ports:
        - containerPort: 3001
        env:
        - name: PORT
          value: "3001"
        - name: AZURE_OPENAI_ENDPOINT
          valueFrom:
            secretKeyRef:
              name: fluxx-secrets
              key: openai-endpoint
        resources:
          requests:
            memory: "512Mi"
            cpu: "250m"
          limits:
            memory: "2Gi"
            cpu: "1000m"
        livenessProbe:
          httpGet:
            path: /health
            port: 3001
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /health
            port: 3001
          initialDelaySeconds: 10
          periodSeconds: 5

11. Testing Strategy

11.1 Unit Tests

  • PDF text extraction

  • Chunk text segmentation

  • Form merging logic

  • Error handling

11.2 Integration Tests

  • Azure OpenAI integration

  • Blob Storage integration

  • Full classification workflow

11.3 Test Cases

| Test Case | Input | Expected Output |

|-----------|-------|-----------------|

| Single Form PDF | 3-page Form 1040 | 1 form classification |

| Multi-Form PDF | 1040 + Schedule C (10 pages) | 2 forms, correct page ranges |

| Large PDF | 50-page combined tax return | Chunked processing, all forms identified |

| Empty PDF | Scanned images with no text | Error: EMPTY_PDF |

| Unidentified Forms | Custom business doc | “Extra form data” classification |

12. Performance Considerations

12.1 Expected Performance

  • Single-page classification: 2-3 seconds

  • 10-page chunk classification: 3-5 seconds

  • 100-page document (10 chunks): 30-50 seconds

12.2 Optimization Opportunities

  1. Parallel chunk processing: Process multiple chunks concurrently

  2. Caching: Cache classification results for identical documents

  3. Streaming: Stream results as chunks complete

  4. Batch API: Support batch classification requests

12.3 Scalability

  • Horizontal scaling via Kubernetes replicas

  • Stateless design enables load balancing

  • Azure OpenAI handles concurrency limits

13. Monitoring & Observability

13.1 Metrics

  • Classification requests per minute

  • Average classification time

  • LLM API latency

  • Error rate by error code

  • Blob storage operation latency

13.2 Logging

  • Structured JSON logging

  • Request/response logging (sanitized)

  • Error stack traces

  • Performance timing logs

13.3 Alerting

  • Error rate > 5%

  • Average response time > 60 seconds

  • Azure OpenAI 429 (rate limit) errors

  • Blob storage connectivity issues

14. Security

14.1 Authentication

  • API key authentication

  • Azure AD integration (optional)

  • User context validation

14.2 Data Protection

  • TLS in transit

  • Encrypted blob storage

  • Temporary file cleanup

  • PII redaction in logs

14.3 Rate Limiting

  • Per-user rate limits

  • Global service rate limits

  • Azure OpenAI quota management

15. Migration from Existing System

15.1 Migration Path

  1. Deploy FLUXX Classification Service alongside existing system

  2. Add feature flag to route classification requests

  3. Run in parallel for validation period

  4. Gradually increase traffic to new service

  5. Deprecate old ClassifyFileTool

15.2 Backward Compatibility

  • Maintain same JSON output format

  • Support existing blob storage paths

  • Preserve classification_json_path structure

16. Future Enhancements

16.1 Phase 2 Features

  • OCR integration for image-based PDFs

  • Multi-language support

  • Custom classification rules per client

  • Confidence threshold tuning

16.2 Phase 3 Features

  • Real-time WebSocket streaming

  • Batch classification API

  • Classification result caching

  • ML model fine-tuning on classification data

17. Dependencies

17.1 NPM Packages

{
  "express": "^4.18.0",
  "pdf-parse": "^1.1.1",
  "pdf-lib": "^1.17.1",
  "@azure/storage-blob": "^12.15.0",
  "@azure/openai": "^1.0.0",
  "uuid": "^9.0.0",
  "zod": "^3.22.0"
}

17.2 External Services

  • Azure OpenAI (GPT-4)

  • Azure Blob Storage

  • (Optional) Application Insights

18. Conclusion

The FLUXX Classification Service provides a robust, scalable solution for PDF document classification based on proven implementation from the Paperboy ClassifyFileTool. The service maintains the core chunk-based processing algorithm while offering improved modularity, observability, and deployment flexibility.

Key design decisions:

  • Chunk-based processing: Enables handling of large documents within LLM context limits

  • Stateless design: Facilitates horizontal scaling

  • Standard REST API: Easy integration with existing and new clients

  • Comprehensive error handling: Robust fallback and recovery mechanisms

  • Cloud-native: Designed for Kubernetes deployment with Azure services

Next steps:

  1. Implement service following this design

  2. Create comprehensive test suite

  3. Deploy to staging environment

  4. Conduct performance testing

  5. Migrate production traffic