> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://cla-demo.postmansolutions.com/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://cla-demo.postmansolutions.com/_mcp/server.

# Low Level Design-(LLD)---fluxx general classifier

**Service Name:** fluxx-general-classifier\
**Version:** 1.0.0\
**Date:** March 27, 2026\
**Status:** Draft

---

## Table of Contents

1. [Repository Structure](#repository-structure)
2. [Component Design](#component-design)
3. [Data Models](#data-models)
4. [API Implementation](#api-implementation)
5. [Service Layer](#service-layer)
6. [Processing Pipeline](#processing-pipeline)
7. [Repository Layer](#repository-layer)
8. [Configuration](#configuration)
9. [Error Handling](#error-handling)
10. [Testing Strategy](#testing-strategy)
11. [Sequence Diagrams](#sequence-diagrams)
12. [Database Schema](#database-schema)

---

## Repository Structure

```
fluxx-general-classifier/
├── docs/
│   ├── openapi.yaml                    # OpenAPI 3.0.3 specification
│   ├── README.md                       # Service documentation
│   └── ops/
│       └── runbook.md                  # Operational runbook
├── postman/
│   └── collections/
│       ├── classify-api.json
│       └── environments/
│           ├── dev.json
│           └── prod.json
├── src/
│   ├── features/                       # Feature-based organization
│   │   └── classifications/            # Classification feature
│   │       └── v1/                     # Version 1 implementation
│   │           ├── routes/
│   │           │   └── classifications.routes.ts
│   │           ├── services/
│   │           │   ├── classification.service.ts
│   │           │   ├── job-status.service.ts
│   │           │   ├── pdf-splitter.service.ts
│   │           │   ├── image-converter.service.ts
│   │           │   ├── batch-processor.service.ts
│   │           │   ├── llm-invocation.service.ts
│   │           │   └── result-collator.service.ts
│   │           ├── repositories/
│   │           │   ├── job.repository.ts
│   │           │   └── classification-result.repository.ts
│   │           ├── workers/
│   │           │   └── classification-worker.ts
│   │           ├── models/
│   │           │   ├── job.model.ts
│   │           │   ├── classification-result.model.ts
│   │           │   └── dto/
│   │           │       ├── classify-request.dto.ts
│   │           │       ├── classify-response.dto.ts
│   │           │       ├── status-response.dto.ts
│   │           │       └── llm-request.dto.ts
│   │           └── middleware/
│   │               ├── file-upload.middleware.ts
│   │               └── validation.middleware.ts
│   ├── shared/                         # Shared utilities
│   │   ├── config/
│   │   │   ├── app-config.json
│   │   │   ├── dependency-config.ts
│   │   │   └── resilience-config.ts
│   │   ├── middleware/
│   │   │   └── error-handler.ts
│   │   ├── utils/
│   │   │   ├── errors.ts
│   │   │   ├── validators.ts
│   │   │   └── constants.ts
│   │   └── types/
│   │       ├── fastify.d.ts
│   │       └── index.d.ts
│   └── index.ts                        # Application entry point
├── tests/
│   ├── unit/
│   │   ├── services/
│   │   ├── repositories/
│   │   └── utils/
│   ├── integration/
│   │   ├── routes/
│   │   └── workers/
│   └── fixtures/
│       ├── sample.pdf
│       └── mock-responses.json
├── .env.example
├── .gitignore
├── .eslintrc.json
├── .prettierrc
├── Dockerfile
├── package.json
├── tsconfig.json
├── vite.config.ts                      # Vitest configuration
└── README.md
```

---

## Architecture Principles

### Feature-Based Organization

This service follows the **feature-based organization** pattern as specified in API\_DEVELOPMENT\_STANDARDS.md:

**Key Benefits:**

* **Feature Isolation**: All code related to classifications (routes, services, repositories, models) is grouped together
* **Version Management**: Each feature owns its versions (v1/, v2/), making version upgrades independent
* **Team Scalability**: Different teams can work on different features without conflicts
* **Modular Design**: Features can be extracted into separate services if needed
* **Clear Boundaries**: Business logic is organized by domain, not technical layer

**Structure Explanation:**

```
src/features/classifications/v1/
├── routes/          # API endpoints for v1
├── services/        # Business logic for v1
├── repositories/    # Data access for v1
├── workers/         # Background jobs for v1
├── models/          # Data models and DTOs for v1
└── middleware/      # Feature-specific middleware for v1
```

**Shared Components:**

```
src/shared/
├── config/          # Application-wide configuration
├── middleware/      # Global middleware (error handlers, auth)
├── utils/           # Common utilities
└── types/           # Shared type definitions
```

**Future Versioning:**
When v2 is needed, it can be added as `src/features/classifications/v2/` with:

* Independent implementation
* Different data models
* Backward-compatible or breaking changes
* Side-by-side operation with v1

---

## Component Design

### 1. API Layer Components

#### ClassificationsRouteHandler

**Responsibility**: Handle POST /api/v1/Classifications and GET /api/v1/Classifications/\{jobId} requests

```typescript
// src/features/classifications/v1/routes/classifications.routes.ts
import { FastifyInstance, FastifyRequest, FastifyReply } from 'fastify';
import { ClassificationService } from '../services/classification.service';
import { JobStatusService } from '../services/job-status.service';
import { ClassifyRequestDTO } from '../models/dto/classify-request.dto';
import { Logger } from 'framework-nodejs-extensions';

interface ClassificationsRouteHandlerDeps {
  classificationService: ClassificationService;
  jobStatusService: JobStatusService;
  logger: Logger;
}

export async function classificationsRoutes(
  fastify: FastifyInstance,
  opts: ClassificationsRouteHandlerDeps
) {
  const { classificationService, jobStatusService, logger } = opts;

  // POST /api/v1/Classifications
  fastify.post<{
    Body: ClassifyRequestDTO;
  }>(
    '/api/v1/Classifications',
    {
      schema: {
        description: 'Submit PDF for asynchronous classification',
        tags: ['Classification'],
        consumes: ['multipart/form-data'],
        body: {
          type: 'object',
          required: ['file', 'classificationPrompt'],
          properties: {
            file: {
              type: 'string',
              format: 'binary',
              description: 'PDF file to classify',
            },
            classificationPrompt: {
              type: 'string',
              description: 'LLM prompt for classification',
              minLength: 10,
              maxLength: 2000,
            },
          },
        },
        response: {
          201: {
            description: 'Classification job created',
            type: 'object',
            properties: {
              jobId: { type: 'string', format: 'uuid' },
              status: { type: 'string', enum: ['in-progress'] },
              createdAt: { type: 'string', format: 'date-time' },
              statusUrl: { type: 'string' },
            },
          },
          400: { $ref: '#/components/schemas/Error' },
          413: { $ref: '#/components/schemas/Error' },
          422: { $ref: '#/components/schemas/Error' },
        },
      },
      config: {
        authRequired: true,
        allowedScopes: ['classifier.write'],
      },
    },
    async (request: FastifyRequest, reply: FastifyReply) => {
      logger.info('Received classification request', {
        correlationId: request.id,
      });

      // Extract file from multipart request
      const data = await request.file();
      
      if (!data) {
        return reply.status(400).send({
          error: 'MISSING_FILE',
          message: 'PDF file is required',
        });
      }

      // Validate file type
      if (data.mimetype !== 'application/pdf') {
        return reply.status(400).send({
          error: 'INVALID_FILE_TYPE',
          message: 'Only PDF files are supported',
        });
      }

      // Extract classification prompt from fields
      const fields = data.fields as any;
      const classificationPrompt = fields?.classificationPrompt?.value;

      if (!classificationPrompt) {
        return reply.status(400).send({
          error: 'MISSING_PROMPT',
          message: 'classificationPrompt is required',
        });
      }

      // Create classification job
      const result = await classificationService.createJob({
        fileBuffer: await data.toBuffer(),
        fileName: data.filename,
        classificationPrompt,
        userId: request.user?.id,
      });

      reply.status(201)
        .header('Location', `/api/v1/Classifications/${result.jobId}`)
        .send(result);
    }
  );

  // GET /api/v1/Classifications/:jobId
  fastify.get<{
    Params: { jobId: string };
  }>(
    '/api/v1/Classifications/:jobId',
    {
      schema: {
        description: 'Get classification job status',
        tags: ['Status'],
        params: {
          type: 'object',
          required: ['jobId'],
          properties: {
            jobId: {
              type: 'string',
              format: 'uuid',
              description: 'Classification job ID',
            },
          },
        },
        response: {
          200: {
            description: 'Job status retrieved',
            type: 'object',
            properties: {
              jobId: { type: 'string', format: 'uuid' },
              status: {
                type: 'string',
                enum: ['in-progress', 'success', 'failed'],
              },
              createdAt: { type: 'string', format: 'date-time' },
              updatedAt: { type: 'string', format: 'date-time' },
              completedAt: { type: 'string', format: 'date-time' },
              result: { type: 'object' },
              error: { type: 'object' },
            },
          },
          404: { $ref: '#/components/schemas/Error' },
        },
      },
      config: {
        authRequired: true,
        allowedScopes: ['classifier.read'],
      },
    },
    async (request: FastifyRequest<{ Params: { jobId: string } }>, reply: FastifyReply) => {
      const { jobId } = request.params;

      logger.info('Fetching job status', { jobId });

      const status = await jobStatusService.getJobStatus(jobId);

      if (!status) {
        return reply.status(404).send({
          error: 'JOB_NOT_FOUND',
          message: `Job with ID ${jobId} not found`,
        });
      }

      reply.send(status);
    }
  );
}
```

---

### 2. Service Layer Components

#### ClassificationService

**Responsibility**: Orchestrate classification job creation and submission

```typescript
// src/features/classifications/v1/services/classification.service.ts
import { v4 as uuidv4 } from 'uuid';
import { Logger, ConfigClient } from 'framework-nodejs-extensions';
import { BlobStorageClient } from 'framework-nodejs-storage';
import { ServiceBusClient } from 'framework-nodejs-pubsub';
import { JobRepository } from '../repositories/job.repository';
import { ClassifyRequestDTO } from '../models/dto/classify-request.dto';
import { ClassifyResponseDTO } from '../models/dto/classify-response.dto';
import { JobStatus } from '../models/job.model';

export class ClassificationService {
  constructor(
    private logger: Logger,
    private blobStorageClient: BlobStorageClient,
    private serviceBusClient: ServiceBusClient,
    private jobRepository: JobRepository,
    private configClient: ConfigClient
  ) {}

  async createJob(request: ClassifyRequestDTO): Promise<ClassifyResponseDTO> {
    const jobId = uuidv4();
    const timestamp = new Date();

    this.logger.info('Creating classification job', { jobId });

    try {
      // 1. Validate file size
      const maxFileSizeMB = await this.configClient.get<number>('MAX_PDF_SIZE_MB') || 50;
      const fileSizeBytes = request.fileBuffer.length;
      const fileSizeMB = fileSizeBytes / (1024 * 1024);

      if (fileSizeMB > maxFileSizeMB) {
        throw new Error(`File size ${fileSizeMB.toFixed(2)}MB exceeds maximum ${maxFileSizeMB}MB`);
      }

      // 2. Upload PDF to blob storage
      const blobPath = `uploaded-pdfs/${jobId}/${request.fileName}`;
      await this.blobStorageClient.upload(
        'classification-storage',
        blobPath,
        request.fileBuffer
      );

      this.logger.info('PDF uploaded to blob storage', { jobId, blobPath });

      // 3. Create job record in database
      await this.jobRepository.create({
        jobId,
        userId: request.userId,
        fileName: request.fileName,
        fileSizeBytes,
        pdfBlobPath: blobPath,
        classificationPrompt: request.classificationPrompt,
        status: JobStatus.InProgress,
        createdAt: timestamp,
        updatedAt: timestamp,
      });

      this.logger.info('Job record created in database', { jobId });

      // 4. Send message to Service Bus queue
      const queueMessage = {
        jobId,
        pdfBlobPath: blobPath,
        classificationPrompt: request.classificationPrompt,
        createdAt: timestamp.toISOString(),
      };

      await this.serviceBusClient.sendMessage('classification-jobs', queueMessage);

      this.logger.info('Job message sent to queue', { jobId });

      // 5. Return response
      return {
        jobId,
        status: 'in-progress',
        createdAt: timestamp.toISOString(),
        statusUrl: `/api/v1/Classifications/${jobId}`,
      };
    } catch (error) {
      this.logger.error('Failed to create classification job', { jobId, error });
      
      // Attempt cleanup
      await this.jobRepository.updateStatus(jobId, JobStatus.Failed, {
        errorCode: 'JOB_CREATION_FAILED',
        errorMessage: error.message,
      });

      throw error;
    }
  }
}
```

#### JobStatusService

**Responsibility**: Retrieve and format job status information

```typescript
// src/features/classifications/v1/services/job-status.service.ts
import { Logger } from 'framework-nodejs-extensions';
import { BlobStorageClient } from 'framework-nodejs-storage';
import { JobRepository } from '../repositories/job.repository';
import { ClassificationResultRepository } from '../repositories/classification-result.repository';
import { StatusResponseDTO } from '../models/dto/status-response.dto';
import { JobStatus } from '../models/job.model';

export class JobStatusService {
  constructor(
    private logger: Logger,
    private jobRepository: JobRepository,
    private classificationResultRepository: ClassificationResultRepository,
    private blobStorageClient: BlobStorageClient
  ) {}

  async getJobStatus(jobId: string): Promise<StatusResponseDTO | null> {
    this.logger.info('Retrieving job status', { jobId });

    // Fetch job from database
    const job = await this.jobRepository.findById(jobId);

    if (!job) {
      this.logger.warn('Job not found', { jobId });
      return null;
    }

    // Build base response
    const response: StatusResponseDTO = {
      jobId: job.jobId,
      status: job.status,
      createdAt: job.createdAt.toISOString(),
      updatedAt: job.updatedAt.toISOString(),
    };

    // Add status-specific data
    switch (job.status) {
      case JobStatus.InProgress:
        response.progress = {
          totalPages: job.totalPages || 0,
          processedPages: job.processedPages || 0,
          percentComplete: job.totalPages 
            ? Math.round((job.processedPages / job.totalPages) * 100)
            : 0,
        };
        break;

      case JobStatus.Success:
        response.completedAt = job.completedAt?.toISOString();
        
        // Fetch classification result
        const result = await this.classificationResultRepository.findByJobId(jobId);
        if (result) {
          response.result = result.classificationData;
        }
        break;

      case JobStatus.Failed:
        response.failedAt = job.failedAt?.toISOString();
        response.error = {
          code: job.errorCode,
          message: job.errorMessage,
          details: job.errorDetails,
        };
        break;
    }

    return response;
  }
}
```

#### PDFSplitterService

**Responsibility**: Split PDF into individual pages

```typescript
// src/features/classifications/v1/services/pdf-splitter.service.ts
import { PDFDocument } from 'pdf-lib';
import { Logger } from 'framework-nodejs-extensions';

export interface PDFPage {
  pageNumber: number;
  pdfBytes: Buffer;
}

export class PDFSplitterService {
  constructor(private logger: Logger) {}

  async splitPDF(pdfBuffer: Buffer): Promise<PDFPage[]> {
    this.logger.info('Splitting PDF into pages');

    try {
      const pdfDoc = await PDFDocument.load(pdfBuffer);
      const pageCount = pdfDoc.getPageCount();

      this.logger.info('PDF loaded', { pageCount });

      const pages: PDFPage[] = [];

      for (let i = 0; i < pageCount; i++) {
        // Create new PDF with single page
        const singlePageDoc = await PDFDocument.create();
        const [copiedPage] = await singlePageDoc.copyPages(pdfDoc, [i]);
        singlePageDoc.addPage(copiedPage);

        // Save as bytes
        const pdfBytes = await singlePageDoc.save();

        pages.push({
          pageNumber: i + 1,
          pdfBytes: Buffer.from(pdfBytes),
        });
      }

      this.logger.info('PDF split complete', { pageCount: pages.length });

      return pages;
    } catch (error) {
      this.logger.error('Failed to split PDF', { error });
      throw new Error(`PDF splitting failed: ${error.message}`);
    }
  }
}
```

#### ImageConverterService

**Responsibility**: Convert PDF pages to images

```typescript
// src/features/classifications/v1/services/image-converter.service.ts
import { fromBuffer } from 'pdf2pic';
import sharp from 'sharp';
import { Logger } from 'framework-nodejs-extensions';

export interface ConvertedImage {
  pageNumber: number;
  imageBuffer: Buffer;
  format: 'png' | 'jpeg';
  width: number;
  height: number;
}

export class ImageConverterService {
  constructor(private logger: Logger) {}

  async convertToImage(
    pdfBuffer: Buffer,
    pageNumber: number,
    options?: { format?: 'png' | 'jpeg'; quality?: number; dpi?: number }
  ): Promise<ConvertedImage> {
    const { format = 'png', quality = 90, dpi = 150 } = options || {};

    this.logger.debug('Converting PDF page to image', { pageNumber, format, dpi });

    try {
      // Convert PDF to image using pdf2pic
      const converter = fromBuffer(pdfBuffer, {
        density: dpi,
        format: format,
        quality: quality,
        width: 2048,
        height: 2048,
      });

      const result = await converter(1, { responseType: 'buffer' });

      if (!result || !result.buffer) {
        throw new Error('Image conversion produced no output');
      }

      // Optimize with sharp
      const optimized = await sharp(result.buffer)
        .resize(1024, 1024, { fit: 'inside', withoutEnlargement: true })
        .toBuffer();

      const metadata = await sharp(optimized).metadata();

      this.logger.debug('Image conversion complete', {
        pageNumber,
        width: metadata.width,
        height: metadata.height,
        size: optimized.length,
      });

      return {
        pageNumber,
        imageBuffer: optimized,
        format,
        width: metadata.width || 0,
        height: metadata.height || 0,
      };
    } catch (error) {
      this.logger.error('Failed to convert PDF page to image', { pageNumber, error });
      throw new Error(`Image conversion failed for page ${pageNumber}: ${error.message}`);
    }
  }

  async convertPages(pdfPages: Buffer[]): Promise<ConvertedImage[]> {
    const images: ConvertedImage[] = [];

    for (let i = 0; i < pdfPages.length; i++) {
      const image = await this.convertToImage(pdfPages[i], i + 1);
      images.push(image);
    }

    return images;
  }
}
```

#### BatchProcessorService

**Responsibility**: Group images into batches and coordinate LLM processing

```typescript
// src/features/classifications/v1/services/batch-processor.service.ts
import { Logger, ConfigClient } from 'framework-nodejs-extensions';
import { LLMInvocationService } from './llm-invocation.service';
import { ConvertedImage } from './image-converter.service';

export interface BatchResult {
  batchNumber: number;
  pageNumbers: number[];
  classification: any;
  success: boolean;
  error?: string;
}

export class BatchProcessorService {
  private readonly BATCH_SIZE = 20;

  constructor(
    private logger: Logger,
    private configClient: ConfigClient,
    private llmInvocationService: LLMInvocationService
  ) {}

  async processBatches(
    images: ConvertedImage[],
    classificationPrompt: string
  ): Promise<BatchResult[]> {
    this.logger.info('Processing image batches', {
      totalImages: images.length,
      batchSize: this.BATCH_SIZE,
    });

    const batches = this.createBatches(images);
    const results: BatchResult[] = [];

    // Process batches sequentially (can be parallelized with concurrency limit)
    for (let i = 0; i < batches.length; i++) {
      const batch = batches[i];
      const batchNumber = i + 1;

      this.logger.info('Processing batch', {
        batchNumber,
        totalBatches: batches.length,
        imageCount: batch.length,
      });

      try {
        const classification = await this.llmInvocationService.classifyImages(
          batch,
          classificationPrompt
        );

        results.push({
          batchNumber,
          pageNumbers: batch.map((img) => img.pageNumber),
          classification,
          success: true,
        });

        this.logger.info('Batch processed successfully', { batchNumber });
      } catch (error) {
        this.logger.error('Batch processing failed', { batchNumber, error });

        results.push({
          batchNumber,
          pageNumbers: batch.map((img) => img.pageNumber),
          classification: null,
          success: false,
          error: error.message,
        });

        // Optionally retry failed batch
        // await this.retryBatch(batch, classificationPrompt);
      }
    }

    return results;
  }

  private createBatches(images: ConvertedImage[]): ConvertedImage[][] {
    const batches: ConvertedImage[][] = [];

    for (let i = 0; i < images.length; i += this.BATCH_SIZE) {
      batches.push(images.slice(i, i + this.BATCH_SIZE));
    }

    this.logger.info('Created batches', {
      totalImages: images.length,
      batchCount: batches.length,
    });

    return batches;
  }
}
```

#### LLMInvocationService

**Responsibility**: Invoke LLM API for document classification

```typescript
// src/features/classifications/v1/services/llm-invocation.service.ts
import { Logger, ConfigClient } from 'framework-nodejs-extensions';
import { ServiceInvocationClient } from 'framework-nodejs-service-invocation';
import { ConvertedImage } from './image-converter.service';

export class LLMInvocationService {
  constructor(
    private logger: Logger,
    private configClient: ConfigClient,
    private httpClient: ServiceInvocationClient
  ) {}

  async classifyImages(
    images: ConvertedImage[],
    classificationPrompt: string
  ): Promise<any> {
    this.logger.info('Invoking LLM for classification', {
      imageCount: images.length,
    });

    const apiKey = await this.configClient.get<string>('OPENAI_API_KEY');
    const apiUrl = await this.configClient.get<string>('OPENAI_API_URL');
    const model = await this.configClient.get<string>('OPENAI_MODEL') || 'gpt-4-vision-preview';

    // Build LLM request
    const messages = [
      {
        role: 'system',
        content: `You are an expert tax and business document classification system.

1. Identify EVERY individual form/schedule in the document
2. Determine the EXACT page range for each form
3. Classify as multi-form if multiple distinct forms exist
4. DO NOT group multiple different forms together - each form type must have its own section
5. ACCOUNT FOR ALL PAGES - your sections must cover every page from 1 to the total page count

SUPPORTED FORMS - Identify these specifically by name:

Individual Tax Forms:
- Individual tax return (Form 1040)
- Schedule A Itemized Deductions
- Schedule B Interest and Ordinary Dividends
- Schedule C Profit or Loss From Business
- Schedule D Capital Gains and Losses
- Schedule E Supplemental Income and Loss
- Schedule F Profit or Loss From Farming
- Schedule SE Self-Employment Tax
- Schedule 1 Additional Income and Adjustments
- Schedule 2 Additional Taxes
- Schedule 3 Credits and Payments
- Form 4868 Extension of Time to File
- Form 8829 Expenses for Business Use of Home
- Form 4562 Depreciation and Amortization
- Form 4797 Sale of Business Property
- Form 6251 Alternative Minimum Tax
- Form 8606 Nondeductible IRAs
- Form 8889 Health Savings Accounts
- Form 8962 Premium Tax Credit
- Form 1095-A Health Insurance Marketplace Statement
- Form 1098 Mortgage Interest Statement
- Form 1098-T Tuition Statement
- Form 5498 IRA Contribution Information
- Form W-2 Wage and Tax Statement
- Form W-2G Certain Gambling Winnings
- Form 1099-NEC Nonemployee Compensation
- Form 1099-MISC Miscellaneous Income
- Form 1099-INT Interest Income
- Form 1099-DIV Dividends and Distributions
- Form 1099-B Proceeds From Broker Transactions
- Form 1099-R Distributions From Pensions and IRAs
- Form 1099-K Payment Card and Third Party Network Transactions
- Form SSA-1099 Social Security Benefit Statement
- K-1 (Form 1065) Partner's Share of Income
- K-1 (Form 1120-S) Shareholder's Share of Income

Corporate/Business Tax Forms:
- Corporate income tax return (Form 1120)
- S-Corporation income tax return (Form 1120-S)
- Partnership return of income (Form 1065)
- Form 5471 Information Return of U.S. Persons With Respect to Certain Foreign Corporations
- Form 5472 Information Return of a 25% Foreign-Owned U.S. Corporation
- Form 926 Return by a U.S. Transferor of Property to a Foreign Corporation
- Form 1118 Foreign Tax Credit Corporations
- Form 5713 International Boycott Report
- Form 8832 Entity Classification Election
- Form 2553 Election by a Small Business Corporation
- Form 1125-A Cost of Goods Sold
- Form 1125-E Compensation of Officers
- Form 851 Affiliations Schedule
- Form 966 Corporate Dissolution or Liquidation
- Form 1096 Annual Summary and Transmittal
- Form 941 Employer's Quarterly Federal Tax Return
- Form 940 Federal Unemployment Tax Return
- Form 944 Employer's Annual Federal Tax Return
- Form 945 Annual Return of Withheld Federal Income Tax

Business Documents (Common - Identify these carefully):
- Invoice (vendor invoices, bills, statements with amounts due)
- Financial Statement (balance sheets, income statements, cash flow statements)
- Receipt
- Purchase Order
- Bill of Sale

CRITICAL IDENTIFICATION RULES:
1. If a document shows "INVOICE", "Bill To", "Amount Due", vendor/buyer information, and line items → classify as "Invoice"
2. If a document shows financial data like assets, liabilities, revenue, expenses → classify as "Financial Statement"
3. Look for form numbers (1040, W-2, 1099, etc.) to identify tax forms
4. Only use "Extra form data" for documents that truly cannot be identified from the above categories

CRITICAL: If you cannot identify a specific form from the list above, classify it as "Extra form data". DO NOT classify common business documents like invoices as "Extra form data".

Set "is_multi_form": true if the document contains multiple distinct forms.

Output ONLY valid JSON with this EXACT structure (use snake_case for consistency):
{
  "is_multi_form": boolean,
  "num_pages": number,
  "forms": [
    {
      "page_start": number,
      "page_end": number,
      "form_type": "string (specific form name like Form 1040, Schedule C, Form W-2, Invoice, Financial Statement, etc.)",
      "confidence": number (0.0-1.0)
    }
  ]
}`,
      },
      {
        role: 'user',
        content: [
          {
            type: 'text',
            text: classificationPrompt,
          },
          ...images.map((img) => ({
            type: 'image_url',
            image_url: {
              url: `data:image/${img.format};base64,${img.imageBuffer.toString('base64')}`,
            },
          })),
        ],
      },
    ];

    const requestBody = {
      model,
      messages,
      max_tokens: 4096,
      response_format: { type: 'json_object' },
    };

    try {
      const response = await this.httpClient.post(
        `${apiUrl}/chat/completions`,
        requestBody,
        {
          headers: {
            'Content-Type': 'application/json',
            Authorization: `Bearer ${apiKey}`,
          },
          timeout: 60000, // 60 seconds
        }
      );

      const classification = JSON.parse(response.data.choices[0].message.content);

      this.logger.info('LLM classification success', {
        imageCount: images.length,
      });

      return classification;
    } catch (error) {
      this.logger.error('LLM invocation failed', { error });
      throw new Error(`LLM classification failed: ${error.message}`);
    }
  }
}
```

#### ResultCollatorService

**Responsibility**: Merge and collate classification results from multiple batches

```typescript
// src/features/classifications/v1/services/result-collator.service.ts
import { Logger } from 'framework-nodejs-extensions';
import { BatchResult } from './batch-processor.service';

export interface FormClassification {
  page_start?: number;
  page_end?: number;
  form_type: string;
  confidence: number;
  page_ranges?: Array<{ start: number; end: number }>;
}

export interface CollatedResult {
  is_multi_form: boolean;
  num_pages: number;
  forms: FormClassification[];
  classified_in_chunks: boolean;
  num_chunks: number;
}

export class ResultCollatorService {
  constructor(private logger: Logger) {}

  async collateResults(batchResults: BatchResult[]): Promise<CollatedResult> {
    this.logger.info('Collating batch results', {
      batchCount: batchResults.length,
    });

    const successfulBatches = batchResults.filter((b) => b.success);

    // Extract all classifications
    const classifications = successfulBatches.map((b) => b.classification);

    // Merge and normalize forms from all batches
    const allForms = this.mergeForms(classifications);

    // Determine if document contains multiple forms
    const isMultiForm = allForms.length > 1;

    // Count total pages
    const totalPages = this.countTotalPages(batchResults);

    const collatedResult: CollatedResult = {
      is_multi_form: isMultiForm,
      num_pages: totalPages,
      forms: allForms,
      classified_in_chunks: true,
      num_chunks: batchResults.length,
    };

    this.logger.info('Result collation complete', {
      isMultiForm,
      totalPages,
      formsCount: allForms.length,
      numChunks: batchResults.length,
    });

    return collatedResult;
  }

  private mergeForms(classifications: any[]): FormClassification[] {
    const allForms: FormClassification[] = [];

    for (const classification of classifications) {
      if (classification.forms && Array.isArray(classification.forms)) {
        for (const form of classification.forms) {
          allForms.push({
            page_start: form.page_start,
            page_end: form.page_end,
            form_type: form.form_type,
            confidence: form.confidence,
            page_ranges: form.page_ranges,
          });
        }
      }
    }

    // Sort forms by page_start (or first range start)
    allForms.sort((a, b) => {
      const aStart = a.page_start || (a.page_ranges?.[0]?.start ?? Infinity);
      const bStart = b.page_start || (b.page_ranges?.[0]?.start ?? Infinity);
      return aStart - bStart;
    });

    // Optionally merge contiguous forms of the same type
    return this.mergeContiguousForms(allForms);
  }

  private mergeContiguousForms(forms: FormClassification[]): FormClassification[] {
    if (forms.length === 0) return [];

    const merged: FormClassification[] = [];
    let current = { ...forms[0] };

    for (let i = 1; i < forms.length; i++) {
      const next = forms[i];

      // Only merge if same form type and contiguous pages
      if (
        current.form_type === next.form_type &&
        current.page_end !== undefined &&
        next.page_start !== undefined &&
        current.page_end + 1 === next.page_start
      ) {
        // Merge into current
        current.page_end = next.page_end;
        // Average confidence
        current.confidence = (current.confidence + next.confidence) / 2;
      } else {
        // Push current and start new
        merged.push(current);
        current = { ...next };
      }
    }

    // Don't forget the last one
    merged.push(current);

    return merged;
  }

  private countTotalPages(batchResults: BatchResult[]): number {
    const allPages = batchResults.flatMap((b) => b.pageNumbers);
    return Math.max(...allPages, 0);
  }
}
```

---

### 3. Worker Component

#### ClassificationWorker

**Responsibility**: Background worker that processes classification jobs from queue

```typescript
// src/features/classifications/v1/workers/classification-worker.ts
import { Logger } from 'framework-nodejs-extensions';
import { ServiceBusClient } from 'framework-nodejs-pubsub';
import { BlobStorageClient } from 'framework-nodejs-storage';
import { JobRepository } from '../repositories/job.repository';
import { ClassificationResultRepository } from '../repositories/classification-result.repository';
import { PDFSplitterService } from '../services/pdf-splitter.service';
import { ImageConverterService } from '../services/image-converter.service';
import { BatchProcessorService } from '../services/batch-processor.service';
import { ResultCollatorService } from '../services/result-collator.service';
import { JobStatus } from '../models/job.model';

export class ClassificationWorker {
  constructor(
    private logger: Logger,
    private serviceBusClient: ServiceBusClient,
    private blobStorageClient: BlobStorageClient,
    private jobRepository: JobRepository,
    private classificationResultRepository: ClassificationResultRepository,
    private pdfSplitterService: PDFSplitterService,
    private imageConverterService: ImageConverterService,
    private batchProcessorService: BatchProcessorService,
    private resultCollatorService: ResultCollatorService
  ) {}

  async start(): Promise<void> {
    this.logger.info('Starting classification worker');

    // Subscribe to Service Bus queue
    await this.serviceBusClient.subscribeToQueue(
      'classification-jobs',
      async (message) => {
        await this.processJob(message);
      }
    );

    this.logger.info('Worker subscribed to classification-jobs queue');
  }

  private async processJob(message: any): Promise<void> {
    const { jobId, pdfBlobPath, classificationPrompt } = message;

    this.logger.info('Processing classification job', { jobId });

    try {
      // 1. Download PDF from blob storage
      const pdfBuffer = await this.blobStorageClient.download(
        'classification-storage',
        pdfBlobPath
      );

      this.logger.info('PDF downloaded from storage', { jobId });

      // 2. Split PDF into pages
      const pages = await this.pdfSplitterService.splitPDF(pdfBuffer);

      // Update job with total pages
      await this.jobRepository.updateProgress(jobId, {
        totalPages: pages.length,
        processedPages: 0,
      });

      this.logger.info('PDF split into pages', { jobId, pageCount: pages.length });

      // 3. Convert pages to images
      const images = await this.imageConverterService.convertPages(
        pages.map((p) => p.pdfBytes)
      );

      this.logger.info('Pages converted to images', { jobId, imageCount: images.length });

      // 4. Process images in batches
      const batchResults = await this.batchProcessorService.processBatches(
        images,
        classificationPrompt
      );

      this.logger.info('Batch processing complete', {
        jobId,
        batchCount: batchResults.length,
      });

      // 5. Collate results
      const collatedResult = await this.resultCollatorService.collateResults(batchResults);

      this.logger.info('Results collated', { jobId });

      // 6. Save result to database
      await this.classificationResultRepository.create({
        jobId,
        classificationData: collatedResult,
        createdAt: new Date(),
      });

      // 7. Upload result to blob storage
      const resultBlobPath = `classification-results/${jobId}/result.json`;
      await this.blobStorageClient.upload(
        'classification-storage',
        resultBlobPath,
        Buffer.from(JSON.stringify(collatedResult, null, 2))
      );

      this.logger.info('Result saved to storage', { jobId, resultBlobPath });

      // 8. Update job status to success
      await this.jobRepository.updateStatus(jobId, JobStatus.Success, {
        completedAt: new Date(),
        processedPages: collatedResult.processedPages,
        resultBlobPath,
      });

      this.logger.info('Classification job completed successfully', { jobId });
    } catch (error) {
      this.logger.error('Classification job failed', { jobId, error });

      // Update job status to failed
      await this.jobRepository.updateStatus(jobId, JobStatus.Failed, {
        failedAt: new Date(),
        errorCode: 'PROCESSING_ERROR',
        errorMessage: error.message,
        errorDetails: error.stack,
      });
    }
  }

  async stop(): Promise<void> {
    this.logger.info('Stopping classification worker');
    await this.serviceBusClient.close();
  }
}
```

---

## Data Models

### Job Model

```typescript
// src/features/classifications/v1/models/job.model.ts
export enum JobStatus {
  InProgress = 'in-progress',
  Success = 'success',
  Failed = 'failed',
}

export interface Job {
  jobId: string;
  userId?: string;
  fileName: string;
  fileSizeBytes: number;
  pdfBlobPath: string;
  classificationPrompt: string;
  status: JobStatus;
  totalPages?: number;
  processedPages?: number;
  resultBlobPath?: string;
  errorCode?: string;
  errorMessage?: string;
  errorDetails?: string;
  createdAt: Date;
  updatedAt: Date;
  completedAt?: Date;
  failedAt?: Date;
}
```

### Classification Result Model

```typescript
// src/features/classifications/v1/models/classification-result.model.ts
export interface ClassificationResult {
  id?: number;
  jobId: string;
  classificationData: any; // JSON blob
  createdAt: Date;
}
```

### DTOs

```typescript
// src/features/classifications/v1/models/dto/classify-request.dto.ts
export interface ClassifyRequestDTO {
  fileBuffer: Buffer;
  fileName: string;
  classificationPrompt: string;
  userId?: string;
}

// src/features/classifications/v1/models/dto/classify-response.dto.ts
export interface ClassifyResponseDTO {
  jobId: string;
  status: string;
  createdAt: string;
  statusUrl: string;
}

// src/features/classifications/v1/models/dto/status-response.dto.ts
export interface StatusResponseDTO {
  jobId: string;
  status: string;
  createdAt: string;
  updatedAt: string;
  completedAt?: string;
  failedAt?: string;
  progress?: {
    totalPages: number;
    processedPages: number;
    percentComplete: number;
  };
  result?: any;
  error?: {
    code: string;
    message: string;
    details?: string;
  };
}
```

---

## Repository Layer

### JobRepository

```typescript
// src/features/classifications/v1/repositories/job.repository.ts
import { Logger } from 'framework-nodejs-extensions';
import { DbConnector } from 'framework-nodejs-sql';
import { Job, JobStatus } from '../models/job.model';

export class JobRepository {
  constructor(
    private logger: Logger,
    private dbConnector: DbConnector
  ) {}

  async create(job: Job): Promise<void> {
    const query = `
      INSERT INTO ClassificationJobs (
        jobId, userId, fileName, fileSizeBytes, pdfBlobPath,
        classificationPrompt, status, createdAt, updatedAt
      ) VALUES (
        @jobId, @userId, @fileName, @fileSizeBytes, @pdfBlobPath,
        @classificationPrompt, @status, @createdAt, @updatedAt
      )
    `;

    await this.dbConnector.executeQuery(query, {
      jobId: job.jobId,
      userId: job.userId,
      fileName: job.fileName,
      fileSizeBytes: job.fileSizeBytes,
      pdfBlobPath: job.pdfBlobPath,
      classificationPrompt: job.classificationPrompt,
      status: job.status,
      createdAt: job.createdAt,
      updatedAt: job.updatedAt,
    });

    this.logger.info('Job created in database', { jobId: job.jobId });
  }

  async findById(jobId: string): Promise<Job | null> {
    const query = `
      SELECT * FROM ClassificationJobs WHERE jobId = @jobId
    `;

    const result = await this.dbConnector.executeQuery(query, { jobId });

    if (result.recordset.length === 0) {
      return null;
    }

    return this.mapToJob(result.recordset[0]);
  }

  async updateStatus(
    jobId: string,
    status: JobStatus,
    updates?: Partial<Job>
  ): Promise<void> {
    const fields: string[] = ['status = @status', 'updatedAt = @updatedAt'];
    const params: any = {
      jobId,
      status,
      updatedAt: new Date(),
    };

    if (updates) {
      Object.keys(updates).forEach((key) => {
        fields.push(`${key} = @${key}`);
        params[key] = updates[key];
      });
    }

    const query = `
      UPDATE ClassificationJobs
      SET ${fields.join(', ')}
      WHERE jobId = @jobId
    `;

    await this.dbConnector.executeQuery(query, params);

    this.logger.info('Job status updated', { jobId, status });
  }

  async updateProgress(
    jobId: string,
    progress: { totalPages: number; processedPages: number }
  ): Promise<void> {
    const query = `
      UPDATE ClassificationJobs
      SET totalPages = @totalPages,
          processedPages = @processedPages,
          updatedAt = @updatedAt
      WHERE jobId = @jobId
    `;

    await this.dbConnector.executeQuery(query, {
      jobId,
      totalPages: progress.totalPages,
      processedPages: progress.processedPages,
      updatedAt: new Date(),
    });

    this.logger.debug('Job progress updated', { jobId, progress });
  }

  private mapToJob(row: any): Job {
    return {
      jobId: row.jobId,
      userId: row.userId,
      fileName: row.fileName,
      fileSizeBytes: row.fileSizeBytes,
      pdfBlobPath: row.pdfBlobPath,
      classificationPrompt: row.classificationPrompt,
      status: row.status,
      totalPages: row.totalPages,
      processedPages: row.processedPages,
      resultBlobPath: row.resultBlobPath,
      errorCode: row.errorCode,
      errorMessage: row.errorMessage,
      errorDetails: row.errorDetails,
      createdAt: row.createdAt,
      updatedAt: row.updatedAt,
      completedAt: row.completedAt,
      failedAt: row.failedAt,
    };
  }
}
```

### ClassificationResultRepository

```typescript
// src/features/classifications/v1/repositories/classification-result.repository.ts
import { Logger } from 'framework-nodejs-extensions';
import { DbConnector } from 'framework-nodejs-sql';
import { ClassificationResult } from '../models/classification-result.model';

export class ClassificationResultRepository {
  constructor(
    private logger: Logger,
    private dbConnector: DbConnector
  ) {}

  async create(result: ClassificationResult): Promise<void> {
    const query = `
      INSERT INTO ClassificationResults (jobId, classificationData, createdAt)
      VALUES (@jobId, @classificationData, @createdAt)
    `;

    await this.dbConnector.executeQuery(query, {
      jobId: result.jobId,
      classificationData: JSON.stringify(result.classificationData),
      createdAt: result.createdAt,
    });

    this.logger.info('Classification result saved', { jobId: result.jobId });
  }

  async findByJobId(jobId: string): Promise<ClassificationResult | null> {
    const query = `
      SELECT * FROM ClassificationResults WHERE jobId = @jobId
    `;

    const result = await this.dbConnector.executeQuery(query, { jobId });

    if (result.recordset.length === 0) {
      return null;
    }

    const row = result.recordset[0];
    return {
      id: row.id,
      jobId: row.jobId,
      classificationData: JSON.parse(row.classificationData),
      createdAt: row.createdAt,
    };
  }
}
```

---

## Configuration

### Application Configuration

```json
// src/shared/config/app-config.json
{
  "FRAMEWORK_APPLICATION_NAME": "fluxx-general-classifier",
  "FRAMEWORK_APPLICATION_VERSION": "1.0.0",
  "FRAMEWORK_HTTP_PORT": 3000,
  "FRAMEWORK_LOGGING_LEVEL": "info",
  
  "MAX_PDF_SIZE_MB": 50,
  "MAX_PDF_PAGES": 500,
  "IMAGE_BATCH_SIZE": 20,
  "IMAGE_FORMAT": "png",
  "IMAGE_DPI": 150,
  
  "OPENAI_MODEL": "gpt-4-vision-preview",
  "OPENAI_MAX_TOKENS": 4096,
  "OPENAI_TIMEOUT_MS": 60000,
  
  "BLOB_CONTAINER_NAME": "classification-storage",
  "SERVICE_BUS_QUEUE_NAME": "classification-jobs",
  
  "DB_POOL_MIN": 2,
  "DB_POOL_MAX": 10,
  "DB_IDLE_TIMEOUT_MS": 30000,
  
  "FEATURE_PARALLEL_BATCH_PROCESSING": false,
  "FEATURE_AUTO_RETRY_FAILED_BATCHES": true
}
```

### Dependency Injection Setup

```typescript
// src/shared/config/dependency-config.ts
import { FrameworkFastify } from 'framework-nodejs-fastify';
import { DependencyKeys, DependencyLifetime } from 'framework-nodejs-extensions';

// Services
import { ClassificationService } from '../features/classifications/v1/services/classification.service';
import { JobStatusService } from '../features/classifications/v1/services/job-status.service';
import { PDFSplitterService } from '../features/classifications/v1/services/pdf-splitter.service';
import { ImageConverterService } from '../features/classifications/v1/services/image-converter.service';
import { BatchProcessorService } from '../features/classifications/v1/services/batch-processor.service';
import { LLMInvocationService } from '../features/classifications/v1/services/llm-invocation.service';
import { ResultCollatorService } from '../features/classifications/v1/services/result-collator.service';

// Repositories
import { JobRepository } from '../features/classifications/v1/repositories/job.repository';
import { ClassificationResultRepository } from '../features/classifications/v1/repositories/classification-result.repository';

// Worker
import { ClassificationWorker } from '../features/classifications/v1/workers/classification-worker';

export function registerDependencies(framework: FrameworkFastify): void {
  const container = framework.dependencies;

  // Repositories (Scoped - per request)
  container.registerClass(
    'jobRepository',
    JobRepository,
    DependencyLifetime.Scoped
  );
  container.registerClass(
    'classificationResultRepository',
    ClassificationResultRepository,
    DependencyLifetime.Scoped
  );

  // Services (Scoped - per request)
  container.registerClass(
    'classificationService',
    ClassificationService,
    DependencyLifetime.Scoped
  );
  container.registerClass(
    'jobStatusService',
    JobStatusService,
    DependencyLifetime.Scoped
  );

  // Processing Services (Transient - new instance each time)
  container.registerClass(
    'pdfSplitterService',
    PDFSplitterService,
    DependencyLifetime.Transient
  );
  container.registerClass(
    'imageConverterService',
    ImageConverterService,
    DependencyLifetime.Transient
  );
  container.registerClass(
    'batchProcessorService',
    BatchProcessorService,
    DependencyLifetime.Transient
  );
  container.registerClass(
    'llmInvocationService',
    LLMInvocationService,
    DependencyLifetime.Transient
  );
  container.registerClass(
    'resultCollatorService',
    ResultCollatorService,
    DependencyLifetime.Transient
  );

  // Worker (Singleton - one instance)
  container.registerClass(
    'classificationWorker',
    ClassificationWorker,
    DependencyLifetime.Singleton
  );
}
```

### Resilience Configuration

```typescript
// src/shared/config/resilience-config.ts
export const resiliencePolicies = {
  llmApi: {
    retryOptions: {
      enabled: true,
      maxAttempts: 3,
      interval: 1000,
      intervalType: 'exponential',
      maxInterval: 10000,
    },
    circuitBreakerOptions: {
      enabled: true,
      errorThresholdPercentage: 50,
      resetTime: 60000,
      minimumThroughput: 3,
    },
    timeoutOptions: {
      enabled: true,
      timeoutMs: 60000,
    },
  },
  
  blobStorage: {
    retryOptions: {
      enabled: true,
      maxAttempts: 3,
      interval: 500,
      intervalType: 'exponential',
    },
  },
  
  database: {
    retryOptions: {
      enabled: true,
      maxAttempts: 3,
      interval: 100,
      intervalType: 'exponential',
    },
    circuitBreakerOptions: {
      enabled: true,
      errorThresholdPercentage: 20,
      resetTime: 30000,
    },
  },
};
```

---

## Error Handling

### Custom Error Classes

```typescript
// src/shared/utils/errors.ts
export class PDFProcessingError extends Error {
  constructor(message: string, public cause?: Error) {
    super(message);
    this.name = 'PDFProcessingError';
  }
}

export class ImageConversionError extends Error {
  constructor(message: string, public pageNumber?: number, public cause?: Error) {
    super(message);
    this.name = 'ImageConversionError';
  }
}

export class LLMInvocationError extends Error {
  constructor(message: string, public statusCode?: number, public cause?: Error) {
    super(message);
    this.name = 'LLMInvocationError';
  }
}

export class JobNotFoundError extends Error {
  constructor(public jobId: string) {
    super(`Job with ID ${jobId} not found`);
    this.name = 'JobNotFoundError';
  }
}

export class FileTooLargeError extends Error {
  constructor(public fileSizeMB: number, public maxSizeMB: number) {
    super(`File size ${fileSizeMB}MB exceeds maximum ${maxSizeMB}MB`);
    this.name = 'FileTooLargeError';
  }
}

export class InvalidFileTypeError extends Error {
  constructor(public mimeType: string) {
    super(`Invalid file type: ${mimeType}. Only PDF files are supported.`);
    this.name = 'InvalidFileTypeError';
  }
}
```

### Global Error Handler

```typescript
// src/shared/middleware/error-handler.ts
import { FastifyInstance, FastifyError, FastifyRequest, FastifyReply } from 'fastify';
import {
  PDFProcessingError,
  ImageConversionError,
  LLMInvocationError,
  JobNotFoundError,
  FileTooLargeError,
  InvalidFileTypeError,
} from '../utils/errors';

export async function errorHandlerPlugin(fastify: FastifyInstance) {
  fastify.setErrorHandler(
    (error: FastifyError, request: FastifyRequest, reply: FastifyReply) => {
      request.log.error({ err: error }, 'Request error');

      // Custom error handling
      if (error instanceof JobNotFoundError) {
        return reply.status(404).send({
          error: 'JOB_NOT_FOUND',
          message: error.message,
          timestamp: new Date().toISOString(),
          requestId: request.id,
        });
      }

      if (error instanceof FileTooLargeError) {
        return reply.status(413).send({
          error: 'FILE_TOO_LARGE',
          message: error.message,
          timestamp: new Date().toISOString(),
          requestId: request.id,
        });
      }

      if (error instanceof InvalidFileTypeError) {
        return reply.status(400).send({
          error: 'INVALID_FILE_TYPE',
          message: error.message,
          timestamp: new Date().toISOString(),
          requestId: request.id,
        });
      }

      if (error instanceof PDFProcessingError || error instanceof ImageConversionError) {
        return reply.status(422).send({
          error: 'PROCESSING_ERROR',
          message: error.message,
          timestamp: new Date().toISOString(),
          requestId: request.id,
        });
      }

      if (error instanceof LLMInvocationError) {
        return reply.status(502).send({
          error: 'LLM_SERVICE_ERROR',
          message: 'External classification service is unavailable',
          timestamp: new Date().toISOString(),
          requestId: request.id,
        });
      }

      // Standard Fastify errors
      if (error.statusCode === 401) {
        return reply.status(401).send({
          error: 'UNAUTHORIZED',
          message: 'Authentication required',
          timestamp: new Date().toISOString(),
          requestId: request.id,
        });
      }

      if (error.statusCode === 403) {
        return reply.status(403).send({
          error: 'FORBIDDEN',
          message: 'Insufficient permissions',
          timestamp: new Date().toISOString(),
          requestId: request.id,
        });
      }

      if (error.statusCode && error.statusCode < 500) {
        return reply.status(error.statusCode).send({
          error: 'BAD_REQUEST',
          message: error.message,
          timestamp: new Date().toISOString(),
          requestId: request.id,
        });
      }

      // Internal server errors
      return reply.status(500).send({
        error: 'INTERNAL_SERVER_ERROR',
        message: 'An unexpected error occurred',
        timestamp: new Date().toISOString(),
        requestId: request.id,
      });
    }
  );
}
```

---

## Testing Strategy

### Unit Tests

```typescript
// tests/unit/services/result-collator.service.test.ts
import { describe, it, expect, vi, beforeEach } from 'vitest';
import { ResultCollatorService } from '../../../src/features/classifications/v1/services/result-collator.service';
import { Logger } from 'framework-nodejs-extensions';

describe('ResultCollatorService', () => {
  let service: ResultCollatorService;
  let mockLogger: Logger;

  beforeEach(() => {
    mockLogger = {
      info: vi.fn(),
      error: vi.fn(),
      warn: vi.fn(),
      debug: vi.fn(),
    } as any;

    service = new ResultCollatorService(mockLogger);
  });

  it('should collate results from multiple batches', async () => {
    const batchResults = [
      {
        batchNumber: 1,
        pageNumbers: [1, 2, 3],
        success: true,
        classification: {
          forms: [
            {
              page_start: 1,
              page_end: 3,
              form_type: 'Form 1040',
              confidence: 0.95,
            },
          ],
        },
      },
      {
        batchNumber: 2,
        pageNumbers: [4, 5, 6],
        success: true,
        classification: {
          forms: [
            {
              page_start: 4,
              page_end: 6,
              form_type: 'Form 1040',
              confidence: 0.93,
            },
          ],
        },
      },
    ];

    const result = await service.collateResults(batchResults);

    expect(result.is_multi_form).toBe(false);
    expect(result.num_pages).toBe(6);
    expect(result.forms).toHaveLength(1);
    expect(result.forms[0].form_type).toBe('Form 1040');
    expect(result.forms[0].page_start).toBe(1);
    expect(result.forms[0].page_end).toBe(6);
    expect(result.classified_in_chunks).toBe(true);
    expect(result.num_chunks).toBe(2);
  });

  it('should handle failed batches', async () => {
    const batchResults = [
      {
        batchNumber: 1,
        pageNumbers: [1, 2, 3],
        success: true,
        classification: {
          forms: [
            {
              page_start: 1,
              page_end: 3,
              form_type: 'Schedule C',
              confidence: 0.95,
            },
          ],
        },
      },
      {
        batchNumber: 2,
        pageNumbers: [4, 5, 6],
        success: false,
        classification: null,
        error: 'LLM timeout',
      },
    ];

    const result = await service.collateResults(batchResults);

    expect(result.num_pages).toBe(6);
    expect(result.forms.length).toBeGreaterThan(0);
  });
});
```

### Integration Tests

```typescript
// tests/integration/routes/classifications.routes.test.ts
import { describe, it, expect, beforeAll, afterAll } from 'vitest';
import { FrameworkFastify } from 'framework-nodejs-fastify';
import fs from 'fs';
import path from 'path';

describe('POST /api/v1/Classifications', () => {
  let app: FrameworkFastify;
  let testPdfBuffer: Buffer;

  beforeAll(async () => {
    // Initialize test app
    app = await FrameworkFastify.create({
      /* test config */
    });
    await app.start(0); // Random port

    // Load test PDF
    testPdfBuffer = fs.readFileSync(
      path.join(__dirname, '../../fixtures/sample.pdf')
    );
  });

  afterAll(async () => {
    await app.stop();
  });

  it('should create classification job for valid PDF', async () => {
    const response = await app.server.inject({
      method: 'POST',
      url: '/api/v1/Classifications',
      payload: {
        file: testPdfBuffer,
        classificationPrompt: 'Classify this document',
      },
      headers: {
        'content-type': 'multipart/form-data',
        authorization: 'Bearer test-token',
      },
    });

    expect(response.statusCode).toBe(201);
    const body = JSON.parse(response.body);
    expect(body.jobId).toBeDefined();
    expect(body.status).toBe('in-progress');
    expect(body.statusUrl).toContain(`/api/v1/Classifications/${body.jobId}`);
    expect(response.headers.location).toContain('/api/v1/Classifications/');
  });

  it('should return 400 for missing file', async () => {
    const response = await app.server.inject({
      method: 'POST',
      url: '/api/v1/Classifications',
      payload: {
        classificationPrompt: 'Classify this document',
      },
      headers: {
        authorization: 'Bearer test-token',
      },
    });

    expect(response.statusCode).toBe(400);
    const body = JSON.parse(response.body);
    expect(body.error).toBe('MISSING_FILE');
  });

  it('should return 400 for non-PDF file', async () => {
    const response = await app.server.inject({
      method: 'POST',
      url: '/api/v1/Classifications',
      payload: {
        file: Buffer.from('not a pdf'),
        classificationPrompt: 'Classify this document',
      },
      headers: {
        'content-type': 'multipart/form-data',
        authorization: 'Bearer test-token',
      },
    });

    expect(response.statusCode).toBe(400);
    const body = JSON.parse(response.body);
    expect(body.error).toBe('INVALID_FILE_TYPE');
  });

  it('should return 401 for missing authentication', async () => {
    const response = await app.server.inject({
      method: 'POST',
      url: '/api/v1/Classifications',
      payload: {
        file: testPdfBuffer,
        classificationPrompt: 'Classify this document',
      },
      headers: {
        'content-type': 'multipart/form-data',
      },
    });

    expect(response.statusCode).toBe(401);
  });
});

describe('GET /api/v1/Classifications/:jobId', () => {
  let app: FrameworkFastify;

  beforeAll(async () => {
    app = await FrameworkFastify.create({
      /* test config */
    });
    await app.start(0);
  });

  afterAll(async () => {
    await app.stop();
  });

  it('should return correct result format on success', async () => {
    const jobId = '550e8400-e29b-41d4-a716-446655440000';

    const response = await app.server.inject({
      method: 'GET',
      url: `/api/v1/Classifications/${jobId}`,
      headers: {
        authorization: 'Bearer test-token',
      },
    });

    expect(response.statusCode).toBe(200);
    const body = JSON.parse(response.body);
    
    expect(body.jobId).toBe(jobId);
    expect(body.status).toBe('success');
    expect(body.result).toBeDefined();
    expect(body.result.is_multi_form).toBeDefined();
    expect(body.result.num_pages).toBeGreaterThan(0);
    expect(body.result.forms).toBeInstanceOf(Array);
    expect(body.result.classified_in_chunks).toBe(true);
    expect(body.result.num_chunks).toBeGreaterThan(0);
    
    // Validate form structure
    if (body.result.forms.length > 0) {
      const firstForm = body.result.forms[0];
      expect(firstForm.form_type).toBeDefined();
      expect(firstForm.confidence).toBeGreaterThan(0);
      
      // Form should have either page_start/page_end OR page_ranges
      const hasPageRange = firstForm.page_start !== undefined && firstForm.page_end !== undefined;
      const hasPageRanges = firstForm.page_ranges !== undefined && Array.isArray(firstForm.page_ranges);
      expect(hasPageRange || hasPageRanges).toBe(true);
    }
  });

  it('should return 404 for non-existent job', async () => {
    const response = await app.server.inject({
      method: 'GET',
      url: '/api/v1/Classifications/00000000-0000-0000-0000-000000000000',
      headers: {
        authorization: 'Bearer test-token',
      },
    });

    expect(response.statusCode).toBe(404);
    const body = JSON.parse(response.body);
    expect(body.error).toBe('JOB_NOT_FOUND');
  });
});
```

### Test Coverage Requirements

| Component    | Coverage Target |
| ------------ | --------------- |
| Services     | ≥ 90%           |
| Repositories | ≥ 85%           |
| Routes       | ≥ 80%           |
| Utilities    | ≥ 95%           |
| Overall      | ≥ 85%           |

---

## Sequence Diagrams

### Classification Job Creation Flow

```
Client                API Server           Blob Storage         Service Bus          Database
  |                        |                    |                    |                   |
  |-- POST /api/v1/Classifications ->|                    |                    |                   |
  |                        |                    |                    |                   |
  |                        |-- Validate --------|                    |                   |
  |                        |                    |                    |                   |
  |                        |-- Upload PDF ----->|                    |                   |
  |                        |<-- Success --------|                    |                   |
  |                        |                    |                    |                   |
  |                        |-- Create Job ------|--------------------|------------------->|
  |                        |                    |                    |<-- Job Created --|
  |                        |                    |                    |                   |
  |                        |-- Send Message ----|--------------------|------------------->|
  |                        |<-- Message Sent ---|                    |                   |
  |                        |                    |                    |                   |
  |<-- 201 Created --------|                    |                    |                   |
  |    (jobId, statusUrl)  |                    |                    |                   |
```

### Background Processing Flow

```
Service Bus          Worker            PDF Splitter      Image Converter    LLM API          Result Collator      Database           Blob Storage
      |                |                    |                  |                |                  |                   |                   |
      |-- Message ---->|                    |                  |                |                  |                   |                   |
      |                |                    |                  |                |                  |                   |                   |
      |                |-- Download PDF ----|------------------|-------------------------------------------------|------------------>|
      |                |<-- PDF ------------|------------------|-------------------------------------------------|-------------------|
      |                |                    |                  |                |                  |                   |                   |
      |                |-- Split PDF ------>|                  |                |                  |                   |                   |
      |                |<-- Pages ----------|                  |                |                  |                   |                   |
      |                |                    |                  |                |                  |                   |                   |
      |                |-- Convert to Images|----------------->|                |                  |                   |                   |
      |                |<-- Images ---------|------------------|                |                  |                   |                   |
      |                |                    |                  |                |                  |                   |                   |
      |                |-- Batch 1 (20 imgs)|------------------|--------------->|                  |                   |                   |
      |                |<-- Classification --|------------------|----------------|                  |                   |                   |
      |                |                    |                  |                |                  |                   |                   |
      |                |-- Batch 2 (20 imgs)|------------------|--------------->|                  |                   |                   |
      |                |<-- Classification --|------------------|----------------|                  |                   |                   |
      |                |                    |                  |                |                  |                   |                   |
      |                |-- Collate Results --|------------------|--------------------------------------------------->|                   |
      |                |<-- Final Result ---|------------------|----------------------------------------------------|                   |
      |                |                    |                  |                |                  |                   |                   |
      |                |-- Save Result ------|------------------|-------------------------------------------------|------------------>|
      |                |<-- Saved -----------|------------------|-------------------------------------------------|-------------------|
      |                |                    |                  |                |                  |                   |                   |
      |                |-- Update Job Status|------------------|-------------------------------------------------|------------------>|
      |                |<-- Updated ---------|------------------|-------------------------------------------------|-------------------|
```

### Status Polling Flow

```
Client              API Server         Database          Blob Storage
  |                      |                  |                  |
  |-- GET /api/v1/Classifications/{jobId} -->|                  |                  |
  |                      |                  |                  |
  |                      |-- Query Job ---->|                  |
  |                      |<-- Job Data -----|                  |
  |                      |                  |                  |
  |                      |-- Get Result ----|----------------->|
  |                      |<-- Result -------|------------------|
  |                      |                  |                  |
  |<-- 200 OK -----------|                  |                  |
  |    (status + result) |                  |                  |
```

---

## Database Schema

### ClassificationJobs Table

```sql
CREATE TABLE ClassificationJobs (
    id INT IDENTITY(1,1) PRIMARY KEY,
    jobId NVARCHAR(50) NOT NULL UNIQUE,
    userId NVARCHAR(100),
    fileName NVARCHAR(255) NOT NULL,
    fileSizeBytes BIGINT NOT NULL,
    pdfBlobPath NVARCHAR(500) NOT NULL,
    classificationPrompt NVARCHAR(MAX) NOT NULL,
    status NVARCHAR(20) NOT NULL, -- 'in-progress', 'success', 'failed'
    totalPages INT,
    processedPages INT,
    resultBlobPath NVARCHAR(500),
    errorCode NVARCHAR(50),
    errorMessage NVARCHAR(1000),
    errorDetails NVARCHAR(MAX),
    createdAt DATETIME2 NOT NULL DEFAULT GETUTCDATE(),
    updatedAt DATETIME2 NOT NULL DEFAULT GETUTCDATE(),
    completedAt DATETIME2,
    failedAt DATETIME2,
    INDEX IX_ClassificationJobs_JobId (jobId),
    INDEX IX_ClassificationJobs_Status (status),
    INDEX IX_ClassificationJobs_CreatedAt (createdAt)
);
```

### ClassificationResults Table

```sql
CREATE TABLE ClassificationResults (
    id INT IDENTITY(1,1) PRIMARY KEY,
    jobId NVARCHAR(50) NOT NULL,
    classificationData NVARCHAR(MAX) NOT NULL, -- JSON blob
    createdAt DATETIME2 NOT NULL DEFAULT GETUTCDATE(),
    FOREIGN KEY (jobId) REFERENCES ClassificationJobs(jobId),
    INDEX IX_ClassificationResults_JobId (jobId)
);
```

---

## Application Entry Point

```typescript
// src/index.ts
import 'dotenv/config';
import { FrameworkFastify } from 'framework-nodejs-fastify';
import { LogLevelEnum, DependencyKeys } from 'framework-nodejs-extensions';
import { AppInsightsLogConfigProfile } from 'framework-nodejs-logging';
import { DefaultAzureCredential } from '@azure/identity';
import { registerDependencies } from './shared/config/dependency-config';
import { classificationsRoutes } from './features/classifications/v1/routes/classifications.routes';
import { errorHandlerPlugin } from './shared/middleware/error-handler';
import { ClassificationWorker } from './features/classifications/v1/workers/classification-worker';

async function main() {
  // Create framework instance
  const framework = await FrameworkFastify.create({
    healthCheck: {
      enabled: true,
      path: '/health',
    },
    telemetryOptions: {
      serviceName: 'fluxx-general-classifier',
      serviceNamespace: 'production',
      enableLiveMetrics: true,
    },
    loggerOptions: {
      level: LogLevelEnum.info,
      config: new AppInsightsLogConfigProfile(
        process.env.APPLICATIONINSIGHTS_CONNECTION_STRING!,
        new DefaultAzureCredential()
      ),
    },
    serverOptions: {
      caseSensitive: false,
      ignoreTrailingSlash: true,
      bodyLimit: 52428800, // 50 MB
    },
  });

  // Initialize configuration
  await framework.initAppConfig({
    useEnvironmentVariables: true,
    azureAppConfigOptions: {
      useManagedIdentity: true,
      endpoint: process.env.FRAMEWORK_AZURE_APP_CONFIG_ENDPOINT,
    },
    fileConfigPath: './shared/config/app-config.json',
  });

  // Enable middleware
  await framework.enableMiddleware();

  // Enable CORS
  framework.enableCORS({
    origins: [
      process.env.ALLOWED_ORIGIN_1,
      process.env.ALLOWED_ORIGIN_2,
    ].filter(Boolean),
    methods: ['GET', 'POST'],
    allowedHeaders: ['Content-Type', 'Authorization'],
  });

  // Enable authentication
  framework.enableAuthentication();

  // Enable CSRF protection
  framework.enableCSRFProtection();

  // Initialize Swagger
  framework.initializeSwagger({
    specificationPath: './docs/openapi.yaml',
    swaggerUIConfig: {
      routePrefix: '/api-docs',
    },
  });

  // Register dependencies
  registerDependencies(framework);

  // Register error handler
  await framework.server.register(errorHandlerPlugin);

  // Register routes
  const classificationService = framework.dependencies.resolve('classificationService');
  const jobStatusService = framework.dependencies.resolve('jobStatusService');
  const logger = framework.dependencies.resolve(DependencyKeys.Logger);

  await framework.server.register(classificationsRoutes, {
    classificationService,
    jobStatusService,
    logger,
  });

  // Start background worker
  const worker = framework.dependencies.resolve<ClassificationWorker>('classificationWorker');
  await worker.start();

  // Graceful shutdown
  framework.onShutdown.on('onShutdown', async () => {
    logger.info('Stopping classification worker');
    await worker.stop();
  });

  // Start server
  const port = process.env.FRAMEWORK_HTTP_PORT || 3000;
  await framework.start(Number(port));

  logger.info(`fluxx-general-classifier started on port ${port}`);
}

main().catch((error) => {
  console.error('Failed to start application:', error);
  process.exit(1);
});
```

---

## Deployment

### Dockerfile

```dockerfile
# Build stage
FROM node:20-alpine AS builder

WORKDIR /app

COPY package*.json ./
RUN npm ci

COPY . .
RUN npm run build

# Production stage
FROM node:20-alpine

# Install Ghostscript (required for pdf2pic)
RUN apk add --no-cache ghostscript graphicsmagick

WORKDIR /app

COPY package*.json ./
RUN npm ci --production

COPY --from=builder /app/dist ./dist
COPY --from=builder /app/docs ./docs

ENV NODE_ENV=production
EXPOSE 3000

USER node

CMD ["node", "dist/index.js"]
```

### Kubernetes Deployment

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: fluxx-general-classifier
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: fluxx-general-classifier
  template:
    metadata:
      labels:
        app: fluxx-general-classifier
    spec:
      containers:
      - name: api
        image: myregistry.azurecr.io/fluxx-general-classifier:1.0.0
        ports:
        - containerPort: 3000
        env:
        - name: NODE_ENV
          value: "production"
        - name: FRAMEWORK_HTTP_PORT
          value: "3000"
        - name: FRAMEWORK_AZURE_APP_CONFIG_ENDPOINT
          valueFrom:
            configMapKeyRef:
              name: app-config
              key: azure-app-config-endpoint
        - name: APPLICATIONINSIGHTS_CONNECTION_STRING
          valueFrom:
            secretKeyRef:
              name: app-secrets
              key: app-insights-connection-string
        - name: OPENAI_API_KEY
          valueFrom:
            secretKeyRef:
              name: app-secrets
              key: openai-api-key
        resources:
          requests:
            memory: "1Gi"
            cpu: "500m"
          limits:
            memory: "2Gi"
            cpu: "2000m"
        livenessProbe:
          httpGet:
            path: /health
            port: 3000
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /health
            port: 3000
          initialDelaySeconds: 10
          periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
  name: fluxx-general-classifier
  namespace: production
spec:
  selector:
    app: fluxx-general-classifier
  ports:
  - protocol: TCP
    port: 80
    targetPort: 3000
  type: ClusterIP
```

---

**Document Owner**: Engineering Team\
**Reviewers**: Tech Lead, Senior Engineers\
**Implementation Start Date**: TBD\
**Target Completion Date**: TBD