Skip to navigation
Account Payable to Fluxx HLDDocuments AI Workflow Orchestrator

Workflow Stories Needed

This document outlines the specific work items needed to complete the /v1/invoice_extraction endpoint implementation, based on the Invoice Orchestrator User Stories.

Current Status ✅

The following components are COMPLETE:

  • ✅ Basic API structure (invoiceExtraction.route.ts, invoiceExtraction.controller.ts, invoiceExtraction.service.ts)

  • ✅ Database integration (invoiceExtraction.model.ts, utils/database.ts)

  • ✅ Status tracking with simplified response (complete, processing, failed)

  • ✅ Asynchronous processing architecture

  • ✅ Interface definitions and type safety

  • ✅ OpenAPI schema documentation

  • ✅ Route registration in main application

Required Implementation Work

CRITICAL - MVP Stories 🔥

Story 2: OpenAI Client Implementation (2 points)

Status: ⚠️ Needs Implementation

Location: src/api/v1/invoice_extraction/invoiceExtraction.service.ts - Line 357 (extractInvoice method)

Current Code:

// TODO: Implement actual OpenAI/AI extraction integration using client config
const mockExtractedData = { ... };

Required Implementation:

interface OpenAIClient {
  extractInvoiceData(imageUrls: string[], prompt: string, model: string): Promise<any>;
  retryExtraction(imageUrls: string[], retryPrompt: string): Promise<any>;
  consolidatePages(pageExtractions: any[], cprompt: string): Promise<any>;
}

Acceptance Criteria:

  • ✅ Create OpenAI API connection with proper authentication

  • ❌ Implement prompt template management system using clientConfig.ingest_config.openai_config

  • ❌ Add response validation and JSON parsing

  • ❌ Include error handling and retry logic

  • ❌ Support for all configured prompts (main, retry, consolidation)

Story 3: Azure Document Intelligence Client Implementation (2 points)

Status: ⚠️ Needs Implementation  

Location: src/api/v1/invoice_extraction/invoiceExtraction.service.ts - Line 302 (getBoundingRegions method)

Current Code:

// TODO: Implement actual Azure Document Intelligence integration
const mockBoundingRegions = [...];

Required Implementation:

interface DocumentIntelligenceClient {
  analyzePdf(imageUrls: string[]): Promise<DocumentAnalysisResult>;
  extractTables(imageUrls: string[]): Promise<TableData[]>;
  getBoundingRegions(imageUrls: string[]): Promise<BoundingRegion[]>;
  getDocumentStructure(imageUrls: string[]): Promise<DocumentStructure>;
}

Acceptance Criteria:

  • ❌ Create Form Recognizer API connection

  • ❌ Implement document analysis for invoices using image URLs

  • ❌ Add bounding box and table extraction

  • ❌ Include confidence scoring and validation

  • ❌ Support for custom invoice models

Story 4: Azure Blob Storage Client Implementation (2 points)

Status: ⚠️ Needs Implementation  

Location: src/api/v1/invoice_extraction/invoiceExtraction.service.ts - Line 254 (generateImages method)

Current Code:

// TODO: Implement actual PDF to image conversion
// This would require PDF.js or similar library and Azure Blob Storage upload
const mockImageUrls = [...];

Required Implementation:

interface BlobStorageClient {
  getBlobContent(containerName: string, blobName: string): Promise<Buffer>;
  uploadBlob(containerName: string, blobName: string, content: Buffer): Promise<string>;
  getSasUrl(containerName: string, blobName: string): Promise<string>;
  blobExists(containerName: string, blobName: string): Promise<boolean>;
}

Acceptance Criteria:

  • ❌ Create Azure Storage account connection

  • ❌ Download PDF from document.BlobPath

  • ❌ Implement blob upload/download with progress tracking

  • ❌ Add SAS token generation for secure image access

  • ❌ Support for cleanup and archival operations

Story 8: PDF to Image Conversion Pipeline (5 points)

Status: ⚠️ Needs Implementation  

Location: src/api/v1/invoice_extraction/invoiceExtraction.service.ts - Line 254 (generateImages method)

Current Code:

// TODO: Implement actual PDF to image conversion
// This would require PDF.js or similar library and Azure Blob Storage upload

Required Implementation:

interface PDFProcessor {
  convertPdfToImages(pdfBuffer: Buffer): Promise<Buffer[]>;
  uploadImagesToBlob(images: Buffer[], documentId: number): Promise<string[]>;
  generateImageSasUrls(imageBlobs: string[]): Promise<string[]>;
}

Acceptance Criteria:

  • ❌ Implement PDF-to-image conversion using PDF.js or similar library

  • ❌ Convert PDF pages to high-resolution images (300 DPI)

  • ❌ Upload images to Azure blob storage with proper naming convention

  • ❌ Generate SAS URLs for secure image access by AI services

  • ❌ Update workflow data with image URLs and prefixes

  • ❌ Add cleanup for temporary image files

Story 13: Freight Processing and Business Rules (3 points)

Status: ⚠️ Needs Implementation  

Location: src/api/v1/invoice_extraction/invoiceExtraction.service.ts - Line 507 (postProcessing method)

Current Code:

// TODO: Implement actual freight pattern matching logic
for (const pattern of freightPatterns) {
    console.log(`Processing freight pattern: ${JSON.stringify(pattern)}`);
}

Required Implementation:

interface FreightProcessor {
  applyFreightPatterns(invoiceData: InvoiceExtractionData, patterns: FreightPattern[]): Promise<InvoiceExtractionData>;
  convertFreightToLineItems(invoiceData: InvoiceExtractionData): Promise<InvoiceExtractionData>;
  validateBusinessRules(invoiceData: InvoiceExtractionData, clientConfig: ClientConfig): Promise<ValidationResult>;
}

Acceptance Criteria:

  • ❌ Implement freight pattern matching for vendor-specific rules

  • ❌ Process freight_patterns configuration from clientConfig.ingest_config

  • ❌ Convert freight charges to line items when configured

  • ❌ Apply client-specific business rules

  • ❌ Handle split_invoices and freight_as_line_item flags

SECONDARY - Post-MVP Stories 📋

Story 12: Data Fusion and Extraction Consolidation (3 points)

Location: src/api/v1/invoice_extraction/invoiceExtraction.service.ts - Line 480 (transformInvoice method)

Current Implementation: Basic confidence combination

Enhancement Needed: Intelligent conflict resolution between DI and OpenAI results

Story 11: Enhanced Invoice Data Mapping and Validation (3 points)

Location: src/api/v1/invoice_extraction/invoiceExtraction.service.ts - Line 415 (mapInvoice method)

Current Implementation: Basic mapping with ProcessedDocument

Enhancement Needed: Advanced validation and data quality scoring

Configuration Requirements

Current Client Configuration Structure

const extractionInput = {
  clientConfig: {
    ingest_config: {
      openai_config: {
        prompt: string;
        retry_prompt: string;
        cprompt: string;
        role: string;
        mprompt: string;
        mrole: string;
        model: string;
      };
      freight_patterns: {
        vendor: string;
        patterns: string[];
      }[];
      unique_identifier_fields: string[];
    };
    intake_config: {
      verify_po_number: boolean;
    };
  }
}

Required Azure Configuration

interface AzureConfig {
  blobStorage: {
    connectionString: string;
    containerName: string;
  };
  documentIntelligence: {
    endpoint: string;
    apiKey: string;
  };
  openai: {
    apiKey: string;
    endpoint?: string;
  };
}

Implementation Priority

Phase 1: Core AI Integration (MVP - Week 1-2)

  1. Story 4: Azure Blob Storage Client (PDF download)

  2. Story 8: PDF to Image Conversion Pipeline  

  3. Story 2: OpenAI Client Implementation

  4. Story 3: Azure Document Intelligence Client

Phase 2: Business Logic (MVP - Week 3)

  1. Story 13: Freight Processing and Business Rules

Phase 3: Enhancements (Post-MVP - Week 4+)

  1. Story 12: Data Fusion and Consolidation

  2. Story 11: Enhanced Data Validation

Testing Requirements

Unit Tests Needed

  • ✅ Route handler tests (basic structure exists)

  • ❌ OpenAI client integration tests

  • ❌ Azure Document Intelligence tests

  • ❌ PDF processing pipeline tests

  • ❌ Freight pattern processing tests

Integration Tests Needed

  • ❌ End-to-end extraction workflow

  • ❌ Error handling scenarios

  • ❌ Large document processing

  • ❌ Multiple client configuration scenarios

Dependencies

External Services

  • Azure Blob Storage: PDF and image storage

  • Azure Document Intelligence: OCR and structure analysis  

  • OpenAI GPT-4o: Invoice data extraction

  • SQL Server: ExtractedEntities table

Node.js Packages Needed

npm install @azure/storage-blob @azure/form-recognizer openai pdf-poppler

Completion Checklist

MVP Requirements

  • OpenAI Integration: Real AI extraction replacing mock data

  • Azure DI Integration: Real document analysis replacing mock regions

  • PDF Processing: Convert PDF to images and upload to blob storage

  • Freight Processing: Apply business rules from client configuration

  • Error Handling: Comprehensive error management and retry logic

Success Criteria

  • Endpoint accepts DocumentHash and returns ExtractedEntityId immediately

  • Background processing extracts real invoice data using AI services

  • Status endpoint returns accurate progress (processing, complete, failed)

  • Extracted data is properly stored in ExtractedEntities table

  • All client configuration options are respected

  • System handles errors gracefully without crashing

Total Estimated Points: 15 points (MVP) + 6 points (Post-MVP) = 21 story points