Skip to navigation
Account Payable to Fluxx HLDDocuments AI Workflow Orchestrator

Workflow implimenation

Overview

This document provides a comprehensive step-by-step analysis of the Azure Functions invoice processing orchestrator flow. The system processes invoices through multiple stages from initial ingestion to final ERP integration, with each stage handled by dedicated sub-orchestrators and activities.

System Architecture

The invoice processing system follows a hierarchical orchestrator pattern:

  • Main Orchestrator: invoiceOrchestrator - Controls overall workflow

  • Sub-Orchestrators: Handle specific processing stages

  • Activities: Perform atomic operations within each stage

Configuration Reference

The system operates based on client configuration that defines workflow stages, business rules, and processing parameters:

{
  "accountId": "A5555",
  "workflowId": "1234567",
  "pdfPath": "/path/to/pdf/file.pdf",
  "pdfPrefix": "file.pdf",
  "clientConfig": {
    "client_name": "default_client",
    "stages": {
      "ingest": {
        "is_entry": true,
        "workflow_name": "WorkflowInvoiceIngest",
        "continue_on_success": true
      },
      "intake": {
        "workflow_name": "WorkflowInvoiceIntake",
        "continue_on_success": true
      },
      "matching": {
        "workflow_name": "WorkflowInvoiceMatching",
        "continue_on_success": false
      },
      "manual": {
        "workflow_name": "WorkflowInvoiceManual",
        "continue_on_success": false
      },
      "filing": {
        "workflow_name": "WorkflowInvoiceFiling",
        "continue_on_success": false
      },
      "closed": {
        "is_exit": true,
        "workflow_name": "WorkflowInvoiceClosed"
      }
    },
    "match_config": {
      "AMOUNT_TOLERANCE": 0,
      "QUANTITY_TOLERANCE": 0,
      "UNIT_PRICE_TOLERANCE": 0
    }
  }
}

Database Schema

ExtractedEntities Table (MSSQL)

[ExtractedEntities] (
    [CreatedOn] DATETIMEOFFSET DEFAULT SYSDATETIMEOFFSET(),
    [DateExported] DATETIMEOFFSET,
    [DateFiled] DATETIMEOFFSET,
    [DocumentId] BIGINT NOT NULL,
    [ExtractedEntitiesPayload] NVARCHAR(MAX) NOT NULL,
    [ExtractedEntityId] BIGINT IDENTITY(1,1) PRIMARY KEY,
    [ModifiedOn] DATETIMEOFFSET DEFAULT SYSDATETIMEOFFSET(),
    [Stage] VARCHAR(50) NOT NULL,
    [Status] VARCHAR(50) NOT NULL,
    [Substatus] VARCHAR(50),
    FOREIGN KEY ([DocumentId]) REFERENCES [dbo].[Documents]([DocumentId])
)

Complete Workflow Steps

Main Orchestrator: invoiceOrchestrator

Step 1: Initialize Workflow

Function: Main orchestrator setup  

Functionality:

  • Calculates client ID hash from account ID

  • Initializes document record in database

  • Creates workflow data structure

  • Sets up initial workflow record

Implementation Status: 🔶 Partial  

TODO/Implementation Needs:

  • MSSQL Client: Need calculateClientIdHash activity to connect to MSSQL for client lookups

  • Database Activity: Need initializeDocument activity to insert into Documents table

  • Database Activity: Need createWorkflowRecordActivity to track workflow progress

Step 2: Entry Stage Detection

Function: Find workflow entry point  

Functionality:

  • Scans client configuration stages for is_entry: true

  • Sets current stage to ‘ingest’ (WorkflowInvoiceIngest)

  • Validates configuration structure

Implementation Status: ✅ Complete  

Notes: No additional implementation needed

Sub-Orchestrator 1: invoiceIngestOrchestrator (WorkflowInvoiceIngest)

Step 3: Initialize Ingest Workflow

Function: initializeWorkflow activity  

Functionality: Sets up workflow tracking for the ingest stage

Implementation Status: 🔶 Partial  

TODO/Implementation Needs:

  • Database Client: Connect to MSSQL to update workflow status

Step 4: Generate Invoice ID

Function: generateInvoiceId activity  

Functionality:

  • Downloads PDF from blob storage

  • Calculates SHA256 hash of PDF content (first 32 characters)

  • Creates unique invoice ID

  • Checks for duplicate documents in database

Implementation Status: 🔶 Partial  

TODO/Implementation Needs:

  • Azure Blob Client: Need getBlobContent() function to download PDF

  • MSSQL Client: Need queryDocuments() to check for duplicates

  • Crypto Utils: ✅ Already implemented with Node.js crypto module

Step 5: Generate Images from PDF

Function: generateImages activity  

Functionality:

  • Converts PDF pages to images (PNG/JPEG format)

  • Uploads images to Azure blob storage

  • Generates SAS URLs for image access

  • Updates workflow data with image URLs and prefixes

Implementation Status: ❌ Not Implemented  

TODO/Implementation Needs:

  • PDF Processing: Need PDF-to-image conversion library (PDF.js or similar)

  • Azure Blob Client: Need uploadBlob() and getSasUrl() functions

  • Image Processing: Convert PDF pages to images with proper resolution (300 DPI recommended)

Step 6: Extract Bounding Regions

Function: getBoundingRegions activity  

Functionality:

  • Sends images to Azure Document Intelligence

  • Extracts text regions, tables, and document structure

  • Saves DI results to blob storage as JSON

  • Provides bounding box coordinates for extracted elements

Implementation Status: ❌ Not Implemented  

TODO/Implementation Needs:

  • Azure Document Intelligence Client: Need analyzePdf() connection to Form Recognizer

  • Azure Blob Client: Store DI results as JSON blobs

  • Document Intelligence API: Configure form recognizer for invoice processing

Step 7: AI Invoice Extraction

Function: extractInvoice activity  

Functionality:

  • Sends PDF content to OpenAI with structured prompts

  • Uses configured OpenAI prompts from client config (openai_config)

  • Extracts structured invoice data (vendor info, line items, totals, etc.)

  • Handles retry logic with different prompts if initial extraction fails

  • Supports multi-page invoice processing with consolidation

Implementation Status: ❌ Not Implemented  

TODO/Implementation Needs:

  • OpenAI Client: Need connection to OpenAI API (GPT-4o model as configured)

  • Prompt Management: Use the configured prompts from clientConfig.ingest_config.openai_config

  • Message Bus: Send completion notification when extraction finishes

  • Retry Logic: Implement fallback to retry_prompt if initial extraction fails

OpenAI Configuration Details:

  • Model: GPT-4o

  • Role: Expert Accountant for ledger and tax filing

  • Primary Prompt: Extracts 25+ invoice fields including line items with detailed attributes

  • Retry Prompt: Simplified version for fallback processing

  • Consolidation Prompt: Combines multi-page extractions into single JSON

  • Special Prompts: Invoice number detection and line item description matching

Step 8: Map Invoice Data

Function: mapInvoice activity  

Functionality:

  • Maps OpenAI extracted JSON to standardized invoice structure

  • Validates required fields and data types

  • Handles field transformations and data type conversions

  • Ensures consistent data format for downstream processing

Implementation Status: ❌ Not Implemented  

TODO/Implementation Needs:

  • Data Validation: Implement field validation and transformation logic

  • Schema Mapping: Create mapping from OpenAI response to Invoice entity

  • Error Handling: Handle malformed or incomplete extraction results

Step 9: Transform and Combine Extractions

Function: transformInvoice activity  

Functionality:

  • Combines Document Intelligence and OpenAI extraction results

  • Resolves conflicts between different extraction methods

  • Creates final consolidated invoice data structure

  • Applies confidence scoring to choose best extraction for each field

Implementation Status: ❌ Not Implemented  

TODO/Implementation Needs:

  • Data Fusion Logic: Implement conflict resolution between DI and AI extractions

  • Quality Assessment: Score and weight different extraction sources

  • Consolidation Rules: Define rules for combining overlapping data

Step 10: Post-Processing

Function: postProcessing activity  

Functionality:

  • Handles freight/shipping line item processing based on vendor patterns

  • Applies client-specific business rules from configuration

  • Processes freight patterns for specific vendors

  • Prepares data for next stage (intake)

Implementation Status: 🔶 Partial  

TODO/Implementation Needs:

  • Business Rules Engine: Implement freight pattern matching from clientConfig

  • Vendor-Specific Logic: Handle freight_patterns configuration:

  - norsemandrilltool: patterns ['shipping', 'freight']

  - vestilmanufacturingcompany: patterns ['freight']

  • Line Item Processing: Convert freight charges to line items if freight_as_line_item: true

Step 11: Complete Ingest & Update Database

Function: completeWorkflow activity  

Functionality:

  • Updates ExtractedEntities table with final extraction results

  • Sets stage status to ‘succeeded-auto’

  • Triggers automatic transition to next stage (intake)

Implementation Status: ❌ Not Implemented  

TODO/Implementation Needs:

  • MSSQL Client: Insert/Update ExtractedEntities table with ExtractedEntitiesPayload JSON

  • Database Operations: Update DocumentId, Stage, Status, and ModifiedOn fields

  • JSON Serialization: Properly serialize extracted invoice data as NVARCHAR(MAX)

Sub-Orchestrator 2: invoiceIntakeOrchestrator (WorkflowInvoiceIntake)

Step 12: Invoice Intake Processing

Function: Invoice validation and intake processing  

Functionality:

  • Validates extracted invoice data quality and completeness

  • Performs PO number verification if configured (verify_po_number: true)

  • Checks business rules and data integrity

  • Updates stepper UI with intake results

  • Automatically continues to matching stage (continue_on_success: true)

Implementation Status: 🔶 Partial  

TODO/Implementation Needs:

  • Validation Logic: Implement invoice data quality checks

  • PO Validation: Connect to ERP system for PO number verification (PO numbers typically start with “400”)

  • Business Rules: Implement intake_config validation rules

  • Data Integrity: Validate line item calculations and totals

Sub-Orchestrator 3: invoiceMatchingOrchestrator (WorkflowInvoiceMatching)

Step 13: Get Purchase Order Data

Function: getPODataActivity  

Functionality:

  • Retrieves purchase order data from ERP system

  • Matches invoice PO number to existing ERP records

  • Loads PO line items for comparison

Implementation Status: ❌ Not Implemented  

TODO/Implementation Needs:

  • ERP Integration: Connect to ERP system APIs

  • PO Data Retrieval: Implement PO lookup logic by PO number

  • Data Caching: Cache PO data for performance

Step 14: Match Invoice to PO

Function: matchPOActivity  

Functionality:

  • Compares invoice line items to PO line items

  • Applies strict tolerance settings from match_config (all set to 0)

  • Identifies quantity, amount, and price variances

  • Flags discrepancies for manual review

Implementation Status: ❌ Not Implemented  

TODO/Implementation Needs:

  • Matching Engine: Implement tolerance-based matching using:

  - AMOUNT_TOLERANCE: 0 (exact amount matching required)

  - QUANTITY_TOLERANCE: 0 (exact quantity matching required)

  - UNIT_PRICE_TOLERANCE: 0 (exact price matching required)

  • Variance Detection: Identify and classify discrepancies

  • Matching Algorithms: Implement line-by-line and fuzzy matching logic

Step 15: Asset Auto-Fill

Function: assetAutoFillActivity  

Functionality:

  • Auto-fills asset/GL account codes based on line item descriptions

  • Uses historical data and ML models for code predictions

  • Assigns cost centers and accounting classifications

Implementation Status: ❌ Not Implemented  

TODO/Implementation Needs:

  • ML Model: Implement asset classification model

  • Historical Data: Connect to past invoice data for learning

  • GL Code Mapping: Implement chart of accounts integration

Step 16: Classify Line Items

Function: classifyLineItemsActivity  

Functionality:

  • Categorizes line items (expense, asset, inventory, etc.)

  • Assigns GL codes and cost centers automatically

  • Uses description matching and business rules

Implementation Status: ❌ Not Implemented  

TODO/Implementation Needs:

  • Classification Logic: Implement line item categorization algorithms

  • GL Code Assignment: Connect to chart of accounts

  • Business Rules: Implement client-specific classification rules

Note: Matching stage requires manual intervention (continue_on_success: false) due to strict tolerance settings.

Sub-Orchestrator 4: manualReviewOrchestrator (WorkflowInvoiceManual)

Step 17: Manual Review Processing

Function: Human review workflow for exceptions  

Functionality:

  • Holds workflow for manual review (continue_on_success: false)

  • Presents invoice and matching results to user

  • Waits for external event to continue workflow

  • Handles approve/reject decisions from reviewers

Implementation Status: ✅ Complete  

Notes:

  • Workflow waits for external signals (waitForExternalEvent)

  • UI integration required for user interaction

  • Supports both approve and reject resolution paths

Sub-Orchestrator 5: erpIntegrationOrchestrator (WorkflowInvoiceFiling)

Step 18: ERP Integration & Filing

Function: Export approved invoices to ERP system  

Functionality:

  • Formats invoice data for ERP consumption

  • Submits approved invoices to target ERP system

  • Handles posting and filing processes

  • Updates invoice status after successful export

  • Requires manual approval before proceeding (continue_on_success: false)

Implementation Status: ❌ Not Implemented  

TODO/Implementation Needs:

  • ERP Client: Connect to target ERP system APIs

  • Data Transformation: Format invoice data for ERP import

  • Integration Logic: Handle ERP-specific posting requirements

  • Error Handling: Manage failed exports and retry logic

  • Audit Trail: Track export status and timestamps

Sub-Orchestrator 6: workflowCompletionOrchestrator (WorkflowInvoiceClosed)

Step 19: Workflow Completion

Function: Final workflow closure and cleanup  

Functionality:

  • Marks workflow as completed in database

  • Updates final status and completion timestamps

  • Performs cleanup operations on temporary files

  • Archives processed documents

  • Generates completion metrics and reports

Implementation Status: 🔶 Partial  

TODO/Implementation Needs:

  • Cleanup Logic: Implement temporary file cleanup from blob storage

  • Archival: Move completed workflows to archive storage

  • Final Reporting: Generate completion metrics and audit logs

  • Database Updates: Set DateFiled and final status in ExtractedEntities table

Required Utility Clients

1. MSSQL Database Client

Purpose: Handle all database operations for workflow and document management

Required Functions:

interface DatabaseClient {
  calculateClientIdHash(accountId: string): Promise<string>;
  initializeDocument(pdfPath: string, clientId: string, workflowId: string): Promise<[string, string]>;
  createWorkflowRecord(workflowData: ProcessData): Promise<ProcessData>;
  updateItem(params: UpdateItemParams): Promise<any>;
  queryDocuments(criteria: QueryCriteria): Promise<Document[]>;
  storeDocument(dbName: string, doc: any): Promise<any>;
  updateWorkflowRecord(workflowData: ProcessData): Promise<void>;
  logWorkflowEvent(eventData: WorkflowEvent): Promise<void>;
}

TODO Implementation:

  • ❌ Connection string configuration

  • ❌ Error handling and retry logic  

  • ❌ Connection pooling

  • ❌ Transaction management

  • ❌ ExtractedEntities table operations

2. OpenAI Client

Purpose: Handle AI-powered invoice extraction using GPT-4o

Required Functions:

interface OpenAIClient {
  extractInvoiceData(pdfContent: Buffer, prompt: string): Promise<any>;
  retryExtraction(pdfContent: Buffer, retryPrompt: string): Promise<any>;
  consolidatePages(pageExtractions: any[], cprompt: string): Promise<any>;
  detectInvoiceNumber(pageContent: Buffer, sprompt: string): Promise<string>;
  compareDescriptions(invoiceDesc: string, poDesc: string, mprompt: string): Promise<boolean>;
}

Configuration Requirements:

  • Model: GPT-4o

  • Role: Expert Accountant

  • Custom prompts from client configuration

  • Retry logic with fallback prompts

  • Message bus notifications on completion

TODO Implementation:

  • ❌ OpenAI API connection setup

  • ❌ Prompt template management

  • ❌ Response validation and parsing

  • ❌ Error handling and retries

  • ❌ Multi-page consolidation logic

3. Azure Blob Storage Client

Purpose: Handle PDF and image storage operations

Required Functions:

interface BlobStorageClient {
  getBlobContent(containerName: string, blobName: string): Promise<Buffer>;
  uploadBlob(containerName: string, blobName: string, content: Buffer): Promise<string>;
  getSasUrl(containerName: string, blobName: string, permissions: string): Promise<string>;
  blobExists(containerName: string, blobName: string): Promise<boolean>;
  deleteBlob(containerName: string, blobName: string): Promise<void>;
}

TODO Implementation:

  • ❌ Azure Storage account connection

  • ❌ Container management

  • ❌ SAS token generation

  • ❌ Upload/download with progress tracking

  • ❌ Cleanup and archival operations

4. Azure Message Bus Client

Purpose: Handle asynchronous messaging for workflow events

Required Functions:

interface MessageBusClient {
  sendExtractionCompleteNotification(workflowId: string, results: any): Promise<void>;
  sendWorkflowStatusUpdate(workflowId: string, status: string): Promise<void>;
  sendErrorNotification(workflowId: string, error: Error): Promise<void>;
  subscribeToWorkflowEvents(callback: (event: WorkflowEvent) => void): Promise<void>;
}

TODO Implementation:

  • ❌ Service Bus connection setup

  • ❌ Topic and subscription management

  • ❌ Message serialization/deserialization

  • ❌ Error handling and dead letter queues

  • ❌ Event correlation and tracking

5. Azure Document Intelligence Client

Purpose: Handle OCR and document structure extraction

Required Functions:

interface DocumentIntelligenceClient {
  analyzePdf(pdfUrl: string): Promise<DocumentAnalysisResult>;
  extractTables(pdfUrl: string): Promise<TableData[]>;
  getBoundingRegions(pdfUrl: string): Promise<BoundingRegion[]>;
  getDocumentStructure(pdfUrl: string): Promise<DocumentStructure>;
}

TODO Implementation:

  • ❌ Form Recognizer API connection

  • ❌ Custom model training for invoices

  • ❌ Bounding box extraction

  • ❌ Table and text region identification

  • ❌ Confidence scoring and validation

Implementation Priority

Phase 1: Core Infrastructure

  1. MSSQL Database Client - Foundation for all workflow tracking

  2. Azure Blob Storage Client - Required for PDF/image operations

  3. Basic Activity Registration - Enable orchestrator execution

Phase 2: AI Processing

  1. OpenAI Client - Core extraction functionality

  2. Document Intelligence Client - Enhanced OCR capabilities

  3. PDF to Image Conversion - Required for processing pipeline

Phase 3: Integration & Optimization

  1. Message Bus Client - Async processing and notifications

  2. ERP Integration - Final export functionality

  3. Advanced Matching Logic - Tolerance-based comparison

  4. UI Integration - Manual review interfaces

Phase 4: Production Features

  1. Error Handling & Retry Logic - Production reliability

  2. Monitoring & Logging - Operational visibility

  3. Performance Optimization - Scale and efficiency

  4. Security Enhancements - Data protection

Configuration Impact

The provided client configuration has specific implications:

  • Strict Matching: All tolerance values set to 0 require exact matches

  • Manual Intervention: Multiple stages require human approval

  • PO Verification: Intake stage validates purchase order numbers

  • Freight Processing: Special handling for vendor-specific freight patterns

  • OpenAI Prompts: Comprehensive extraction with 25+ fields per invoice

  • Auto-Progression: Only ingest and intake stages continue automatically

This configuration ensures high accuracy but requires more manual oversight compared to more tolerant settings.