> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://cla-demo.postmansolutions.com/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://cla-demo.postmansolutions.com/_mcp/server. # User Stories for Document AI Microservices Implementation ## Document AI Invoice File Intake Service **Service Name**: `docai_invoice_file_intake_service` **Called By**: `fluxx-file-intake` microservice ### Story 1: Database Integration **As a** developer **I want** to replace the mock database implementation with actual SQL Server connections **So that** the Document AI Invoice File Intake Service can persist documents to the production database **Acceptance Criteria:** * Replace `MockFileIntakeDatabaseService` with SQL Server implementation * Implement connection pooling and proper error handling * Support the actual Documents table schema with proper foreign key relationships * Handle database transactions for document creation and updates * Implement proper logging for database operations **Technical Tasks:** * Install and configure SQL Server driver (e.g., `mssql` or `tedious`) * Create `SqlServerFileIntakeDatabaseService` implementing `IFileIntakeDatabaseService` * Update service factory to use SQL Server implementation * Add connection string configuration management * Implement database connection health checks --- ### Story 2: Azure Blob Storage Integration **As a** developer **I want** to integrate with Azure Blob Storage for PDF file operations **So that** the service can validate and process PDF documents from blob storage **Acceptance Criteria:** * Integrate Azure Blob Storage client for PDF file access * Implement PDF file validation and metadata extraction * Support blob path validation and file existence checks * Handle Azure storage authentication and error scenarios * Implement blob content hashing for duplicate detection **Technical Tasks:** * Install and configure Azure Storage SDK (`@azure/storage-blob`) * Create blob storage utility functions for PDF operations * Implement PDF file validation logic * Add Azure storage configuration to service initialization * Create error handling for blob storage operations --- ### Story 3: Client Configuration and Testing **As a** developer **I want** comprehensive testing and client configuration validation **So that** the Document AI Invoice File Intake Service is production-ready with proper quality assurance **Acceptance Criteria:** * Create comprehensive unit tests for all service methods * Implement integration tests with actual database and blob storage * Add client configuration validation using actual config schema * Implement proper error handling and logging throughout the service * Create performance benchmarks and monitoring capabilities **Technical Tasks:** * Set up Jest/Vitest testing framework with appropriate mocks * Create unit tests for each service method with edge cases * Implement integration tests with test database * Add client configuration validation logic using provided schema * Implement structured logging with correlation IDs * Create performance monitoring and metrics collection --- ## Document AI Invoice Extraction Service **Service Name**: `docai_invoice_extraction_service` **Called By**: `fluxx-extraction-service` microservice ### Story 1: Azure Services Integration **As a** developer **I want** to integrate with Azure Document Intelligence and OpenAI APIs **So that** the Document AI Invoice Extraction Service can perform OCR and AI-based data extraction **Acceptance Criteria:** * Integrate Azure Document Intelligence for PDF OCR and bounding region extraction * Implement OpenAI API integration using client-specific configurations * Support multiple AI models and prompts based on client configuration * Handle API rate limiting, retries, and error scenarios * Implement confidence scoring and extraction validation **Technical Tasks:** * Install Azure Document Intelligence SDK (`@azure/ai-form-recognizer`) * Install OpenAI SDK (`openai`) * Implement Document Intelligence client with proper authentication * Create OpenAI client with client-specific prompt handling * Implement retry logic and rate limiting for both services * Add API response validation and error handling --- ### Story 2: PDF Processing and Image Generation **As a** developer **I want** to implement PDF to image conversion and processing **So that** the service can generate images for OCR and AI analysis **Acceptance Criteria:** * Convert PDF pages to high-quality images suitable for OCR * Support multi-page PDF processing with proper page sequencing * Implement image optimization for AI processing * Store generated images in Azure Blob Storage with proper naming * Handle PDF processing errors and edge cases (corrupted files, large files) **Technical Tasks:** * Install PDF processing library (`pdf2pic` or `pdf-poppler`) * Implement PDF to image conversion with quality optimization * Create image storage utilities for Azure Blob Storage * Add support for multi-page PDF processing * Implement proper cleanup of temporary files and images * Add error handling for PDF corruption and processing failures --- ### Story 3: Database Integration and Freight Processing **As a** developer **I want** to integrate with SQL Server and implement freight pattern processing **So that** the service can persist extracted data and apply client-specific business rules **Acceptance Criteria:** * Replace mock database with actual SQL Server implementation for ExtractedEntities table * Implement freight pattern matching using client configuration * Support complex invoice data transformation and validation * Handle database transactions for extraction results * Implement post-processing business logic for freight, taxes, and line items **Technical Tasks:** * Create `SqlServerExtractionDatabaseService` implementing `IExtractionDatabaseService` * Implement freight pattern matching algorithm using regex patterns from client config * Add invoice data transformation and validation logic * Create complex JSON payload handling for ExtractedEntitiesPayload column * Implement database transaction management for extraction operations * Add business logic for freight calculations and tax processing --- ### Story 4: Testing and Production Readiness **As a** developer **I want** comprehensive testing and monitoring for the Document AI Invoice Extraction Service **So that** the service is reliable and maintainable in production **Acceptance Criteria:** * Create unit tests for all extraction pipeline methods * Implement integration tests with actual Azure services (using test accounts) * Add performance testing for large PDF processing * Implement comprehensive error handling and logging * Create monitoring and alerting for extraction failures and performance **Technical Tasks:** * Set up testing framework with Azure service mocks * Create unit tests for AI extraction, PDF processing, and database operations * Implement integration tests with test Azure accounts * Add performance testing for various PDF sizes and complexity * Implement structured logging with extraction correlation IDs * Create health checks and monitoring endpoints * Add alerting for extraction failures and performance degradation --- ## Cross-Service Stories ### Story 1: Service Communication and Integration **As a** developer **I want** proper communication between Document AI services **So that** the services work together seamlessly in the Fluxx microservice architecture **Acceptance Criteria:** * Define clear API contracts between services * Implement proper service discovery and communication * Handle service failures and retry logic * Support asynchronous processing with proper status tracking * Implement end-to-end tracing and monitoring **Technical Tasks:** * Define service API interfaces and communication protocols * Implement HTTP/REST APIs or message queue integration * Add service health checks and discovery mechanisms * Create retry and circuit breaker patterns for service communication * Implement distributed tracing across services * Add end-to-end monitoring and alerting --- ### Story 2: Configuration Management and Deployment **As a** developer **I want** proper configuration management and deployment pipelines **So that** both Document AI services can be deployed and managed efficiently in production **Acceptance Criteria:** * Implement environment-based configuration management * Create Docker containers for both services * Set up CI/CD pipelines for automated deployment * Implement proper secret management for API keys and connection strings * Support blue-green or canary deployment strategies **Technical Tasks:** * Create environment-specific configuration files * Implement configuration validation and loading * Create Dockerfiles and docker-compose configurations * Set up CI/CD pipelines (GitHub Actions/Azure DevOps) * Integrate Azure Key Vault for secret management * Implement deployment automation and rollback procedures --- ### Story 3: Monitoring and Observability **As a** developer **I want** comprehensive monitoring and observability across both Document AI services **So that** we can track performance, debug issues, and ensure service health **Acceptance Criteria:** * Implement structured logging with correlation IDs * Create performance metrics and dashboards * Set up alerting for critical failures and performance degradation * Implement health checks and service status monitoring * Support distributed tracing across the entire processing pipeline **Technical Tasks:** * Integrate logging framework (Winston, Pino) with structured output * Set up metrics collection (Prometheus, Application Insights) * Create monitoring dashboards (Grafana, Azure Monitor) * Implement alerting rules and notification channels * Add health check endpoints for both services * Configure distributed tracing (Jaeger, Application Insights) --- ## Priority Matrix ### High Priority (MVP) 1. **Document AI Invoice File Intake Service - Database Integration** (Story 1) 2. **Document AI Invoice Extraction Service - Azure Services Integration** (Story 1) 3. **Document AI Invoice Extraction Service - Database Integration** (Story 3) ### Medium Priority (Production Ready) 4. **Document AI Invoice File Intake Service - Azure Blob Storage Integration** (Story 2) 5. **Document AI Invoice Extraction Service - PDF Processing** (Story 2) 6. **Cross-Service Communication** (Story 1) ### Lower Priority (Operational Excellence) 7. **Document AI Invoice File Intake Service - Testing** (Story 3) 8. **Document AI Invoice Extraction Service - Testing** (Story 4) 9. **Configuration Management** (Story 2) 10. **Monitoring and Observability** (Story 3) --- ## Estimated Effort \| Story | Complexity | Estimated Story Points | \|-------|------------|----------------------| \| DocAI File Intake - Database Integration | Medium | 5 | \| DocAI File Intake - Blob Storage Integration | Medium | 5 | \| DocAI File Intake - Testing & Config | High | 8 | \| DocAI Extraction - Azure Services Integration | High | 13 | \| DocAI Extraction - PDF Processing | Medium | 8 | \| DocAI Extraction - Database & Freight | High | 13 | \| DocAI Extraction - Testing | High | 8 | \| Cross-Service Communication | Medium | 5 | \| Configuration Management | Medium | 5 | \| Monitoring & Observability | High | 8 | **Total Estimated Effort: 78 Story Points**