> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://cla-demo.postmansolutions.com/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://cla-demo.postmansolutions.com/_mcp/server. # Test Plan for E2E Automation ## Objective The objective of this Test Plan is to validate the functionality, reliability, and performance of the tax document ingestion and processing platform through a comprehensive backend-focused E2E automation framework. The framework will integrate with and test backend APIs, Azure Service Bus (SB) flows, BLOB Storage operations, and SQL DB persistence across a phased rollout of supported AI extraction tools. The focus on backend automation - excluding the Document AI UI- stems from practical priorities: * **Increased Test Stability** in complex microservices environment * **Faster feedback** in CI/CD * **Reduced flakiness** due to browser or UI changes * **Reliability and repeatability** in headless environments (e.g., AKS) ## Scope ### In Scope * Fluxx Platform, Document AI, SaaS deployment * Upload, Scanning, and processing of various document types via Blob and/or API * Service Bus message flow validations and schema conformity * Extraction result comparison with Golden Samples (GS) for: * JSON Structure/Schema * Key-value data * Confidence score (on JSON) validation with threshold checks * SQL DB and blob storage persistence and metadata validation * TBD: Post-processing flows (External Blobs, Data Lake) integration with schema (JSON data) compliance * Performance and throughput analysis (files per minute, extraction latency) * E-mail file intake - TBD ### Out of Scope * Classifier Service * File splitting * File conversion * Post-Processing * Document AI UI interactions * Model accuracy tuning or training validations (owned by Data Science team) * UI-triggered-only flows (unless backend alternatives are available) ## Product Scope & Use Case Coverage | Phase | Input Type | Key Flow | Output/Validation | | ----- | --------------------------------------------------- | --------------------------------------- | ------------------------------------------------------------------------------------ | | 1 | Individual File Processing in Fluxx (Invoice files) | Upload -> Scan -> Extract -> Store | Extracted JSON, SB, messages, SQL read/write, blob storage | | 2 | Document AI as a client flow - mock UI | Upload -> Scan -> Extract -> Store | Extracted JSON, SB, messages, SQL read/write, blob storage | | 2 | Document AI as a client flow - email upload | File Intake -> Scan -> Extract -> Store | Extracted JSON, SB, messages, SQL read/write, blob storage, Complete Invoice process | ## End-to-End Test Flow | # | Function | Actions | Verification Approach | | -- | ----------------------------- | --------------------------------- | --------------------------------------------------------------------------------------------- | | 1 | Registration Service | Create Registration Endpoint POST | Verify records created | | 2 | Upload file | File-Intake POST | Triggers Blob Created msg: Submitted-File topic, Status Check service: created, file metadata | | 3 | File Validation | Scan, file type check | Status check service: Validated or failed validation | | 4 | Save to internal file storage | AZ blob storage | Verify internal storage, status check: copied or failed copying | | 5 | File-Intake Service | publishes SB msg | Verify SB msg: accepted-file topic | | 6 | Extractor | | Polling: isReady = True or SB msg? | | 7 | Extraction Stored | AZ SQL storage | Verify extracted data stored, Status Check: extracted | | 8 | Postal Service | Sends POST or SB msg | SB Msg: To-Send topic | | 9 | STatus check | verify completed | Status check service: completed | | 10 | Documents AI | extractor sends message to Fluxx | verify Files match | | 11 | Compare Data | Data compared to Golden Sample | Provide accuracy rating of extraction to Golden Sample | Steps in Flow 1. Registration Service 1. Create Registration Endpoint POST 2. Creates records in SQL DB 2. Upload file to blob (email, direct) 1. Triggers Blob Created Event - Submitted-File 2. Status Check service: created 3. File Validation 1. Scan, file type, quarantine if bad 2. Status Check service: validated OR 3. Status Check service: failed validation 4. Save to Internal file storage - AZ blob 1. Verify file present in internal storage bucket 2. Status Check service: copied OR 3. Status Check service: failed copying 5. File Intake file service 1. Publishes new file related data event to SB - accepted-file topic 6. Classify - OOS for MVP 1. Convert 2. Splitter 3. Process 7. Document Extractor(s) 1. External processing, polling for isReady: True 8. Raw Extract Data Storage - Az Blob 1. Status Check service: extracted 9. Post-Processing - OOS 10. Postal Service 11. To-Send topic created, sends http POST or SB message 12. Completion 13. Status Check Service: completed 14. UI support for current status - GET /status/files/\ ## Functional Requirement Verifications | # | Function | Verification Approach | | -- | --------------------------------- | ------------------------------------------------------------------------------------------------------------------- | | 1 | Blob Update | SB message, File presence, checksum, metadata validation | | 2 | Virus Scan | SB message, allow/block logic (stretch goal), logging | | 6 | **Extraction** | Pubish or polling, message (extraction start, complete), Extraction Results: Verify extracted JSON & compared to GS | | 7 | **Confidence Scores** | Threshold comparison vs GS with warning for drop | | 8 | NoSQL Persistence | FileId, metadata (all file and location info & process status updates etc., insert/update validation | | 9 | Access/Data Lake upload | Schema of payload to Acc. Tax & Data Lake, API Status codes | | 10 | Re-upload Same Doc ??? | Validate Overwrite, extraction re-trigger and SQL metadata updates (TBD) | | 12 | Documents AI - Invoice processing | TBD | ## Non-Functional Requirement Verifications (TBD) * Performance: Time to extract per file batch (10, 50, 100). Files from the primary test dataset used for functional testing * Throughput: Parallel upload handling, Blob lag * Reliability: Retry logic, SB message persistence and reprocessing validation * Observability: Timestamps for SB/Blob checkpoints * Scalability: Simulate high file count across doc types (TBD) ## Additional Information ### Integration with Azure Service Bus * Each processing module emits completion events via Azure Service Bus. * Test Framework subscribes to relevant SB topics/queues to: * Detect when file processing is complete * Trigger result verification against GS * Capture delays and issues ## Test Data * 5 PDF documents from APAI team will be uploaded and extracted * ClientId 66 in DocsAI is used * Client created during runtime with DocsAI Extractor details