Skip to navigation
Account Payable to Fluxx HLDExtractors

TaxDoc Extractor Service

📘 Project Overview

Repository Name: poc-autoextract-platform-taxdoc-extractor-service

📌 Overview

The Document Extraction Service is a containerized Node.js application that listens to Azure Service Bus messages on the classified-file topic. Upon receiving a message, it orchestrates a local workflow to:

Download and prepare a PDF document from Azure Blob Storage. Submit the document to Azure Cognitive Services for analysis. Poll for the result. Publish a success message to the extracted-result topic. This replaces the Camunda BPMN-based orchestration with a lightweight, in-code orchestration model. The service is deployed to Azure Kubernetes Service (AKS) using Spacelift for infrastructure automation.

Follow the Framework V2 format for all components

🧩 Components

🔍 Low-Level Design (LLD)

1. Message Handling

  • Subscriber: Listens to classified-file topic.
  • Payload:
{
  "eventType": "file.classified",
  "fileId": "uuid",
  "parentUrl": "https://...",
  "childUrl": "https://...",
  "clientApplicationId": "uuid",
  "classifications": ["TaxDoc"]
}

2. Orchestration Flow

prepareDocument():

Downloads the PDF from childUrl. Filters pages based on docType mapping. Uploads the filtered PDF to a prepared folder.

extractDocument():

Streams the prepared PDF. Submits it to Azure Cognitive Services. Retrieves the operation-location.

pollForResult():

Polls the operation-location until status === succeeded. Extracts and stores the analyzeResult.

publishResult():

Sends a message to extracted-result topic:

{
  "eventType": "result.extracted",
  "fileId": "uuid",
  "clientApplicationId": "uuid",
  "resultUrl": "https://.../results/<fileId>",
  "timestamp": "ISO8601"
}

3. Error Handling

Centralized retry logic with exponential backoff. Graceful failure publishing (optional dead-letter queue). Logging and alerting for diagnostics.

🗂️ File Layout

📁 poc-autoextract-platform-taxdoc-extractor-service/
├── 📁 src/
│   ├── 📁 orchestrators/
│   │   └── documentOrchestrator.js         # Main orchestration logic
│   ├── 📁 services/
│   │   ├── blobStorageApiService.js        # Azure Blob Storage interaction
│   │   ├── cognitiveServicesApiService.js  # Submit and poll document analysis
│   │   └── serviceBusPublisher.js          # Publishes to 'extracted-result'
│   ├── 📁 subscribers/
│   │   └── classifiedFileSubscriber.js     # Listens to 'classified-file' topic
│   ├── 📁 tasks/
│   │   ├── prepareDocument.js              # Filters and uploads PDF
│   │   └── extractDocument.js              # Submits PDF to Cognitive Services
│   ├── 📁 utilities/
│   │   ├── errorHandling.js                # Retry and error logic
│   │   ├── mappingUtils.js                 # Maps docType to model/phrases
│   │   ├── pathUtils.js                    # Folder name helpers
│   │   └── constants.js                    # Shared constants
│   └── index.js                            # Entry point: initializes subscriber
├── 📁 config/
│   ├── default.json                        # Default config (topics, retries)
│   └── production.json                     # Production overrides
├── 📁 deployment/
│   ├── Dockerfile                          # Container definition
│   ├── deployment.yaml                     # Kubernetes deployment spec
│   ├── service.yaml                        # Kubernetes service spec (if needed)
│   └── spacelift-stack.yaml                # Spacelift stack configuration
├── .env                                    # Environment variables
├── .gitignore
├── package.json
└── README.md

🧪 Testing Strategy

  • Unit Tests: Jest for all service modules
  • Integration Tests: Mock Azure SDKs and Service Bus
  • E2E Tests: Simulate full message flow in AKS dev cluster

🚀 Deployment

  • CI/CD: GitHub Actions → Spacelift → AKS
  • Helm: Used for Kubernetes manifests
  • Spacelift:
    • Stack: aks-stack.yaml
    • Modules: azure-kubernetes/main.tf
    • Triggers: GitHub PRs and merges to main
    • Secrets: Managed via Spacelift + Azure Key Vault