> For clean Markdown of any page, append .md to the page URL.
> For a complete documentation index, see https://cla-demo.postmansolutions.com/llms.txt.
> For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://cla-demo.postmansolutions.com/_mcp/server.

# 1099B NonGovt Extractor Service

# 📄 1099B-NonGovt-Extractor-Service Design

## 📘 Project Overview

Repository Name: `poc-autoextract-platform-1099B-NonGovt-Extractor-Service`

## 🧭 Overview

The `1099B-NonGovt-Extractor-Service` is a cloud-native Node.js microservice deployed on Azure Kubernetes Service (AKS). It automates the extraction of structured data from 1099-B Non-Government tax documents using Azure Document Intelligence and OpenAI. The service is triggered by messages from Azure Service Bus and publishes results to a downstream topic once processing is complete. Follow the Framework V2 format for all components

### 🔄 Workflow Summary

1. **Trigger**: Listens to the `classified-file` topic on Azure Service Bus.

```
    {
      "eventType": "file.classified",
      "fileId": "uuid",
      "parentUrl": "https://...",
      "childUrl": "https://...",
      "clientApplicationId": "uuid",
      "classifications": ["1099NonGov"]
    }
```

2. **Download**: Retrieves the document from Azure Blob Storage.
3. **Analyze**: Uses Azure Document Intelligence and OpenAI to extract structured data.
4. **Merge**: Combines outputs from both services into a unified JSON format.
5. **Upload**: Stores the result back in Blob Storage.
6. **Notify**: Publishes a message to the `extracted-result` topic with a result URL.

```
    {
      "eventType": "result.extracted",
      "fileId": "uuid",
      "clientApplicationId": "uuid",
      "resultUrl": "https://.../results/<fileId>",
      "timestamp": "ISO8601"
    }
```

---

## 🧩 Component List

| Component                              | Description                                                                                            |
| -------------------------------------- | ------------------------------------------------------------------------------------------------------ |
| **Azure Service Bus Subscriber**       | Listens to the `classified-file` topic and initiates the extraction pipeline.                          |
| **Blob Downloader**                    | Downloads the original document from Azure Blob Storage.                                               |
| **PDF Parser**                         | Parses the PDF using a Node.js library (e.g., `pdf-lib` or `pdf2json`) to extract raw text and tables. |
| **Azure Document Intelligence Client** | Sends the document to Azure Form Recognizer for structured analysis.                                   |
| **OpenAI Client**                      | Sends extracted text to Azure OpenAI for LLM-based interpretation and validation.                      |
| **Result Merger**                      | Merges outputs from Document Intelligence and OpenAI into a single JSON structure.                     |
| **Blob Uploader**                      | Uploads the final result JSON to a designated Blob Storage container.                                  |
| **Azure Service Bus Publisher**        | Sends a message to the `extracted-result` topic with metadata and result URL.                          |
| **AKS Deployment**                     | Hosts the Node.js service in a scalable containerized environment.                                     |
| **Spacelift Automation**               | Manages infrastructure provisioning and deployment pipelines for AKS, Service Bus, and Key Vault.      |
| **Azure Key Vault + CSI Driver**       | Securely injects secrets (API keys, connection strings) into the AKS pods.                             |

---

## ⚙️ Infrastructure

* **Platform**: Azure Kubernetes Service (AKS)
* **Automation**: [https://spacelift.io/](https://spacelift.io/) (manages AKS, Azure Service Bus, Key Vault, and Helm deployments)
* **Secrets**: Azure Key Vault + CSI Driver
* **Observability**: Azure Monitor, Application Insights

---

## 📁 File Structure

1099B-NonGovt-Extractor-Service/
├── src/
│   ├── index.js
│   ├── config/
│   │   └── env.js
│   ├── messaging/
│   │   ├── subscriber.js
│   │   └── publisher.js
│   ├── services/
│   │   ├── blobService.js
│   │   ├── pdfParser.js
│   │   ├── documentAI.js
│   │   ├── openAIExtractor.js
│   │   └── resultMerger.js
│   └── utils/
│       └── logger.js
├── Dockerfile
├── helm/
│   └── chart.yaml
├── spacelift/
│   ├── stacks/
│   │   └── aks-stack.yaml
│   └── modules/
│       └── azure-kubernetes/
│           └── main.tf
├── package.json
└── README.md

---

## 🧩 Module Responsibilities

### `index.js`

* Entry point
* Initializes environment and starts the subscriber

### `config/env.js`

* Loads and validates environment variables

### `messaging/subscriber.js`

* Subscribes to `classified-file` topic
* Parses incoming payload:

### `messaging/publisher.js`

* Publishes to `extracted-result` topic:

### `services/blobService.js`

* Downloads and uploads blobs from Azure Blob Storage

### `services/pdfParser.js`

* Parses PDF using `pdf-lib` or `pdf2json`
* Extracts tables and text

### `services/documentAI.js`

* Calls Azure Document Intelligence with custom model

### `services/openAIExtractor.js`

* Sends prompt to Azure OpenAI
* Validates and refines LLM output

### `services/resultMerger.js`

* Merges Document AI and LLM results
* Formats final JSON output

### `utils/logger.js`

* Centralized logging with context and correlation IDs

---

## 🧪 Testing Strategy

* **Unit Tests**: Jest for all service modules
* **Integration Tests**: Mock Azure SDKs and Service Bus
* **E2E Tests**: Simulate full message flow in AKS dev cluster

---

## 🚀 Deployment

* **CI/CD**: GitHub Actions → Spacelift → AKS
* **Helm**: Used for Kubernetes manifests
* **Spacelift**:
  * Stack: `aks-stack.yaml`
  * Modules: `azure-kubernetes/main.tf`
  * Triggers: GitHub PRs and merges to `main`
  * Secrets: Managed via Spacelift + Azure Key Vault