> For clean Markdown of any page, append .md to the page URL. > For a complete documentation index, see https://cla-demo.postmansolutions.com/llms.txt. > For AI client integration (Claude Code, Cursor, etc.), connect to the MCP server at https://cla-demo.postmansolutions.com/_mcp/server. # Extracted Entities---Streaming Invoice Images and Bounding Regions Endpoints ## Overview This document outlines the implementation stories for two new endpoints that will stream invoice images directly and provide access to bounding region data associated with extracted entities. These endpoints will improve security by eliminating the need for client-side token management and optimize data storage by storing persistent blob URLs instead of time-limited URLs in the database. \--- ## Background & Context ### Current State * The `ExtractedEntities` table currently stores image URLs and bounding region URLs * These URLs may be stored with query parameters (expiring tokens) or as persistent blob URLs * There's no dedicated endpoint to stream images directly to the client * There's no dedicated endpoint to fetch bounding region JSON data from blob storage ### Business Value * **Security**: Stream images directly from API without exposing storage credentials or tokens * **Data Integrity**: Store persistent blob URLs instead of expiring URLs in the database * **Flexibility**: Separate concerns between data storage and data access * **Client Experience**: Provide clients with ready-to-use binary image data without manual token or URL management ## Story 1: Stream Invoice Images ### Story Description **As a** client application consuming invoice extraction data **I want to** stream invoice images directly from the API for an extracted entity **So that** I can display images in the UI without managing blob storage authentication or token expiration ### Technical Details #### Endpoint Specification ``` GET /v1/extracted_entities/:entityId/images   GET /v1/extracted_entities/:entityId/images/:pageNumber   ``` #### Request Parameters \| Parameter | Type | Required | Description | \|-----------|------|----------|-------------| \| `entityId` | integer | Yes | The ExtractedEntityId | \| `pageNumber` | integer | No | Specific page number to retrieve (omit to get all pages) | #### Response Schema (200 OK) **Single Page Request** (`GET /v1/extracted_entities/:entityId/images/:pageNumber`): ``` Content-Type: image/png (or image/jpeg)   Content-Disposition: inline; filename="page_1.png"   X-Page-Number: 1   X-Total-Pages: 5   [Binary image data]   ``` **Multiple Pages Request** (`GET /v1/extracted_entities/:entityId/images`): ``` Content-Type: multipart/mixed; boundary=----ImageBoundary   ------ImageBoundary   Content-Type: image/png   Content-Disposition: attachment; filename="page_1.png"   X-Page-Number: 1   [Binary image data for page 1]   ------ImageBoundary   Content-Type: image/png   Content-Disposition: attachment; filename="page_2.png"   X-Page-Number: 2   [Binary image data for page 2]   ------ImageBoundary--   ``` #### Error Responses \| Status Code | Scenario | Response Body | \|-------------|----------|---------------| \| 404 | Entity not found | `{ "error": "Entity not found" }` | \| 404 | No images found | `{ "error": "No images found for this entity" }` | \| 404 | Page number out of range | `{ "error": "Page number exceeds available pages" }` | \| 500 | Blob download error | `{ "error": "Failed to retrieve images from storage" }` | \| 500 | Internal error | `{ "error": "Internal Server Error" }` | ### Implementation Tasks #### Task 1.1: Update Type Definitions **File**: `src/api/v1/extractedEntity/extractedEntities.types.ts` ```typescript   export interface ImageMetadata {     pageNumber: number;     blobName: string;     container: string;     contentType: string;     fileName: string;   }   export interface ImageStreamInfo {     entityId: number;     images: ImageMetadata[];     totalImages: number;   }   ``` #### Task 1.2: Create Schema Definition **File**: `src/api/v1/extractedEntity/extractedEntities.schema.ts` Add new schemas: ```typescript streamImages: {     security: [{ bearerAuth: [] }],     params: {       type: 'object',       properties: {         entityId: { type: 'integer', description: 'Extracted Entity ID' }       },       required: ['entityId']     },     tags: ['Extracted Entities'],     response: {       200: {         description: 'Multipart response with image streams',         content: {           'multipart/mixed': {             schema: {               type: 'string',               format: 'binary'             }           }         },         headers: {           'X-Total-Pages': {             schema: { type: 'integer' },             description: 'Total number of image pages'           }         }       },       404: {         type: 'object',         properties: {           error: { type: 'string' }         }       }     }   },   streamSingleImage: {     security: [{ bearerAuth: [] }],     params: {       type: 'object',       properties: {         entityId: { type: 'integer', description: 'Extracted Entity ID' },         pageNumber: { type: 'integer', description: 'Page number to retrieve' }       },       required: ['entityId', 'pageNumber']     },     tags: ['Extracted Entities'],     response: {       200: {         description: 'Binary image data',         content: {           'image/png': {             schema: {               type: 'string',               format: 'binary'             }           },           'image/jpeg': {             schema: {               type: 'string',               format: 'binary'             }           }         },         headers: {           'X-Page-Number': {             schema: { type: 'integer' },             description: 'Current page number'           },           'X-Total-Pages': {             schema: { type: 'integer' },             description: 'Total number of pages'           }         }       },       404: {         type: 'object',         properties: {           error: { type: 'string' }         }       }     }   }   ``` #### Task 1.3: Implement Service Layer Logic **File**: `src/api/v1/extractedEntity/extractedEntities.service.ts` Add new methods: ```typescript import { downloadBlob } from '../../../helpers/blobStorage.js';   import { Readable } from 'stream';   const EntitiesService = {     // ... existing methods ...     async getImageMetadataForEntity(entityId: string): Promise {       // 1. Fetch entity from database       const entity = await EntitiesModel.getEntityByID(entityId);       if (!entity) {         return null;       }       // 2. Parse extractedEntitiesPayload to get image URLs       let payload: any;       try {         payload = JSON.parse(entity.extractedEntitiesPayload);       } catch (err) {         throw new Error('Invalid JSON in extractedEntitiesPayload');       }       // 3. Extract image URLs (could be blob URLs or URLs with query parameters)       const storedImageUrls = payload.imageUrls || payload.imageBlobUrls || [];       if (storedImageUrls.length === 0) {         throw new Error('No images found for this entity');       }       // 4. Parse each URL to extract blob path and create metadata       const images: ImageMetadata[] = storedImageUrls.map((url: string, index: number) => {         const { container, blobName } = parseBlobUrl(url);         const fileExtension = blobName.split('.').pop()?.toLowerCase() || 'png';         const contentType = fileExtension === 'jpg' || fileExtension === 'jpeg' ? 'image/jpeg' : 'image/png';         return {           pageNumber: index + 1,           blobName,           contentType,           fileName: `page_${index + 1}.${fileExtension}`,           container         };       });       // 5. Update database with clean blob URLs if they had query parameters       const needsUpdate = storedImageUrls.some((url: string) => url.includes('?'));       if (needsUpdate) {         const blobUrls = storedImageUrls.map((url: string) => url.split('?')[0]);         payload.imageBlobUrls = blobUrls;         delete payload.imageUrls;         await EntitiesModel.updateEntityByID(entityId, {           extractedEntitiesPayload: JSON.stringify(payload)         });       }       return {         entityId: parseInt(entityId),         images,         totalImages: images.length       };     },     async streamImageForEntity(       entityId: string,       pageNumber?: number     ): Promise<{ buffer: Buffer; contentType: string; fileName: string; totalPages: number } | null> {       const metadata = await this.getImageMetadataForEntity(entityId);       if (!metadata) {         return null;       }       // If pageNumber specified, validate and get single image       if (pageNumber !== undefined) {         const image = metadata.images.find(img => img.pageNumber === pageNumber);         if (!image) {           throw new Error('Page number exceeds available pages');         }         const buffer = await downloadBlob(image.blobName, image.container, 'docai');         return {           buffer,           contentType: image.contentType,           fileName: image.fileName,           totalPages: metadata.totalImages         };       }       return null; // Multipart streaming handled in controller     }   };   // Helper function to parse blob URL (with or without query parameters)   function parseBlobUrl(url: string): { accountName: string; container: string; blobName: string } {     try {       // Remove query parameters if present       const urlWithoutQuery = url.split('?')[0];       // Parse URL: https://{accountName}.blob.core.windows.net/{container}/{blobName}       const urlObj = new URL(urlWithoutQuery);       const pathParts = urlObj.pathname.split('/').filter(p => p);       const accountName = urlObj.hostname.split('.')[0];       const container = pathParts[0];       const blobName = pathParts.slice(1).join('/');       return { accountName, container, blobName };     } catch (err) {       throw new Error(`Failed to parse blob URL: ${url}`);     }   }   export default EntitiesService;   ``` #### Task 1.4: Implement Controller Layer **File**: `src/api/v1/extractedEntity/extractedEntities.controller.ts` Add new methods: ```typescript import { downloadBlob } from '../../../helpers/blobStorage.js';   const EntitiesController = {     // ... existing methods ...     async streamImagesForEntity(fastify: FastifyInstance, entityId: string, pageNumber?: number) {       try {         if (pageNumber !== undefined) {           // Single image stream           return await EntitiesService.streamImageForEntity(entityId, pageNumber);         }         // Multiple images - return metadata for multipart streaming         return await EntitiesService.getImageMetadataForEntity(entityId);       } catch (err) {         errorHandler(fastify, err, 'ExtractedEntities.streamImagesForEntity');       }     }   };   ``` #### Task 1.5: Create Route Handlers **File**: `src/api/v1/extractedEntity/extractedEntities.route.ts` Add new routes: ```typescript import { downloadBlob } from '../../../helpers/blobStorage.js';   import { pipeline } from 'stream/promises';   // Route 1: Stream single image   fastify.get(`${basePath}/extracted_entities/:entityId/images/:pageNumber`, {     schema: entitySchema.streamSingleImage,     preValidation: [fastify.authenticate, fastify.authorize],     config: {       validRoles: [ROLES.Administrator, ROLES.User],       validScopes: [SCOPES.EXTRACTED_ENTITIES_READ],     },     handler: async (request: FastifyRequest, reply: FastifyReply) => {       try {         const { entityId, pageNumber } = request.params as { entityId: string; pageNumber: string };         const result = await EntitiesController.streamImagesForEntity(           fastify,           entityId,           parseInt(pageNumber)         );         if (!result) {           return reply.status(404).send({ error: 'Entity not found' });         }         reply           .header('Content-Type', result.contentType)           .header('Content-Disposition', `inline; filename=\"${result.fileName}\"`)           .header('X-Page-Number', pageNumber)           .header('X-Total-Pages', result.totalPages.toString())           .send(result.buffer);       } catch (err: any) {         request.log.error(err);         if (err.message?.includes('No images found')) {           return reply.status(404).send({ error: err.message });         }         if (err.message?.includes('Page number exceeds')) {           return reply.status(404).send({ error: err.message });         }         reply.status(500).send({ error: 'Internal Server Error' });       }     }   });   // Route 2: Stream all images as multipart   fastify.get(`${basePath}/extracted_entities/:entityId/images`, {     schema: entitySchema.streamImages,     preValidation: [fastify.authenticate, fastify.authorize],     config: {       validRoles: [ROLES.Administrator, ROLES.User],       validScopes: [SCOPES.EXTRACTED_ENTITIES_READ],     },     handler: async (request: FastifyRequest, reply: FastifyReply) => {       try {         const { entityId } = request.params as { entityId: string };         const metadata = await EntitiesController.streamImagesForEntity(fastify, entityId);         if (!metadata) {           return reply.status(404).send({ error: 'Entity not found' });         }         const boundary = '----ImageBoundary';         reply           .header('Content-Type', `multipart/mixed; boundary=${boundary}`)           .header('X-Total-Pages', metadata.totalImages.toString());         // Stream multipart response         reply.raw.write(`--${boundary}\\r\\n`);         for (const image of metadata.images) {           // Download image from blob storage           const buffer = await downloadBlob(image.blobName, image.container, 'docai');           // Write part headers           reply.raw.write(`Content-Type: ${image.contentType}\\r\\n`);           reply.raw.write(`Content-Disposition: attachment; filename=\"${image.fileName}\"\\r\\n`);           reply.raw.write(`X-Page-Number: ${image.pageNumber}\\r\\n`);           reply.raw.write(`\\r\\n`);           // Write image data           reply.raw.write(buffer);           reply.raw.write(`\\r\\n--${boundary}\\r\\n`);         }         // End multipart response         reply.raw.write(`--${boundary}--\\r\\n`);         reply.raw.end();       } catch (err: any) {         request.log.error(err);         if (err.message?.includes('No images found')) {           return reply.status(404).send({ error: err.message });         }         reply.status(500).send({ error: 'Internal Server Error' });       }     }   });   ``` ### Acceptance Criteria #### Functional Requirements * ✅ Endpoint streams binary image data directly to client * ✅ Single page endpoint returns one image with proper content-type headers * ✅ Multiple pages endpoint returns multipart response with all images * ✅ Returns 404 if entity doesn't exist * ✅ Returns 404 with specific message if no images found for entity * ✅ Returns 404 if page number exceeds available pages * ✅ Database is updated to store clean blob URLs if URLs with query parameters were stored * ✅ Response includes page number and total pages in headers * ✅ Handles URLs with or without query parameters from database gracefully * ✅ Images can be displayed directly in browser (Content-Disposition: inline for single page) #### Non-Functional Requirements * ✅ Endpoint requires authentication (bearer token) * ✅ Endpoint requires appropriate scopes (`EXTRACTED_ENTITIES_READ`) * ✅ Single image response time \< 2 seconds per image * ✅ Multiple images stream progressively (no buffering all images in memory) * ✅ Supports images up to 10MB each * ✅ Proper error logging for troubleshooting * ✅ No sensitive data logged (PIDs, credentials) * ✅ Memory efficient streaming (uses Node.js streams, not full buffering) #### Testing Requirements * ✅ Unit test for URL parsing logic (normalize URLs by removing query parameters) * ✅ Unit test for service layer with mocked blob storage * ✅ Integration test with actual blob storage (using Azurite) * ✅ Test single image endpoint returns correct binary data and headers * ✅ Test multiple images endpoint returns valid multipart response * ✅ Test with entity containing no images (404 response) * ✅ Test with non-existent entity (404 response) * ✅ Test with invalid page number (404 response) * ✅ Test database update when URLs with query parameters are normalized * ✅ Test image content-type detection (PNG vs JPEG) * ✅ Test multipart boundary parsing for clients #### Documentation Requirements * ✅ OpenAPI schema updated with new endpoints * ✅ Generated OpenAPI docs include request/response examples * ✅ Code comments explain streaming and multipart logic * ✅ README updated with endpoint usage examples including UI rendering \--- ## Story 2: GET Bounding Regions JSON ### Story Description **As a** client application performing document analysis **I want to** retrieve the bounding regions JSON file from blob storage for an extracted entity **So that** I can display visual overlays, perform OCR validation, or enhance the user interface with region highlighting ### Technical Details #### Endpoint Specification ``` GET /v1/extracted_entities/:entityId/boundingRegions   ``` #### Request Parameters \| Parameter | Type | Required | Description | \|-----------|------|----------|-------------| \| `entityId` | integer | Yes | The ExtractedEntityId | #### Response Schema (200 OK) ```json {     "entityId": 12345,     "boundingRegions": {       "pages": [         {           "pageNumber": 1,           "regions": [             {               "content": "Invoice Number: INV-001",               "boundingBox": [0.1, 0.2, 0.5, 0.3],               "confidence": 0.98             },             {               "content": "Total Amount: $1,234.56",               "boundingBox": [0.1, 0.8, 0.5, 0.85],               "confidence": 0.99             }           ]         }       ],       "extractedAt": "2026-01-05T08:30:00Z",       "version": "1.0"     },     "blobUrl": "https://storage.blob.core.windows.net/container/doc123_bounding_regions.json"   }   ``` #### Error Responses \| Status Code | Scenario | Response Body | \|-------------|----------|---------------| \| 404 | Entity not found | `{ "error": "Entity not found" }` | \| 404 | Bounding regions not found | `{ "error": "No bounding regions found for this entity" }` | \| 500 | Blob download failed | `{ "error": "Failed to fetch bounding regions from storage" }` | \| 500 | Invalid JSON in blob | `{ "error": "Invalid bounding regions data" }` | \| 500 | Internal error | `{ "error": "Internal Server Error" }` | ### Implementation Tasks #### Task 2.1: Update Type Definitions **File**: `src/api/v1/extractedEntity/extractedEntities.types.ts` ```typescript export interface BoundingRegion {     content: string;     boundingBox: number[]; // [x1, y1, x2, y2]     confidence?: number;   }   export interface BoundingRegionPage {     pageNumber: number;     regions: BoundingRegion[];   }   export interface BoundingRegionsData {     pages: BoundingRegionPage[];     extractedAt?: string;     version?: string;   }   export interface BoundingRegionsResponse {     entityId: number;     boundingRegions: BoundingRegionsData;     blobUrl: string;   }   ``` #### Task 2.2: Create Schema Definition **File**: `src/api/v1/extractedEntity/extractedEntities.schema.ts` Add new schema: ```typescript getBoundingRegions: {     security: [{ bearerAuth: [] }],     params: {       type: 'object',       properties: {         entityId: { type: 'integer', description: 'Extracted Entity ID' }       },       required: ['entityId']     },     tags: ['Extracted Entities'],     response: {       200: {         type: 'object',         properties: {           entityId: { type: 'integer' },           boundingRegions: {             type: 'object',             description: 'Bounding regions data from Azure Document Intelligence',             properties: {               pages: {                 type: 'array',                 items: {                   type: 'object',                   properties: {                     pageNumber: { type: 'integer' },                     regions: {                       type: 'array',                       items: {                         type: 'object',                         properties: {                           content: { type: 'string' },                           boundingBox: {                             type: 'array',                             items: { type: 'number' }                           },                           confidence: { type: 'number' }                         }                       }                     }                   }                 }               },               extractedAt: { type: 'string', format: 'date-time' },               version: { type: 'string' }             }           },           blobUrl: {             type: 'string',             format: 'uri',             description: 'Clean blob URL (without query parameters) where the JSON is stored'           }      }       },       404: {         type: 'object',         properties: {           error: { type: 'string' }         }       }     }   }   ``` #### Task 2.3: Implement Blob Storage Helper **File**: `src/helpers/blobStorage.ts` Add new function: ```typescript /**    * Download blob content as a Buffer    * @param blobName - Name of the blob to download    * @param container - Container name (defaults to DEFAULT_CONTAINER)    * @param endpoint - Storage endpoint: 'fluxx' or 'docai' (defaults to 'docai')    * @returns Buffer containing blob data    */   export async function downloadBlob(     blobName: string,     container: string = DEFAULT_CONTAINER,     endpoint: StorageEndpoint = DEFAULT_ENDPOINT   ): Promise {     try {       const containerClient = await getContainerClient(container, endpoint);       const blobClient = containerClient.getBlobClient(blobName);       const downloadResponse = await blobClient.download();       if (!downloadResponse.readableStreamBody) {         throw new Error('No content in blob');       }       const chunks: Buffer[] = [];       for await (const chunk of downloadResponse.readableStreamBody) {         chunks.push(Buffer.from(chunk));       }       return Buffer.concat(chunks);     } catch (error) {       const errorMessage = error instanceof Error ? error.message : String(error);       throw new Error(`Failed to download blob '${blobName}': ${errorMessage}`);     }   }   ``` #### Task 2.4: Implement Service Layer Logic **File**: `src/api/v1/extractedEntity/extractedEntities.service.ts` Add new method: ```typescript import { downloadBlob } from '../../../helpers/blobStorage.js';   const EntitiesService = {     // ... existing methods ...     async getBoundingRegionsForEntity(entityId: string): Promise {       // 1. Fetch entity from database       const entity = await EntitiesModel.getEntityByID(entityId);       if (!entity) {         return null;       }       // 2. Parse extractedEntitiesPayload to get bounding regions URL       let payload: any;       try {         payload = JSON.parse(entity.extractedEntitiesPayload);       } catch (err) {         throw new Error('Invalid JSON in extractedEntitiesPayload');       }       // 3. Get bounding regions URL (could be blob URL or URL with query parameters)       const boundingRegionUrl = payload.boundingRegionUrl ||                                  payload.boundingRegionBlobUrl ||                                  payload.diJsonUrl;       if (!boundingRegionUrl) {         throw new Error('No bounding regions found for this entity');       }       // 4. Parse URL to extract blob path       const { container, blobName } = parseBlobUrl(boundingRegionUrl);       // 5. Download JSON from blob storage       let boundingRegionsData: BoundingRegionsData;       try {         const blobBuffer = await downloadBlob(blobName, container, 'docai');         const jsonString = blobBuffer.toString('utf-8');         boundingRegionsData = JSON.parse(jsonString);       } catch (err) {         throw new Error('Failed to fetch bounding regions from storage');       }       // 6. Create clean blob URL (without query parameters)       const blobUrl = boundingRegionUrl.split('?')[0];       // 7. Update database with clean blob URL if it had query parameters       if (boundingRegionUrl.includes('?')) {         payload.boundingRegionBlobUrl = blobUrl;         delete payload.boundingRegionUrl; // Remove old URL with query params         delete payload.diJsonUrl; // Remove old field name if present              await EntitiesModel.updateEntityByID(entityId, {           extractedEntitiesPayload: JSON.stringify(payload)         });       }       return {         entityId: parseInt(entityId),         boundingRegions: boundingRegionsData,         blobUrl       };     }   };   ``` #### Task 2.5: Implement Controller Layer **File**: `src/api/v1/extractedEntity/extractedEntities.controller.ts` Add new method: ```typescript const EntitiesController = {     // ... existing methods ...     async getBoundingRegionsForEntity(fastify: FastifyInstance, entityId: string) {       try {         return await EntitiesService.getBoundingRegionsForEntity(entityId);       } catch (err) {         errorHandler(fastify, err, 'ExtractedEntities.getBoundingRegionsForEntity');       }     }   };   ``` #### Task 2.6: Create Route Handler **File**: `src/api/v1/extractedEntity/extractedEntities.route.ts` Add new route: ```typescript fastify.get(`${basePath}/extracted_entities/:entityId/boundingRegions`, {     schema: entitySchema.getBoundingRegions,     preValidation: [fastify.authenticate, fastify.authorize],     config: {       validRoles: [ROLES.Administrator, ROLES.User],       validScopes: [SCOPES.EXTRACTED_ENTITIES_READ],     },     handler: async (request: FastifyRequest, reply: FastifyReply) => {       try {         const { entityId } = request.params as { entityId: string };         const response = await EntitiesController.getBoundingRegionsForEntity(           fastify,           entityId         );         if (!response) {           return reply.status(404).send({ error: 'Entity not found' });         }         reply.status(200).send(response);       } catch (err: any) {         request.log.error(err);         if (err.message?.includes('No bounding regions found')) {           return reply.status(404).send({ error: err.message });         }         if (err.message?.includes('Failed to fetch bounding regions')) {           return reply.status(500).send({ error: err.message });         }         reply.status(500).send({ error: 'Internal Server Error' });       }     }   });   ``` ### Acceptance Criteria #### Functional Requirements * ✅ Endpoint downloads and returns bounding regions JSON from blob storage * ✅ Returns 404 if entity doesn't exist * ✅ Returns 404 with specific message if no bounding regions URL found * ✅ Returns 500 if blob download fails or JSON is invalid * ✅ Database is updated to store clean blob URL if URL with query parameters was stored * ✅ Response includes the blob URL for reference * ✅ Handles URLs with or without query parameters from database gracefully * ✅ Properly parses and validates JSON structure before returning #### Non-Functional Requirements * ✅ Endpoint requires authentication (bearer token) * ✅ Endpoint requires appropriate scopes (`EXTRACTED_ENTITIES_READ`) * ✅ Response time \< 5 seconds for JSON files up to 5MB * ✅ Proper error logging for troubleshooting * ✅ No sensitive data logged (PIDs, credentials) * ✅ Handles large JSON files efficiently (streaming if > 10MB) #### Testing Requirements * ✅ Unit test for blob download helper function * ✅ Unit test for URL parsing logic (normalize URLs by removing query parameters) * ✅ Unit test for service layer with mocked blob storage * ✅ Integration test with actual blob storage (using Azurite) * ✅ Test with entity containing no bounding regions URL (404 response) * ✅ Test with non-existent entity (404 response) * ✅ Test with invalid JSON in blob (500 response) * ✅ Test with non-existent blob (500 response) * ✅ Test database update when URL with query parameters is normalized #### Documentation Requirements * ✅ OpenAPI schema updated with new endpoint * ✅ Generated OpenAPI docs include request/response examples * ✅ Code comments explain blob download and parsing logic * ✅ README updated with endpoint usage examples \--- ## Story 3: Generate Updated OpenAPI Documentation ### Story Description **As a** API consumer or developer **I want** the OpenAPI documentation to reflect the two new endpoints **So that** I can understand the API contract and generate client SDKs ### Implementation Tasks #### Task 3.1: Verify Schema Completeness **File**: `src/api/v1/extractedEntity/extractedEntities.schema.ts` Ensure both new schemas are properly defined: * `getImageUrls` schema * `getBoundingRegions` schema #### Task 3.2: Generate OpenAPI Documentation **Command**: Run the documentation generation script ```bash npm run generate-docs   # or   node scripts/generate-docs-server.ts   ``` #### Task 3.3: Validate Generated Documentation **File**: `docs/openapi.yaml` and `docs/openapi.json` Verify the following are present: 1. Both endpoints appear under the "Extracted Entities" tag 2. All request parameters are documented 3. All response schemas are complete 4. Error responses (404, 500) are documented 5. Security requirements are specified #### Task 3.4: Test OpenAPI Specification **Tools**: Use Swagger Editor or Postman to import and validate the OpenAPI spec * Import `docs/openapi.yaml` into Swagger Editor * Verify no schema validation errors * Check that example values are reasonable * Test that all references are resolved correctly ### Acceptance Criteria #### Documentation Requirements * ✅ OpenAPI 3.0 specification is valid (no errors in Swagger Editor) * ✅ Both new endpoints appear in documentation * ✅ All parameters have descriptions * ✅ All response schemas are complete and accurate * ✅ Security schemes are properly defined * ✅ Examples are provided for request/response bodies * ✅ Tags properly organize endpoints * ✅ Generated documentation matches implementation #### Testing Requirements * ✅ OpenAPI spec validates against OpenAPI 3.0 schema * ✅ Documentation can be imported into Postman without errors * ✅ All endpoint paths are unique and correct * ✅ Response codes match actual implementation \--- ## Cross-Story Considerations ### Security * Both endpoints require authentication via bearer token * Both endpoints require `EXTRACTED_ENTITIES_READ` scope * Images streamed directly through authenticated API (no token exposure to client) * No credentials or sensitive data logged * Blob URLs without query parameters are stored in database (non-expiring) ### Performance * Image streaming parallelized for multipart responses * Blob download uses efficient buffering for binary data * Database updates are conditional (only when URL normalization needed) * Response times: \- Single Image: \< 2 seconds per image \- Multiple Images (multipart): \< 5 seconds for 10 images \- Bounding Regions: \< 5 seconds for 5MB JSON ### Error Handling * Consistent error response format across endpoints * Specific error messages for different failure scenarios * Proper HTTP status codes (404, 500) * Error logging without exposing sensitive details * Graceful degradation when blob storage is unavailable ### Data Migration If existing entities have URLs with query parameters in the database: * First API call will normalize URLs to persistent blob URLs * Database update is transactional * Original functionality preserved (streaming works regardless of stored URL format) * No downtime required for migration ### Testing Strategy 1. **Unit Tests**: Service layer logic with mocked dependencies 2. **Integration Tests**: Full request/response cycle with Azurite 3. **E2E Tests**: Real scenarios with database + blob storage 4. **Load Tests**: Performance validation with concurrent requests \--- ## Implementation Order ### Phase 1: Foundation (Story 1 - Stream Images) 1. Update type definitions 2. Create schema definitions (two endpoints) 3. Implement service layer 4. Implement controller layer 5. Create route handlers 6. Write unit tests 7. Write integration tests ### Phase 2: Bounding Regions (Story 2) 1. Update type definitions 2. Create schema definition 3. Implement blob download helper 4. Implement service layer 5. Implement controller layer 6. Create route handler 7. Write unit tests 8. Write integration tests ### Phase 3: Documentation (Story 3) 1. Verify schema completeness 2. Generate OpenAPI docs 3. Validate documentation 4. Update README with examples \--- ## Testing Plan ### Unit Tests **Location**: `src/api/v1/extractedEntity/tests/` #### Test Files to Create: 1. `extractedEntities.imageStreaming.test.ts` 2. `extractedEntities.boundingRegions.test.ts` 3. `blobStorage.download.test.ts` (in `src/helpers/tests/`) #### Test Cases for Image Streaming: * ✅ Parse blob URL correctly * ✅ Parse URLs with query parameters correctly (extract blob path) * ✅ Stream single image with correct headers * ✅ Stream multiple images as multipart response * ✅ Handle entity not found (404) * ✅ Handle no images found (404) * ✅ Handle invalid page number (404) * ✅ Update database when URLs with query parameters detected * ✅ Don't update database when clean blob URLs already stored * ✅ Handle blob storage errors gracefully * ✅ Detect correct content-type (PNG vs JPEG) #### Test Cases for Bounding Regions: * ✅ Download and parse valid JSON from blob * ✅ Handle entity not found (404) * ✅ Handle no bounding regions URL (404) * ✅ Handle blob download failure (500) * ✅ Handle invalid JSON in blob (500) * ✅ Update database when URL with query parameters detected * ✅ Don't update database when clean blob URL already stored * ✅ Handle large JSON files (streaming) ### Integration Tests **Location**: `src/api/v1/extractedEntity/tests/` #### Test Scenarios: 1. **Happy Path - Single Image Stream**: \- Create test entity with blob URLs \- Call GET /extracted\_entities/:id/images/1 \- Verify binary image data returned \- Verify headers (Content-Type, X-Page-Number, X-Total-Pages) 2. **Happy Path - Multiple Images Stream**: \- Create test entity with multiple blob URLs \- Call GET /extracted\_entities/:id/images \- Verify multipart response structure \- Verify all images received \- Parse multipart boundaries correctly 3. **Happy Path - Bounding Regions**: \- Upload test JSON to Azurite \- Create test entity with blob URL \- Call GET /extracted\_entities/:id/boundingRegions \- Verify JSON returned 3. **Migration Path - URL Normalization**: \- Create entity with URLs containing query parameters \- Call image streaming endpoint \- Verify database updated with clean blob URLs 4. **Error Scenarios**: \- Non-existent entity \- Missing images/bounding regions \- Blob storage unavailable \- Invalid JSON in blob ### Load Testing **Tool**: Artillery or K6 #### Scenarios: * 100 concurrent requests to /images/:pageNumber endpoint * 50 concurrent requests to /images endpoint (multipart) * 100 concurrent requests to /boundingRegions endpoint * Mixed workload (image streaming + bounding regions) * Response time \< 2s for p95 (single images) * Response time \< 5s for p95 (multipart/multiple images) \--- ## Rollout Plan ### Pre-Deployment Checklist * ✅ All unit tests passing * ✅ All integration tests passing * ✅ Code review completed * ✅ OpenAPI documentation generated and validated * ✅ README updated with new endpoint examples * ✅ Security review completed (no PII logged) * ✅ Performance testing completed ### Deployment Steps 1. Deploy to development environment 2. Run smoke tests 3. Deploy to QA environment 4. Run full test suite 5. Deploy to production with feature flag (if available) 6. Monitor error rates and response times 7. Enable for all users ### Monitoring & Alerts * Track response times for all image streaming endpoints * Monitor error rates (404, 500) * Track database update frequency (URL normalization) * Alert on blob storage connection failures * Alert on response times > 5s * Monitor streaming throughput and memory usage ### Rollback Plan If issues arise: 1. Disable routes via feature flag 2. Revert deployment if necessary 3. Investigate errors using logs 4. Fix issues and redeploy \--- ## Dependencies & Prerequisites ### Technical Dependencies * ✅ Azure Blob Storage configured (`AZURE_STORAGE_CONNECTION_STRING_DOCAI`) * ✅ `@azure/storage-blob` package installed * ✅ Existing `blobStorage.ts` helper functions * ✅ Database access to `ExtractedEntities` table * ✅ Authentication middleware (`fastify.authenticate`) * ✅ Authorization middleware (`fastify.authorize`) ### Data Prerequisites * ExtractedEntities records must have image URLs or bounding region URLs in `extractedEntitiesPayload` * Blobs must exist in Azure Storage * JSON files must be valid and follow expected schema ### Team Dependencies * Backend developer for implementation * QA engineer for testing * DevOps for deployment * Technical writer for documentation review \--- ## Success Metrics ### Functional Metrics * ✅ 100% of entities with images can stream binary image data * ✅ 100% of entities with bounding regions can retrieve JSON data * ✅ 100% of URLs in database normalized to persistent blob URLs after first access ### Performance Metrics * ✅ p95 response time \< 2s for single image streaming endpoint * ✅ p95 response time \< 5s for multiple images (multipart) endpoint * ✅ p95 response time \< 5s for bounding regions endpoint * ✅ 99.9% uptime for all endpoints ### Quality Metrics * ✅ 85%+ code coverage * ✅ 0 critical security vulnerabilities * ✅ 0 PII logged in production logs \--- ## Future Enhancements ### Potential Improvements (Out of Scope for MVP) 1. **Caching Layer**: Cache image data in Redis for frequently accessed images to reduce blob storage calls 2. **Batch Endpoint**: `POST /extracted_entities/batch/images` to get images for multiple entities 3. **Lazy Loading**: Stream images one at a time with pagination for entities with many images 4. **CDN Integration**: Serve images through Azure CDN for faster delivery 5. **Image Metadata**: Return image dimensions, file size, format in additional headers 6. **Bounding Regions Filtering**: Query parameter to filter regions by confidence score or content 7. **Webhooks**: Notify clients when new images or bounding regions are available 8. **Image Transformations**: Resize, crop, or format conversion on-the-fly 9. **Progressive JPEG**: Stream images in progressive format for faster rendering 10. **Thumbnail Generation**: Generate and cache thumbnail versions for preview \--- ## Appendix ### Example Request/Response #### Example 1: Stream Single Image ```bash GET /v1/extracted_entities/12345/images/1   Authorization: Bearer   Response (200 OK):   Content-Type: image/png   Content-Disposition: inline; filename="page_1.png"   X-Page-Number: 1   X-Total-Pages: 5   [Binary PNG image data]   ``` #### Example 1b: Stream All Images (Multipart) ```bash GET /v1/extracted_entities/12345/images   Authorization: Bearer   Response (200 OK):   Content-Type: multipart/mixed; boundary=----ImageBoundary   X-Total-Pages: 2   ------ImageBoundary   Content-Type: image/png   Content-Disposition: attachment; filename="page_1.png"   X-Page-Number: 1   [Binary PNG data for page 1]   ------ImageBoundary   Content-Type: image/png   Content-Disposition: attachment; filename="page_2.png"   X-Page-Number: 2   [Binary PNG data for page 2]   ------ImageBoundary--   ``` #### Example 2: Get Bounding Regions\`\`\`bash GET /v1/extracted\_entities/12345/boundingRegions ``` Authorization: Bearer   Response (200 OK):   {     "entityId": 12345,     "boundingRegions": {       "pages": [         {           "pageNumber": 1,           "regions": [             {               "content": "Invoice Number: INV-001",               "boundingBox": [0.1, 0.2, 0.5, 0.3],               "confidence": 0.98             }           ]         }       ],       "extractedAt": "2026-01-05T08:30:00Z",       "version": "1.0"     },     "blobUrl": "https://docaistorage.blob.core.windows.net/invoicepdf/doc123_bounding_regions.json"   }   ``` #### Example 3: Entity Not Found ```bash GET /v1/extracted_entities/99999/images   Authorization: Bearer   Response (404 Not Found):   {     "error": "Entity not found"   }   ``` #### Example 4: No Images Found ```bash GET /v1/extracted_entities/12345/images   Authorization: Bearer   Response (404 Not Found):   {     "error": "No images found for this entity"   }   ``` #### Example 5: Page Number Out of Range ```bash GET /v1/extracted_entities/12345/images/99   Authorization: Bearer   Response (404 Not Found):   {     "error": "Page number exceeds available pages"   }   ``` #### Example 6: UI Rendering Implementation ```typescript  // React component to render streamed images   async function displayInvoiceImages(entityId: number) {     try {       // Fetch single image for preview       const response = await fetch(         `/v1/extracted_entities/${entityId}/images/1`,         {           headers: {             'Authorization': `Bearer ${token}`           }         }       );       if (response.ok) {         const blob = await response.blob();         const imageUrl = URL.createObjectURL(blob);         const totalPages = response.headers.get('X-Total-Pages');         // Display image         document.getElementById('preview-image').src = imageUrl;         console.log(`Showing page 1 of ${totalPages}`);       }     } catch (error) {       console.error('Failed to load image:', error);     }   }   // Fetch all images as multipart (for download/processing)   async function downloadAllImages(entityId: number) {     const response = await fetch(       `/v1/extracted_entities/${entityId}/images`,       {         headers: {           'Authorization': `Bearer ${token}`         }       }     );     // Parse multipart response     const contentType = response.headers.get('Content-Type');     const boundary = contentType.split('boundary=')[1];     const text = await response.text();     // Split by boundary and extract images     const parts = text.split(`--${boundary}`);     const images = [];     for (const part of parts) {       if (part.includes('Content-Type: image')) {         // Extract image data and headers         const [headers, ...dataLines] = part.split('\\r\\n\\r\\n');         const pageMatch = headers.match(/X-Page-Number: (\\d+)/);         const pageNumber = pageMatch ? parseInt(pageMatch[1]) : 0;         images.push({           pageNumber,           data: dataLines.join('\\r\\n\\r\\n')         });       }     }     return images;   }   ``` \--- ## Glossary \| Term | Definition | \|------|------------| \| **Blob URL** | Persistent URL to a blob without query parameters or tokens (requires separate authentication) | \| **Image Streaming** | Server-side process of downloading images from blob storage and returning binary data directly to client | \| **Multipart Response** | HTTP response format that contains multiple parts (images) separated by boundaries | \| **Entity** | Record in ExtractedEntities table representing processed invoice data | \| **Bounding Region** | Rectangle coordinates identifying text location in a document image | \| **Document Intelligence** | Azure service for OCR and document analysis (formerly Form Recognizer) | \| **Azurite** | Local Azure Storage emulator for development and testing | \--- ## References * [Azure Blob Storage Documentation](https://learn.microsoft.com/en-us/azure/storage/blobs/) * [Node.js Streams](https://nodejs.org/api/stream.html) * [Fastify Routing](https://fastify.dev/docs/latest/Reference/Routes/) * [OpenAPI 3.0 Specification](https://swagger.io/specification/) * [Multipart HTTP Responses](https://www.w3.org/Protocols/rfc1341/7_2_Multipart.html) * [TypeScript Best Practices](https://www.typescriptlang.org/docs/handbook/declaration-files/do-s-and-don-ts.html) \--- ## Change Log \| Date | Version | Changes | Author | \|------|---------|---------|--------| \| 2026-01-05 | 1.0 | Initial story design created | GitHub Copilot | \--- **Document Status**: ✅ Ready for Review **Next Step**: Team review and story estimation **Estimated Total Story Points**: 13 points (Story 1: 5pts, Story 2: 6pts, Story 3: 2pts)