Extracted Entities---Streaming Invoice Images and Bounding Regions Endpoints

Overview

This document outlines the implementation stories for two new endpoints that will stream invoice images directly and provide access to bounding region data associated with extracted entities. These endpoints will improve security by eliminating the need for client-side token management and optimize data storage by storing persistent blob URLs instead of time-limited URLs in the database.

---

Background & Context

Current State

The ExtractedEntities table currently stores image URLs and bounding region URLs
These URLs may be stored with query parameters (expiring tokens) or as persistent blob URLs
There’s no dedicated endpoint to stream images directly to the client
There’s no dedicated endpoint to fetch bounding region JSON data from blob storage

Business Value

Security: Stream images directly from API without exposing storage credentials or tokens
Data Integrity: Store persistent blob URLs instead of expiring URLs in the database
Flexibility: Separate concerns between data storage and data access
Client Experience: Provide clients with ready-to-use binary image data without manual token or URL management

Story 1: Stream Invoice Images

Story Description

As a client application consuming invoice extraction data

I want to stream invoice images directly from the API for an extracted entity

So that I can display images in the UI without managing blob storage authentication or token expiration

Technical Details

Endpoint Specification

GET /v1/extracted_entities/:entityId/images  
GET /v1/extracted_entities/:entityId/images/:pageNumber

Request Parameters

|-----------|------|----------|-------------|

Response Schema (200 OK)

Single Page Request (GET /v1/extracted_entities/:entityId/images/:pageNumber):

Content-Type: image/png (or image/jpeg)  
Content-Disposition: inline; filename="page_1.png"  
X-Page-Number: 1  
X-Total-Pages: 5  
[Binary image data]

Multiple Pages Request (GET /v1/extracted_entities/:entityId/images):

Content-Type: multipart/mixed; boundary=----ImageBoundary  
  
  
------ImageBoundary  
  
Content-Type: image/png  
  
Content-Disposition: attachment; filename="page_1.png"  
  
X-Page-Number: 1  
  
  
[Binary image data for page 1]  
  
------ImageBoundary  
  
Content-Type: image/png  
  
Content-Disposition: attachment; filename="page_2.png"  
  
X-Page-Number: 2  
  
  
[Binary image data for page 2]  
  
------ImageBoundary--

Error Responses

| Status Code | Scenario | Response Body |

|-------------|----------|---------------|

| 404 | Entity not found | { "error": "Entity not found" } |

| 404 | No images found | { "error": "No images found for this entity" } |

| 404 | Page number out of range | { "error": "Page number exceeds available pages" } |

| 500 | Blob download error | { "error": "Failed to retrieve images from storage" } |

| 500 | Internal error | { "error": "Internal Server Error" } |

Implementation Tasks

Task 1.1: Update Type Definitions

File: src/api/v1/extractedEntity/extractedEntities.types.ts

  
export interface ImageMetadata {  
  
  pageNumber: number;  
  
  blobName: string;  
  
  container: string;  
  
  contentType: string;  
  
  fileName: string;  
  
}  
  
  
export interface ImageStreamInfo {  
  
  entityId: number;  
  
  images: ImageMetadata[];  
  
  totalImages: number;  
  
}

Task 1.2: Create Schema Definition

File: src/api/v1/extractedEntity/extractedEntities.schema.ts

Add new schemas:

  
streamImages: {  
  
  security: [{ bearerAuth: [] }],  
  
  params: {  
  
    type: 'object',  
  
    properties: {  
  
      entityId: { type: 'integer', description: 'Extracted Entity ID' }  
  
    },  
  
    required: ['entityId']  
  
  },  
  
  tags: ['Extracted Entities'],  
  
  response: {  
  
    200: {  
  
      description: 'Multipart response with image streams',  
  
      content: {  
  
        'multipart/mixed': {  
  
          schema: {  
  
            type: 'string',  
  
            format: 'binary'  
  
          }  
  
        }  
  
      },  
  
      headers: {  
  
        'X-Total-Pages': {  
  
          schema: { type: 'integer' },  
  
          description: 'Total number of image pages'  
  
        }  
  
      }  
  
    },  
  
    404: {  
  
      type: 'object',  
  
      properties: {  
  
        error: { type: 'string' }  
  
      }  
  
    }  
  
  }  
  
},  
  
streamSingleImage: {  
  
  security: [{ bearerAuth: [] }],  
  
  params: {  
  
    type: 'object',  
  
    properties: {  
  
      entityId: { type: 'integer', description: 'Extracted Entity ID' },  
  
      pageNumber: { type: 'integer', description: 'Page number to retrieve' }  
  
    },  
  
    required: ['entityId', 'pageNumber']  
  
  },  
  
  tags: ['Extracted Entities'],  
  
  response: {  
  
    200: {  
  
      description: 'Binary image data',  
  
      content: {  
  
        'image/png': {  
  
          schema: {  
  
            type: 'string',  
  
            format: 'binary'  
  
          }  
  
        },  
  
        'image/jpeg': {  
  
          schema: {  
  
            type: 'string',  
  
            format: 'binary'  
  
          }  
  
        }  
  
      },  
  
      headers: {  
  
        'X-Page-Number': {  
  
          schema: { type: 'integer' },  
  
          description: 'Current page number'  
  
        },  
  
        'X-Total-Pages': {  
  
          schema: { type: 'integer' },  
  
          description: 'Total number of pages'  
  
        }  
  
      }  
  
    },  
  
    404: {  
  
      type: 'object',  
  
      properties: {  
  
        error: { type: 'string' }  
  
      }  
  
    }  
  
  }  
  
}

Task 1.3: Implement Service Layer Logic

File: src/api/v1/extractedEntity/extractedEntities.service.ts

Add new methods:

  
import { downloadBlob } from '../../../helpers/blobStorage.js';  
  
import { Readable } from 'stream';  
  
  
const EntitiesService = {  
  
  // ... existing methods ...  
  
  
  async getImageMetadataForEntity(entityId: string): Promise<ImageStreamInfo | null> {  
  
    // 1. Fetch entity from database  
  
    const entity = await EntitiesModel.getEntityByID(entityId);  
  
    if (!entity) {  
  
      return null;  
  
    }  
  
  
    // 2. Parse extractedEntitiesPayload to get image URLs  
  
    let payload: any;  
  
    try {  
  
      payload = JSON.parse(entity.extractedEntitiesPayload);  
  
    } catch (err) {  
  
      throw new Error('Invalid JSON in extractedEntitiesPayload');  
  
    }  
  
  
    // 3. Extract image URLs (could be blob URLs or URLs with query parameters)  
  
    const storedImageUrls = payload.imageUrls || payload.imageBlobUrls || [];  
  
    if (storedImageUrls.length === 0) {  
  
      throw new Error('No images found for this entity');  
  
    }  
  
  
    // 4. Parse each URL to extract blob path and create metadata  
  
    const images: ImageMetadata[] = storedImageUrls.map((url: string, index: number) => {  
  
      const { container, blobName } = parseBlobUrl(url);  
  
      const fileExtension = blobName.split('.').pop()?.toLowerCase() || 'png';  
  
      const contentType = fileExtension === 'jpg' || fileExtension === 'jpeg' ? 'image/jpeg' : 'image/png';  
  
  
      return {  
  
        pageNumber: index + 1,  
  
        blobName,  
  
        contentType,  
  
        fileName: `page_${index + 1}.${fileExtension}`,  
  
        container  
  
      };  
  
    });  
  
  
    // 5. Update database with clean blob URLs if they had query parameters  
  
    const needsUpdate = storedImageUrls.some((url: string) => url.includes('?'));  
  
    if (needsUpdate) {  
  
      const blobUrls = storedImageUrls.map((url: string) => url.split('?')[0]);  
  
      payload.imageBlobUrls = blobUrls;  
  
      delete payload.imageUrls;  
  
      await EntitiesModel.updateEntityByID(entityId, {  
  
        extractedEntitiesPayload: JSON.stringify(payload)  
  
      });  
  
    }  
  
  
    return {  
  
      entityId: parseInt(entityId),  
  
      images,  
  
      totalImages: images.length  
  
    };  
  
  },  
  
  
  async streamImageForEntity(  
  
    entityId: string,  
  
    pageNumber?: number  
  
  ): Promise<{ buffer: Buffer; contentType: string; fileName: string; totalPages: number } | null> {  
  
    const metadata = await this.getImageMetadataForEntity(entityId);  
  
    if (!metadata) {  
  
      return null;  
  
    }  
  
  
    // If pageNumber specified, validate and get single image  
  
    if (pageNumber !== undefined) {  
  
      const image = metadata.images.find(img => img.pageNumber === pageNumber);  
  
      if (!image) {  
  
        throw new Error('Page number exceeds available pages');  
  
      }  
  
  
      const buffer = await downloadBlob(image.blobName, image.container, 'docai');  
  
      return {  
  
        buffer,  
  
        contentType: image.contentType,  
  
        fileName: image.fileName,  
  
        totalPages: metadata.totalImages  
  
      };  
  
    }  
  
  
    return null; // Multipart streaming handled in controller  
  
  }  
  
};  
  
  
// Helper function to parse blob URL (with or without query parameters)  
  
function parseBlobUrl(url: string): { accountName: string; container: string; blobName: string } {  
  
  try {  
  
    // Remove query parameters if present  
  
    const urlWithoutQuery = url.split('?')[0];  
  
  
    // Parse URL: https://{accountName}.blob.core.windows.net/{container}/{blobName}  
  
    const urlObj = new URL(urlWithoutQuery);  
  
    const pathParts = urlObj.pathname.split('/').filter(p => p);  
  
  
    const accountName = urlObj.hostname.split('.')[0];  
  
    const container = pathParts[0];  
  
    const blobName = pathParts.slice(1).join('/');  
  
  
    return { accountName, container, blobName };  
  
  } catch (err) {  
  
    throw new Error(`Failed to parse blob URL: ${url}`);  
  
  }  
  
}  
  
  
export default EntitiesService;

Task 1.4: Implement Controller Layer

File: src/api/v1/extractedEntity/extractedEntities.controller.ts

Add new methods:

  
import { downloadBlob } from '../../../helpers/blobStorage.js';  
  
  
const EntitiesController = {  
  
  // ... existing methods ...  
  
  
  async streamImagesForEntity(fastify: FastifyInstance, entityId: string, pageNumber?: number) {  
  
    try {  
  
      if (pageNumber !== undefined) {  
  
        // Single image stream  
  
        return await EntitiesService.streamImageForEntity(entityId, pageNumber);  
  
      }  
  
  
      // Multiple images - return metadata for multipart streaming  
  
      return await EntitiesService.getImageMetadataForEntity(entityId);  
  
    } catch (err) {  
  
      errorHandler(fastify, err, 'ExtractedEntities.streamImagesForEntity');  
  
    }  
  
  }  
  
};

Task 1.5: Create Route Handlers

File: src/api/v1/extractedEntity/extractedEntities.route.ts

Add new routes:

  
import { downloadBlob } from '../../../helpers/blobStorage.js';  
  
import { pipeline } from 'stream/promises';  
  
  
// Route 1: Stream single image  
  
fastify.get(`${basePath}/extracted_entities/:entityId/images/:pageNumber`, {  
  
  schema: entitySchema.streamSingleImage,  
  
  preValidation: [fastify.authenticate, fastify.authorize],  
  
  config: {  
  
    validRoles: [ROLES.Administrator, ROLES.User],  
  
    validScopes: [SCOPES.EXTRACTED_ENTITIES_READ],  
  
  },  
  
  handler: async (request: FastifyRequest, reply: FastifyReply) => {  
  
    try {  
  
      const { entityId, pageNumber } = request.params as { entityId: string; pageNumber: string };  
  
  
      const result = await EntitiesController.streamImagesForEntity(  
  
        fastify,  
  
        entityId,  
  
        parseInt(pageNumber)  
  
      );  
  
  
      if (!result) {  
  
        return reply.status(404).send({ error: 'Entity not found' });  
  
      }  
  
  
      reply  
  
        .header('Content-Type', result.contentType)  
  
        .header('Content-Disposition', `inline; filename=\"${result.fileName}\"`)  
  
        .header('X-Page-Number', pageNumber)  
  
        .header('X-Total-Pages', result.totalPages.toString())  
  
        .send(result.buffer);  
  
    } catch (err: any) {  
  
      request.log.error(err);  
  
  
      if (err.message?.includes('No images found')) {  
  
        return reply.status(404).send({ error: err.message });  
  
      }  
  
  
      if (err.message?.includes('Page number exceeds')) {  
  
        return reply.status(404).send({ error: err.message });  
  
      }  
  
  
      reply.status(500).send({ error: 'Internal Server Error' });  
  
    }  
  
  }  
  
});  
  
  
// Route 2: Stream all images as multipart  
  
fastify.get(`${basePath}/extracted_entities/:entityId/images`, {  
  
  schema: entitySchema.streamImages,  
  
  preValidation: [fastify.authenticate, fastify.authorize],  
  
  config: {  
  
    validRoles: [ROLES.Administrator, ROLES.User],  
  
    validScopes: [SCOPES.EXTRACTED_ENTITIES_READ],  
  
  },  
  
  handler: async (request: FastifyRequest, reply: FastifyReply) => {  
  
    try {  
  
      const { entityId } = request.params as { entityId: string };  
  
  
      const metadata = await EntitiesController.streamImagesForEntity(fastify, entityId);  
  
  
      if (!metadata) {  
  
        return reply.status(404).send({ error: 'Entity not found' });  
  
      }  
  
  
      const boundary = '----ImageBoundary';  
  
      reply  
  
        .header('Content-Type', `multipart/mixed; boundary=${boundary}`)  
  
        .header('X-Total-Pages', metadata.totalImages.toString());  
  
  
      // Stream multipart response  
  
      reply.raw.write(`--${boundary}\\r\\n`);  
  
  
      for (const image of metadata.images) {  
  
        // Download image from blob storage  
  
        const buffer = await downloadBlob(image.blobName, image.container, 'docai');  
  
  
        // Write part headers  
  
        reply.raw.write(`Content-Type: ${image.contentType}\\r\\n`);  
  
        reply.raw.write(`Content-Disposition: attachment; filename=\"${image.fileName}\"\\r\\n`);  
  
        reply.raw.write(`X-Page-Number: ${image.pageNumber}\\r\\n`);  
  
        reply.raw.write(`\\r\\n`);  
  
  
        // Write image data  
  
        reply.raw.write(buffer);  
  
        reply.raw.write(`\\r\\n--${boundary}\\r\\n`);  
  
      }  
  
  
      // End multipart response  
  
      reply.raw.write(`--${boundary}--\\r\\n`);  
  
      reply.raw.end();  
  
    } catch (err: any) {  
  
      request.log.error(err);  
  
  
      if (err.message?.includes('No images found')) {  
  
        return reply.status(404).send({ error: err.message });  
  
      }  
  
  
      reply.status(500).send({ error: 'Internal Server Error' });  
  
    }  
  
  }  
  
});

Acceptance Criteria

Functional Requirements

✅ Endpoint streams binary image data directly to client
✅ Single page endpoint returns one image with proper content-type headers
✅ Multiple pages endpoint returns multipart response with all images
✅ Returns 404 if entity doesn’t exist
✅ Returns 404 with specific message if no images found for entity
✅ Returns 404 if page number exceeds available pages
✅ Database is updated to store clean blob URLs if URLs with query parameters were stored
✅ Response includes page number and total pages in headers
✅ Handles URLs with or without query parameters from database gracefully
✅ Images can be displayed directly in browser (Content-Disposition: inline for single page)

Non-Functional Requirements

✅ Endpoint requires authentication (bearer token)
✅ Endpoint requires appropriate scopes (EXTRACTED_ENTITIES_READ)
✅ Single image response time < 2 seconds per image
✅ Multiple images stream progressively (no buffering all images in memory)
✅ Supports images up to 10MB each
✅ Proper error logging for troubleshooting
✅ No sensitive data logged (PIDs, credentials)
✅ Memory efficient streaming (uses Node.js streams, not full buffering)

Testing Requirements

✅ Unit test for URL parsing logic (normalize URLs by removing query parameters)
✅ Unit test for service layer with mocked blob storage
✅ Integration test with actual blob storage (using Azurite)
✅ Test single image endpoint returns correct binary data and headers
✅ Test multiple images endpoint returns valid multipart response
✅ Test with entity containing no images (404 response)
✅ Test with non-existent entity (404 response)
✅ Test with invalid page number (404 response)
✅ Test database update when URLs with query parameters are normalized
✅ Test image content-type detection (PNG vs JPEG)
✅ Test multipart boundary parsing for clients

Documentation Requirements

✅ OpenAPI schema updated with new endpoints
✅ Generated OpenAPI docs include request/response examples
✅ Code comments explain streaming and multipart logic
✅ README updated with endpoint usage examples including UI rendering

---

Story 2: GET Bounding Regions JSON

Story Description

As a client application performing document analysis

I want to retrieve the bounding regions JSON file from blob storage for an extracted entity

So that I can display visual overlays, perform OCR validation, or enhance the user interface with region highlighting

Technical Details

Endpoint Specification

  
GET /v1/extracted_entities/:entityId/boundingRegions

Request Parameters

|-----------|------|----------|-------------|

Response Schema (200 OK)

  
{  
  
  "entityId": 12345,  
  
  "boundingRegions": {  
  
    "pages": [  
  
      {  
  
        "pageNumber": 1,  
  
        "regions": [  
  
          {  
  
            "content": "Invoice Number: INV-001",  
  
            "boundingBox": [0.1, 0.2, 0.5, 0.3],  
  
            "confidence": 0.98  
  
          },  
  
          {  
  
            "content": "Total Amount: $1,234.56",  
  
            "boundingBox": [0.1, 0.8, 0.5, 0.85],  
  
            "confidence": 0.99  
  
          }  
  
        ]  
  
      }  
  
    ],  
  
    "extractedAt": "2026-01-05T08:30:00Z",  
  
    "version": "1.0"  
  
  },  
  
  "blobUrl": "https://storage.blob.core.windows.net/container/doc123_bounding_regions.json"  
  
}

Error Responses

| Status Code | Scenario | Response Body |

|-------------|----------|---------------|

| 404 | Entity not found | { "error": "Entity not found" } |

| 404 | Bounding regions not found | { "error": "No bounding regions found for this entity" } |

| 500 | Blob download failed | { "error": "Failed to fetch bounding regions from storage" } |

| 500 | Invalid JSON in blob | { "error": "Invalid bounding regions data" } |

| 500 | Internal error | { "error": "Internal Server Error" } |

Implementation Tasks

Task 2.1: Update Type Definitions

File: src/api/v1/extractedEntity/extractedEntities.types.ts

  
export interface BoundingRegion {  
  
  content: string;  
  
  boundingBox: number[]; // [x1, y1, x2, y2]  
  
  confidence?: number;  
  
}  
  
  
export interface BoundingRegionPage {  
  
  pageNumber: number;  
  
  regions: BoundingRegion[];  
  
}  
  
  
export interface BoundingRegionsData {  
  
  pages: BoundingRegionPage[];  
  
  extractedAt?: string;  
  
  version?: string;  
  
}  
  
  
export interface BoundingRegionsResponse {  
  
  entityId: number;  
  
  boundingRegions: BoundingRegionsData;  
  
  blobUrl: string;  
  
}

Task 2.2: Create Schema Definition

File: src/api/v1/extractedEntity/extractedEntities.schema.ts

Add new schema:

  
getBoundingRegions: {  
  
  security: [{ bearerAuth: [] }],  
  
  params: {  
  
    type: 'object',  
  
    properties: {  
  
      entityId: { type: 'integer', description: 'Extracted Entity ID' }  
  
    },  
  
    required: ['entityId']  
  
  },  
  
  tags: ['Extracted Entities'],  
  
  response: {  
  
    200: {  
  
      type: 'object',  
  
      properties: {  
  
        entityId: { type: 'integer' },  
  
        boundingRegions: {  
  
          type: 'object',  
  
          description: 'Bounding regions data from Azure Document Intelligence',  
  
          properties: {  
  
            pages: {  
  
              type: 'array',  
  
              items: {  
  
                type: 'object',  
  
                properties: {  
  
                  pageNumber: { type: 'integer' },  
  
                  regions: {  
  
                    type: 'array',  
  
                    items: {  
  
                      type: 'object',  
  
                      properties: {  
  
                        content: { type: 'string' },  
  
                        boundingBox: {  
  
                          type: 'array',  
  
                          items: { type: 'number' }  
  
                        },  
  
                        confidence: { type: 'number' }  
  
                      }  
  
                    }  
  
                  }  
  
                }  
  
              }  
  
            },  
  
            extractedAt: { type: 'string', format: 'date-time' },  
  
            version: { type: 'string' }  
  
          }  
  
        },  
  
        blobUrl: {  
  
          type: 'string',  
  
          format: 'uri',  
  
          description: 'Clean blob URL (without query parameters) where the JSON is stored'  
  
        }      }  
  
    },  
  
    404: {  
  
      type: 'object',  
  
      properties: {  
  
        error: { type: 'string' }  
  
      }  
  
    }  
  
  }  
  
}

Task 2.3: Implement Blob Storage Helper

File: src/helpers/blobStorage.ts

Add new function:

  
/**  
  
 * Download blob content as a Buffer  
  
 * @param blobName - Name of the blob to download  
  
 * @param container - Container name (defaults to DEFAULT_CONTAINER)  
  
 * @param endpoint - Storage endpoint: 'fluxx' or 'docai' (defaults to 'docai')  
  
 * @returns Buffer containing blob data  
  
 */  
  
export async function downloadBlob(  
  
  blobName: string,  
  
  container: string = DEFAULT_CONTAINER,  
  
  endpoint: StorageEndpoint = DEFAULT_ENDPOINT  
  
): Promise<Buffer> {  
  
  try {  
  
    const containerClient = await getContainerClient(container, endpoint);  
  
    const blobClient = containerClient.getBlobClient(blobName);  
  
  
    const downloadResponse = await blobClient.download();  
  
  
    if (!downloadResponse.readableStreamBody) {  
  
      throw new Error('No content in blob');  
  
    }  
  
  
    const chunks: Buffer[] = [];  
  
    for await (const chunk of downloadResponse.readableStreamBody) {  
  
      chunks.push(Buffer.from(chunk));  
  
    }  
  
  
    return Buffer.concat(chunks);  
  
  } catch (error) {  
  
    const errorMessage = error instanceof Error ? error.message : String(error);  
  
    throw new Error(`Failed to download blob '${blobName}': ${errorMessage}`);  
  
  }  
  
}

Task 2.4: Implement Service Layer Logic

File: src/api/v1/extractedEntity/extractedEntities.service.ts

Add new method:

  
import { downloadBlob } from '../../../helpers/blobStorage.js';  
  
  
const EntitiesService = {  
  
  // ... existing methods ...  
  
  
  async getBoundingRegionsForEntity(entityId: string): Promise<BoundingRegionsResponse | null> {  
  
    // 1. Fetch entity from database  
  
    const entity = await EntitiesModel.getEntityByID(entityId);  
  
    if (!entity) {  
  
      return null;  
  
    }  
  
  
    // 2. Parse extractedEntitiesPayload to get bounding regions URL  
  
    let payload: any;  
  
    try {  
  
      payload = JSON.parse(entity.extractedEntitiesPayload);  
  
    } catch (err) {  
  
      throw new Error('Invalid JSON in extractedEntitiesPayload');  
  
    }  
  
  
    // 3. Get bounding regions URL (could be blob URL or URL with query parameters)  
  
    const boundingRegionUrl = payload.boundingRegionUrl ||  
  
                               payload.boundingRegionBlobUrl ||  
  
                               payload.diJsonUrl;  
  
  
    if (!boundingRegionUrl) {  
  
      throw new Error('No bounding regions found for this entity');  
  
    }  
  
  
    // 4. Parse URL to extract blob path  
  
    const { container, blobName } = parseBlobUrl(boundingRegionUrl);  
  
  
    // 5. Download JSON from blob storage  
  
    let boundingRegionsData: BoundingRegionsData;  
  
    try {  
  
      const blobBuffer = await downloadBlob(blobName, container, 'docai');  
  
      const jsonString = blobBuffer.toString('utf-8');  
  
      boundingRegionsData = JSON.parse(jsonString);  
  
    } catch (err) {  
  
      throw new Error('Failed to fetch bounding regions from storage');  
  
    }  
  
  
    // 6. Create clean blob URL (without query parameters)  
  
    const blobUrl = boundingRegionUrl.split('?')[0];  
  
  
    // 7. Update database with clean blob URL if it had query parameters  
  
    if (boundingRegionUrl.includes('?')) {  
  
      payload.boundingRegionBlobUrl = blobUrl;  
  
      delete payload.boundingRegionUrl; // Remove old URL with query params  
  
      delete payload.diJsonUrl; // Remove old field name if present       
  
      await EntitiesModel.updateEntityByID(entityId, {  
  
        extractedEntitiesPayload: JSON.stringify(payload)  
  
      });  
  
    }  
  
  
    return {  
  
      entityId: parseInt(entityId),  
  
      boundingRegions: boundingRegionsData,  
  
      blobUrl  
  
    };  
  
  }  
  
};

Task 2.5: Implement Controller Layer

File: src/api/v1/extractedEntity/extractedEntities.controller.ts

Add new method:

  
const EntitiesController = {  
  
  // ... existing methods ...  
  
  
  async getBoundingRegionsForEntity(fastify: FastifyInstance, entityId: string) {  
  
    try {  
  
      return await EntitiesService.getBoundingRegionsForEntity(entityId);  
  
    } catch (err) {  
  
      errorHandler(fastify, err, 'ExtractedEntities.getBoundingRegionsForEntity');  
  
    }  
  
  }  
  
};

Task 2.6: Create Route Handler

File: src/api/v1/extractedEntity/extractedEntities.route.ts

Add new route:

  
fastify.get(`${basePath}/extracted_entities/:entityId/boundingRegions`, {  
  
  schema: entitySchema.getBoundingRegions,  
  
  preValidation: [fastify.authenticate, fastify.authorize],  
  
  config: {  
  
    validRoles: [ROLES.Administrator, ROLES.User],  
  
    validScopes: [SCOPES.EXTRACTED_ENTITIES_READ],  
  
  },  
  
  handler: async (request: FastifyRequest, reply: FastifyReply) => {  
  
    try {  
  
      const { entityId } = request.params as { entityId: string };  
  
  
      const response = await EntitiesController.getBoundingRegionsForEntity(  
  
        fastify,  
  
        entityId  
  
      );  
  
  
      if (!response) {  
  
        return reply.status(404).send({ error: 'Entity not found' });  
  
      }  
  
  
      reply.status(200).send(response);  
  
    } catch (err: any) {  
  
      request.log.error(err);  
  
  
      if (err.message?.includes('No bounding regions found')) {  
  
        return reply.status(404).send({ error: err.message });  
  
      }  
  
  
      if (err.message?.includes('Failed to fetch bounding regions')) {  
  
        return reply.status(500).send({ error: err.message });  
  
      }  
  
  
      reply.status(500).send({ error: 'Internal Server Error' });  
  
    }  
  
  }  
  
});

Acceptance Criteria

Functional Requirements

✅ Endpoint downloads and returns bounding regions JSON from blob storage
✅ Returns 404 if entity doesn’t exist
✅ Returns 404 with specific message if no bounding regions URL found
✅ Returns 500 if blob download fails or JSON is invalid
✅ Database is updated to store clean blob URL if URL with query parameters was stored
✅ Response includes the blob URL for reference
✅ Handles URLs with or without query parameters from database gracefully
✅ Properly parses and validates JSON structure before returning

Non-Functional Requirements

✅ Endpoint requires authentication (bearer token)
✅ Endpoint requires appropriate scopes (EXTRACTED_ENTITIES_READ)
✅ Response time < 5 seconds for JSON files up to 5MB
✅ Proper error logging for troubleshooting
✅ No sensitive data logged (PIDs, credentials)
✅ Handles large JSON files efficiently (streaming if > 10MB)

Testing Requirements

✅ Unit test for blob download helper function
✅ Unit test for URL parsing logic (normalize URLs by removing query parameters)
✅ Unit test for service layer with mocked blob storage
✅ Integration test with actual blob storage (using Azurite)
✅ Test with entity containing no bounding regions URL (404 response)
✅ Test with non-existent entity (404 response)
✅ Test with invalid JSON in blob (500 response)
✅ Test with non-existent blob (500 response)
✅ Test database update when URL with query parameters is normalized

Documentation Requirements

✅ OpenAPI schema updated with new endpoint
✅ Generated OpenAPI docs include request/response examples
✅ Code comments explain blob download and parsing logic
✅ README updated with endpoint usage examples

---

Story 3: Generate Updated OpenAPI Documentation

Story Description

As a API consumer or developer

I want the OpenAPI documentation to reflect the two new endpoints

So that I can understand the API contract and generate client SDKs

Implementation Tasks

Task 3.1: Verify Schema Completeness

File: src/api/v1/extractedEntity/extractedEntities.schema.ts

Ensure both new schemas are properly defined:

getImageUrls schema
getBoundingRegions schema

Task 3.2: Generate OpenAPI Documentation

Command: Run the documentation generation script

  
npm run generate-docs  
  
# or  
  
node scripts/generate-docs-server.ts

Task 3.3: Validate Generated Documentation

File: docs/openapi.yaml and docs/openapi.json

Verify the following are present:

Both endpoints appear under the “Extracted Entities” tag
All request parameters are documented
All response schemas are complete
Error responses (404, 500) are documented
Security requirements are specified

Task 3.4: Test OpenAPI Specification

Tools: Use Swagger Editor or Postman to import and validate the OpenAPI spec

Import docs/openapi.yaml into Swagger Editor
Verify no schema validation errors
Check that example values are reasonable
Test that all references are resolved correctly

Acceptance Criteria

Documentation Requirements

✅ OpenAPI 3.0 specification is valid (no errors in Swagger Editor)
✅ Both new endpoints appear in documentation
✅ All parameters have descriptions
✅ All response schemas are complete and accurate
✅ Security schemes are properly defined
✅ Examples are provided for request/response bodies
✅ Tags properly organize endpoints
✅ Generated documentation matches implementation

Testing Requirements

✅ OpenAPI spec validates against OpenAPI 3.0 schema
✅ Documentation can be imported into Postman without errors
✅ All endpoint paths are unique and correct
✅ Response codes match actual implementation

---

Cross-Story Considerations

Security

Both endpoints require authentication via bearer token
Both endpoints require EXTRACTED_ENTITIES_READ scope
Images streamed directly through authenticated API (no token exposure to client)
No credentials or sensitive data logged
Blob URLs without query parameters are stored in database (non-expiring)

Performance

Image streaming parallelized for multipart responses
Blob download uses efficient buffering for binary data
Database updates are conditional (only when URL normalization needed)
Response times:

- Single Image: < 2 seconds per image

- Multiple Images (multipart): < 5 seconds for 10 images

- Bounding Regions: < 5 seconds for 5MB JSON

Error Handling

Consistent error response format across endpoints
Specific error messages for different failure scenarios
Proper HTTP status codes (404, 500)
Error logging without exposing sensitive details
Graceful degradation when blob storage is unavailable

Data Migration

If existing entities have URLs with query parameters in the database:

First API call will normalize URLs to persistent blob URLs
Database update is transactional
Original functionality preserved (streaming works regardless of stored URL format)
No downtime required for migration

Testing Strategy

Unit Tests: Service layer logic with mocked dependencies
Integration Tests: Full request/response cycle with Azurite
E2E Tests: Real scenarios with database + blob storage
Load Tests: Performance validation with concurrent requests

---

Implementation Order

Phase 1: Foundation (Story 1 - Stream Images)

Update type definitions
Create schema definitions (two endpoints)
Implement service layer
Implement controller layer
Create route handlers
Write unit tests
Write integration tests

Phase 2: Bounding Regions (Story 2)

Update type definitions
Create schema definition
Implement blob download helper
Implement service layer
Implement controller layer
Create route handler
Write unit tests
Write integration tests

Phase 3: Documentation (Story 3)

Verify schema completeness
Generate OpenAPI docs
Validate documentation
Update README with examples

---

Testing Plan

Unit Tests

Location: src/api/v1/extractedEntity/tests/

Test Files to Create:

extractedEntities.imageStreaming.test.ts
extractedEntities.boundingRegions.test.ts
blobStorage.download.test.ts (in src/helpers/tests/)

Test Cases for Image Streaming:

✅ Parse blob URL correctly
✅ Parse URLs with query parameters correctly (extract blob path)
✅ Stream single image with correct headers
✅ Stream multiple images as multipart response
✅ Handle entity not found (404)
✅ Handle no images found (404)
✅ Handle invalid page number (404)
✅ Update database when URLs with query parameters detected
✅ Don’t update database when clean blob URLs already stored
✅ Handle blob storage errors gracefully
✅ Detect correct content-type (PNG vs JPEG)

Test Cases for Bounding Regions:

✅ Download and parse valid JSON from blob
✅ Handle entity not found (404)
✅ Handle no bounding regions URL (404)
✅ Handle blob download failure (500)
✅ Handle invalid JSON in blob (500)
✅ Update database when URL with query parameters detected
✅ Don’t update database when clean blob URL already stored
✅ Handle large JSON files (streaming)

Integration Tests

Location: src/api/v1/extractedEntity/tests/

Test Scenarios:

Happy Path - Single Image Stream:

- Create test entity with blob URLs

- Call GET /extracted_entities/:id/images/1

- Verify binary image data returned

- Verify headers (Content-Type, X-Page-Number, X-Total-Pages)

Happy Path - Multiple Images Stream:

- Create test entity with multiple blob URLs

- Call GET /extracted_entities/:id/images

- Verify multipart response structure

- Verify all images received

- Parse multipart boundaries correctly

Happy Path - Bounding Regions:

- Upload test JSON to Azurite

- Create test entity with blob URL

- Call GET /extracted_entities/:id/boundingRegions

- Verify JSON returned

Migration Path - URL Normalization:

- Create entity with URLs containing query parameters

- Call image streaming endpoint

- Verify database updated with clean blob URLs

Error Scenarios:

- Non-existent entity

- Missing images/bounding regions

- Blob storage unavailable

- Invalid JSON in blob

Load Testing

Tool: Artillery or K6

Scenarios:

100 concurrent requests to /images/:pageNumber endpoint
50 concurrent requests to /images endpoint (multipart)
100 concurrent requests to /boundingRegions endpoint
Mixed workload (image streaming + bounding regions)
Response time < 2s for p95 (single images)
Response time < 5s for p95 (multipart/multiple images)

---

Rollout Plan

Pre-Deployment Checklist

✅ All unit tests passing
✅ All integration tests passing
✅ Code review completed
✅ OpenAPI documentation generated and validated
✅ README updated with new endpoint examples
✅ Security review completed (no PII logged)
✅ Performance testing completed

Deployment Steps

Deploy to development environment
Run smoke tests
Deploy to QA environment
Run full test suite
Deploy to production with feature flag (if available)
Monitor error rates and response times
Enable for all users

Monitoring & Alerts

Track response times for all image streaming endpoints
Monitor error rates (404, 500)
Track database update frequency (URL normalization)
Alert on blob storage connection failures
Alert on response times > 5s
Monitor streaming throughput and memory usage

Rollback Plan

If issues arise:

Disable routes via feature flag
Revert deployment if necessary
Investigate errors using logs
Fix issues and redeploy

---

Dependencies & Prerequisites

Technical Dependencies

✅ Azure Blob Storage configured (AZURE_STORAGE_CONNECTION_STRING_DOCAI)
✅ @azure/storage-blob package installed
✅ Existing blobStorage.ts helper functions
✅ Database access to ExtractedEntities table
✅ Authentication middleware (fastify.authenticate)
✅ Authorization middleware (fastify.authorize)

Data Prerequisites

ExtractedEntities records must have image URLs or bounding region URLs in extractedEntitiesPayload
Blobs must exist in Azure Storage
JSON files must be valid and follow expected schema

Team Dependencies

Backend developer for implementation
QA engineer for testing
DevOps for deployment
Technical writer for documentation review

---

Success Metrics

Functional Metrics

✅ 100% of entities with images can stream binary image data
✅ 100% of entities with bounding regions can retrieve JSON data
✅ 100% of URLs in database normalized to persistent blob URLs after first access

Performance Metrics

✅ p95 response time < 2s for single image streaming endpoint
✅ p95 response time < 5s for multiple images (multipart) endpoint
✅ p95 response time < 5s for bounding regions endpoint
✅ 99.9% uptime for all endpoints

Quality Metrics

✅ 85%+ code coverage
✅ 0 critical security vulnerabilities
✅ 0 PII logged in production logs

---

Future Enhancements

Potential Improvements (Out of Scope for MVP)

Caching Layer: Cache image data in Redis for frequently accessed images to reduce blob storage calls
Batch Endpoint: POST /extracted_entities/batch/images to get images for multiple entities
Lazy Loading: Stream images one at a time with pagination for entities with many images
CDN Integration: Serve images through Azure CDN for faster delivery
Image Metadata: Return image dimensions, file size, format in additional headers
Bounding Regions Filtering: Query parameter to filter regions by confidence score or content
Webhooks: Notify clients when new images or bounding regions are available
Image Transformations: Resize, crop, or format conversion on-the-fly
Progressive JPEG: Stream images in progressive format for faster rendering
Thumbnail Generation: Generate and cache thumbnail versions for preview

---

Appendix

Example Request/Response

Example 1: Stream Single Image

  
GET /v1/extracted_entities/12345/images/1  
  
Authorization: Bearer <token>  
  
  
Response (200 OK):  
  
Content-Type: image/png  
  
Content-Disposition: inline; filename="page_1.png"  
  
X-Page-Number: 1  
  
X-Total-Pages: 5  
  
  
[Binary PNG image data]

Example 1b: Stream All Images (Multipart)

  
GET /v1/extracted_entities/12345/images  
  
Authorization: Bearer <token>  
  
  
Response (200 OK):  
  
Content-Type: multipart/mixed; boundary=----ImageBoundary  
  
X-Total-Pages: 2  
  
  
------ImageBoundary  
  
Content-Type: image/png  
  
Content-Disposition: attachment; filename="page_1.png"  
  
X-Page-Number: 1  
  
  
[Binary PNG data for page 1]  
  
------ImageBoundary  
  
Content-Type: image/png  
  
Content-Disposition: attachment; filename="page_2.png"  
  
X-Page-Number: 2  
  
  
[Binary PNG data for page 2]  
  
------ImageBoundary--

Example 2: Get Bounding Regions```bash

GET /v1/extracted_entities/12345/boundingRegions

Authorization: Bearer <token>  
  
  
Response (200 OK):  
  
{  
  
  "entityId": 12345,  
  
  "boundingRegions": {  
  
    "pages": [  
  
      {  
  
        "pageNumber": 1,  
  
        "regions": [  
  
          {  
  
            "content": "Invoice Number: INV-001",  
  
            "boundingBox": [0.1, 0.2, 0.5, 0.3],  
  
            "confidence": 0.98  
  
          }  
  
        ]  
  
      }  
  
    ],  
  
    "extractedAt": "2026-01-05T08:30:00Z",  
  
    "version": "1.0"  
  
  },  
  
  "blobUrl": "https://docaistorage.blob.core.windows.net/invoicepdf/doc123_bounding_regions.json"  
  
}

Example 3: Entity Not Found

  
GET /v1/extracted_entities/99999/images  
  
Authorization: Bearer <token>  
  
  
Response (404 Not Found):  
  
{  
  
  "error": "Entity not found"  
  
}

Example 4: No Images Found

  
GET /v1/extracted_entities/12345/images  
  
Authorization: Bearer <token>  
  
  
Response (404 Not Found):  
  
{  
  
  "error": "No images found for this entity"  
  
}

Example 5: Page Number Out of Range

  
GET /v1/extracted_entities/12345/images/99  
  
Authorization: Bearer <token>  
  
  
Response (404 Not Found):  
  
{  
  
  "error": "Page number exceeds available pages"  
  
}

Example 6: UI Rendering Implementation

  
// React component to render streamed images  
  
async function displayInvoiceImages(entityId: number) {  
  
  try {  
  
    // Fetch single image for preview  
  
    const response = await fetch(  
  
      `/v1/extracted_entities/${entityId}/images/1`,  
  
      {  
  
        headers: {  
  
          'Authorization': `Bearer ${token}`  
  
        }  
  
      }  
  
    );  
  
  
    if (response.ok) {  
  
      const blob = await response.blob();  
  
      const imageUrl = URL.createObjectURL(blob);  
  
      const totalPages = response.headers.get('X-Total-Pages');  
  
  
      // Display image  
  
      document.getElementById('preview-image').src = imageUrl;  
  
      console.log(`Showing page 1 of ${totalPages}`);  
  
    }  
  
  } catch (error) {  
  
    console.error('Failed to load image:', error);  
  
  }  
  
}  
  
  
// Fetch all images as multipart (for download/processing)  
  
async function downloadAllImages(entityId: number) {  
  
  const response = await fetch(  
  
    `/v1/extracted_entities/${entityId}/images`,  
  
    {  
  
      headers: {  
  
        'Authorization': `Bearer ${token}`  
  
      }  
  
    }  
  
  );  
  
  
  // Parse multipart response  
  
  const contentType = response.headers.get('Content-Type');  
  
  const boundary = contentType.split('boundary=')[1];  
  
  const text = await response.text();  
  
  
  // Split by boundary and extract images  
  
  const parts = text.split(`--${boundary}`);  
  
  const images = [];  
  
  
  for (const part of parts) {  
  
    if (part.includes('Content-Type: image')) {  
  
      // Extract image data and headers  
  
      const [headers, ...dataLines] = part.split('\\r\\n\\r\\n');  
  
      const pageMatch = headers.match(/X-Page-Number: (\\d+)/);  
  
      const pageNumber = pageMatch ? parseInt(pageMatch[1]) : 0;  
  
  
      images.push({  
  
        pageNumber,  
  
        data: dataLines.join('\\r\\n\\r\\n')  
  
      });  
  
    }  
  
  }  
  
  
  return images;  
  
}

---

Glossary

| Term | Definition |

|------|------------|

| Blob URL | Persistent URL to a blob without query parameters or tokens (requires separate authentication) |

| Image Streaming | Server-side process of downloading images from blob storage and returning binary data directly to client |

| Multipart Response | HTTP response format that contains multiple parts (images) separated by boundaries |

| Entity | Record in ExtractedEntities table representing processed invoice data |

| Bounding Region | Rectangle coordinates identifying text location in a document image |

| Document Intelligence | Azure service for OCR and document analysis (formerly Form Recognizer) |

| Azurite | Local Azure Storage emulator for development and testing |

---

References

---

Change Log

|------|---------|---------|--------|

| 2026-01-05 | 1.0 | Initial story design created | GitHub Copilot |

---

Document Status: ✅ Ready for Review

Next Step: Team review and story estimation

Estimated Total Story Points: 13 points (Story 1: 5pts, Story 2: 6pts, Story 3: 2pts)