Process Flow

Where is Document-splitter service in the high level process flow?

https://lucid.app/lucidchart/c0c23b5d-5787-496c-afff-dafb31d6f693/edit?invitationId=inv_21daa08b-cb9b-4082-b27d-6c68118994d6&page=0_0#

Document Splitter Event flow

Azure Service Bus                Document Splitter                
─────────────────                ─────────────────                
                                 ┌─────────────────┐
classified-file topic ──────────►│ messageReceiver │
                                 └────────┬────────┘
                                          │ routes to handler
                                          ▼
                                 ┌─────────────────────────────┐
                                 │   classifiedFileHandler     │
                                 │                             │
                                 │ 1. validate event fields    │
                                 │ 2. create job (SQL)         │
                                 │ 3. download classification ─┼────────────────────► [GET classifiedOutputUrl]
                                 │ 4. download source PDF ─────┼────────────────────► [GET internalUrl]
                                 │ 5. validate page ranges     │
                                 │ 6. split PDF (pdf-lib)      │
                                 │ 7. upload split files ──────┼──── HTTP POST ──────► File Intake Service
                                 │    (parallel)               │                           │
                                 │                             │◄── blob URLs ─────────────┘
                                 │ 8. publish split-file ──────┼──► split-file topic (one per file)
                                 │    events                   │
                                 │ 9. upload manifest JSON ────┼────────────────────► [PUT splits container]
                                 │ 10. publish result event ───┼──► split-file-result topic
                                 │ 11. complete job (SQL)      │
                                 └─────────────────────────────┘
                                          │ on any error
                                          ▼
                                 problem topic ──────────────────► Status / Alert service

On Every Incoming Message

Step 1 , Receive and validate the event

• Deserialises the FileClassifiedEvent from the Service Bus message body
• Validates required fields: applicationId, clientApplicationId, clientApplicationClassifierId, fileId, internalUrl, classifiedOutputUrl
• Rejects messages with missing fields and publishes a ProblemEvent (HTTP 400)

Step 2 , Create a job record in SQL

• Inserts a row into [splitter].[Job] with status = 'Started'
• If the job already exists (retry scenario), upserts it
• Stores the original fileId for correlation

Step 3 , Download the classification JSON

• Parses the classifiedOutputUrl to extract container name and blob path
• Downloads the JSON file from Azure Blob Storage
• Deserialises it as a ClassificationResult:
  • is_multi_form , whether the PDF contains multiple forms
  • num_pages , total page count of the PDF
  • forms[] , array of form objects, each with form_type, page_start, page_end (or page_ranges[] for unknown form types), confidence

Step 4 , Download the source PDF (parent file)

• Parses internalUrl to extract container name and blob path
• Downloads the PDF as a raw binary buffer from Azure Blob Storage

Step 5 , Validate page ranges

Ensures the classification metadata is coherent before touching the PDF:

If any check fails → publish ProblemEvent (HTTP 400) and set job status to Failed.

Step 6 , Split (or pass through) the PDF

Multi-form documents (is_multi_form = true):

• For each form in the classification:
• Extracts the specified pages using pdf-lib
• Saves the result as an in-memory Buffer
• Generates a filename: {timestamp}_{baseName}_pg{start}-{end}_{safeFormType}.pdf
• Sanitises form_type (replaces any character that is not alphanumeric, -, or _ with -)

Single-form documents (is_multi_form = false):

• No splitting is performed
• The original PDF buffer is used as-is
• Defaults page_start = 1, page_end = num_pages

Extra / unknown form types:

• Some forms have page_ranges[] instead of a single page_start / page_end
• The splitter merges those non-contiguous page ranges into a single PDF output

Step 7 , Upload each split file via the File Intake service

• HTTP POST to {FILE_INTAKE_BASE_URL}/v1/upload/{applicationId}?parentFileId={fileId}&clientApplicationClassifierId={classifierId}
• Body: multipart/form-data with the PDF buffer
• The file intake service:
  • Assigns a new child fileId
  • Stores the file in blob storage
  • Returns a blob URL and file metadata
• All uploads happen in parallel (Promise.allSettled)
• Any single upload failure is captured, it will send a Problem event ; it does not cancel other uploads. If one or multiple files encounter an error, the Job will result to Failed.

Step 8 , Publish one FileSplitFileEvent per split file

• Published to the split-file topic on Azure Service Bus
• Contains: clientApplicationId, the new blob URL (internalUrl), fileLocationUrl, timestamp
• Downstream extractors subscribe to this topic and know to process each message as one form

Step 9 , Build and upload the split manifest

• Generates a JSON document listing all split results:
  • Per item: fileName, formType, formIndex, pageStart, pageEnd, blobUrl
• Uploads to the splits blob container at path:
  • {clientApplicationId}/splitResults/{clientApplicationClassifierId}/{timestamp}_{baseName}_splitResult.json

Step 10 , Publish the FileSplitFileResultEvent

• Published to the split-file-result topic
• Contains: clientApplicationId, fileId, parentFileBlobUrl, classificationUrl, totalSplitFiles, splitFiles[], timestamp
• Gives consumers a single message summarising the entire split operation

Step 11 , Update job status in SQL

• Updates the row in [splitter].[Job] to status = 'Completed'
• Stores splitManifestBlobUrl (the URL of the manifest uploaded in step 9)
• Sets completedAt timestamp

On Any Error