Skip to navigation

File Intake Service Evaluation

This documents the evaluation of the File Intake Service as part of User Story 728637: SPIKE: Evaluate the File Intake Service Suggested changes or potential enhancements have been identified for each category. The items have been marked with Severity to indicate importance.

Documentation

Below are the suggested changes to the documentation.

No.DocumentSeverityDescriptionReferences
1Sequence DiagramMediumAdd step to send problem event when there are validation issuesFile Intake Sequence Diagram
2Sequence DiagramLowAdd hashValue to accepted-file event propertiesFile Intake Sequence Diagram
3WikiMediumRemove submitted-file eventFile Intake Service - Overview
4WikiLowAdd file metadata info including possible duplicates behaviorFile Intake Service - Overview
5ReadmeMediumRemove file drop description, update the event topics and align the flow description with the latest behavior
6ReadmeLowContributing section: Update the description on the “short term” process that was done while pipeline was not yet available

Design

The following are potential design improvements:

Possible Enhancements

No.SeverityDescription
1MediumIs the concept of multiple applications per client still applicable? Currently, only Docs AI is the supported “application”
2MediumOnly valid files are now saved. Only internalUrl is used. Consider removal of externalUrl. Note that this impacts status-check as it is required for creation of initial status record 
3LowClient registration exists in both Fluxx and Docs AI. Consider consolidating them or make them consistent (Ex. Fluxx’s ClientID is GUID vs. Doc AI int)

Code

Potential optimizations and improvements have been included in the table below.

Possible Enhancements

No.SeverityDescription
1MediumHave one handling and response type for single and multiple file uploads
2MediumMessageReceiver: Consistent setup of service bus topics and subscribers across all services.
3MediumMessaging: If possible, use common library or framework to standardize sending and receiving messages
4MediumToggle on/off logging of service bus messages. When there are more users, the messages may fill up logs. Consider risks since full message is logged
5MediumReduce signed blob expiry. Current is 24 hours (used by extractor)
6MediumUse Cradle for type safety, similar to what was done in registration service
7LowSince DocsAI has duplicate check based on hashValue, check if this is still needed in the intake?
8LowCleanup test API v1/test-500
Note: Change is available in 696268 working branch but not yet merged pending availability of framework 2.1
9LowFor swagger documentation of v1/file/metadata/:fileId API, add custom tag instead of “default”
10MediumClarify next steps for the openapi.yml endpoint. For example, integration with DevOps