FLUXX Splitter Service---Low Level Design
FLUXX Splitter Service---Low Level Design
Document Version
-
Version: 1.0
-
Date: March 6, 2026
-
Author: System Design
-
Status: Design Phase
1. Overview
1.1 Purpose
The FLUXX Splitter Service is a standalone microservice that takes classified PDF documents and splits them into individual form files based on classification results. This service is extracted from the splitting logic in the existing ClassifyFileTool implementation to provide document splitting capabilities as an independent, reusable service.
1.2 Scope
This service handles:
-
Reading classification results from previous classification phase
-
Extracting page ranges for individual forms
-
Splitting multi-form PDFs into separate PDF files
-
Handling contiguous and non-contiguous page ranges
-
Creating file metadata records for split files
-
Uploading split files to blob storage
-
Generating sanitized filenames from form types
1.3 Reference Implementation
This design is based on the existing implementation in:
-
/backend/src/api/v1/agent/tools/ClassifyFileTool.ts(lines 237-368) -
/backend/src/utils/pdfProcessor.ts(splitPDFByPageRangesfunction)
2. Architecture
2.1 System Context
2.2 Service Components
2.2.1 API Layer
-
Express.js REST API
-
Request validation
-
Authentication and authorization
-
Response formatting
2.2.2 Split Orchestrator
-
Orchestrates the splitting workflow
-
Handles classification result parsing
-
Builds page ranges from form classifications
-
Manages error recovery and rollback
2.2.3 PDF Splitting Engine
-
PDF page extraction using pdf-lib
-
Multi-range splitting for non-contiguous pages
-
PDF document assembly
-
Buffer management
2.2.4 File Management Layer
-
Blob storage integration
-
File metadata creation
-
Filename generation and sanitization
-
Parent-child file relationships
3. API Design
3.1 Endpoints
3.1.1 POST /api/v1/split
Request:
Response (Success - 200):
Response (Error - 400/500):
3.1.2 POST /api/v1/split-batch
Batch splitting for multiple documents.
Request:
Response:
3.1.3 GET /api/v1/split/:split_id
Retrieves information about a previous split operation.
Response:
3.1.4 GET /health
Health check endpoint.
Response:
4. Core Algorithm
4.1 Splitting Workflow
4.2 Page Range Building Logic
4.3 PDF Splitting Logic (pdf-lib)
5. Data Models
5.1 Split Request
5.2 Split Response
5.3 Internal Models
6. Error Handling
6.1 Error Categories
| Error Code | HTTP Status | Description | Recovery |
|------------|-------------|-------------|----------|
| INVALID_CLASSIFICATION | 400 | Invalid classification result | Fix classification data |
| INVALID_PAGE_RANGES | 400 | Page ranges out of bounds | Reclassify document |
| FILE_NOT_ACCESSIBLE | 404 | Cannot download original PDF | Verify file_url |
| PDF_LOAD_ERROR | 500 | Failed to load PDF with pdf-lib | Check PDF integrity |
| SPLIT_FAILED | 500 | PDF splitting operation failed | Retry or manual review |
| STORAGE_ERROR | 503 | Blob storage failure | Retry with backoff |
| PARTIAL_SPLIT_SUCCESS | 207 | Some splits failed, others succeeded | Review partial results |
6.2 Error Response Format
6.3 Partial Failure Handling
When some splits succeed but others fail:
-
Return HTTP 207 (Multi-Status)
-
Include successful split files in response
-
Detail which splits failed and why
-
Do NOT delete successful splits (allow partial success)
-
Client can retry failed splits individually
7. Configuration
7.1 Environment Variables
7.2 Service Configuration
8. Implementation Pseudocode
8.1 Main Split Function
8.2 Filename Sanitization
8.3 Page Range Validation
9. Deployment
9.1 Docker Container
9.2 Kubernetes Deployment
10. Testing Strategy
10.1 Unit Tests
-
Page range building logic
-
Page range validation
-
Filename sanitization
-
Error handling for invalid inputs
10.2 Integration Tests
-
PDF splitting with pdf-lib
-
Blob storage upload
-
Database file metadata creation
-
End-to-end split workflow
10.3 Test Cases
| Test Case | Input | Expected Output |
|-----------|-------|-----------------|
| Single Form | Classification with 1 form | No splits created |
| Multi-Form (2 forms) | 1040 + Schedule C | 2 separate PDFs |
| Non-contiguous Pages | Form with page_ranges | Multiple PDFs for same form type |
| Invalid Page Range | page_start=10, page_end=5 | Error: INVALID_PAGE_RANGES |
| Page Range Out of Bounds | page_end=100, but PDF has 50 pages | Error: INVALID_PAGE_RANGES |
| Large Multi-Form | 50 forms | 50 separate PDFs |
| Special Characters in Form Type | “Form 1099-MISC” | Sanitized filename |
| Partial Split Failure | Valid data, but storage fails for 1 file | HTTP 207, partial results |
11. Performance Considerations
11.1 Expected Performance
-
Small PDF (2 forms, 10 pages): 1-2 seconds
-
Medium PDF (10 forms, 50 pages): 3-5 seconds
-
Large PDF (50 forms, 200 pages): 10-15 seconds
11.2 Performance Factors
-
PDF page copying with pdf-lib (CPU intensive)
-
Buffer creation for each split file
-
Blob storage upload time (network I/O)
-
Database inserts for file metadata
11.3 Optimization Opportunities
-
Parallel uploads: Upload split files to blob storage concurrently
-
Batch database inserts: Insert multiple file metadata records in single transaction
-
Compression: Compress PDF buffers before upload
-
Caching: Cache PDF documents for repeated split operations
11.4 Resource Requirements
-
Memory: ~50MB per split operation + PDF file size
-
CPU: Moderate (pdf-lib page copying)
-
Network: High (downloading original, uploading splits)
12. Monitoring & Observability
12.1 Metrics
-
Split requests per minute
-
Average split time
-
Files created per split
-
Blob upload latency
-
Database insert latency
-
Error rate by error code
-
Partial failure rate
12.2 Logging
-
Structured JSON logging
-
Input validation errors
-
Split operation details (page ranges, form types)
-
File creation success/failure
-
Error stack traces
12.3 Alerting
-
Error rate > 5%
-
Average split time > 30 seconds
-
Blob storage failures > 2%
-
Database connectivity issues
-
Partial failure rate > 10%
13. Security
13.1 Authentication
-
API key authentication
-
Azure AD integration (optional)
-
User/client authorization
13.2 Data Protection
-
TLS in transit
-
Encrypted blob storage
-
Temporary buffer cleanup
-
PII redaction in logs
13.3 Input Validation
-
Page range bounds checking
-
Form type sanitization
-
File size limits
-
PDF format validation
13.4 Rate Limiting
-
Per-user split limits
-
Global service rate limits
-
Maximum splits per request
14. Integration with Classification Service
14.1 Workflow Integration
14.2 Data Flow
- Classification Phase:
- Client sends PDF to classification service
- Receives classification result with form types and page ranges
- Decision Phase:
- Client checks is_multi_form flag
- If true, proceeds to splitting
- Splitting Phase:
- Client sends classification result + file URL to splitter service
- Receives array of split file references
- Processing Phase:
- Client processes each split file individually
- Each file is a single form type
14.3 API Contract
The splitter service expects the exact output format from the classification service:
15. Migration from Existing System
15.1 Migration Path
-
Deploy FLUXX Splitter Service alongside existing system
-
Update ClassifyFileTool to optionally call splitter service instead of inline splitting
-
Add feature flag to control routing
-
Run in parallel for validation period
-
Gradually migrate all splitting to new service
-
Remove inline splitting code from ClassifyFileTool
15.2 Backward Compatibility
-
Maintain same file metadata structure
-
Preserve parent_file_id relationships
-
Keep form_index and form_type fields
-
Support existing blob storage paths
16. Future Enhancements
16.1 Phase 2 Features
-
Parallel split file uploads
-
Batch database inserts
-
PDF compression options
-
Custom filename templates
-
Dry-run mode (validate without creating files)
16.2 Phase 3 Features
-
Merge operation (combine forms back into single PDF)
-
Re-ordering of forms before splitting
-
Watermarking of split files
-
PDF/A conversion for long-term archival
-
Split preview (return file sizes without creating)
17. Dependencies
17.1 NPM Packages
17.2 External Services
-
Azure Blob Storage (for file storage)
-
Database (for file metadata - SQL Server or PostgreSQL)
18. Comparison with Current Implementation
18.1 Similarities
-
Same pdf-lib splitting logic
-
Same filename sanitization approach
-
Same file metadata structure
-
Same parent-child file relationships
18.2 Differences
| Aspect | Current (ClassifyFileTool) | FLUXX Splitter Service |
|--------|---------------------------|------------------------|
| Coupling | Tightly coupled with classification | Loosely coupled, independent service |
| Reusability | Only via tool invocation | Standard REST API, any client |
| Error Handling | Fails entire classification if split fails | Supports partial success, better error detail |
| Batch Support | One document at a time | Batch splitting supported |
| Monitoring | Limited logging | Comprehensive metrics and logging |
| Scalability | Scales with entire backend | Independent horizontal scaling |
| Testing | Integration tests only | Unit + integration tests |
19. Rollback Strategy
19.1 Rollback Triggers
-
Error rate > 10%
-
Data corruption detected
-
Performance degradation > 50%
-
Critical bug discovered
19.2 Rollback Process
-
Switch feature flag to route traffic back to ClassifyFileTool
-
Stop FLUXX Splitter Service
-
Verify inline splitting is working
-
Investigate and fix issue
-
Re-deploy with fix
-
Gradually re-enable new service
19.3 Data Cleanup
-
Split files created by service have traceable metadata
-
Can query by creation timestamp to identify service-created files
-
Rollback does not require deleting split files (backward compatible)
20. Conclusion
The FLUXX Splitter Service provides a robust, scalable solution for PDF document splitting based on classification results. By separating splitting from classification, we achieve:
Benefits:
-
Modularity: Each service has a single, well-defined responsibility
-
Reusability: Any client can use the splitter, not just Paperboy
-
Scalability: Split operations can scale independently from classification
-
Observability: Dedicated metrics and monitoring for splitting operations
-
Testability: Easier unit and integration testing
-
Flexibility: Clients can choose when and how to split
Key Design Decisions:
-
Stateless design: Enables horizontal scaling
-
Standard REST API: Easy integration
-
Partial failure support: Better error resilience
-
Comprehensive validation: Guards against invalid inputs
-
Cloud-native: Designed for Kubernetes with Azure services
Next Steps:
-
Implement service following this design
-
Create comprehensive test suite
-
Deploy to staging environment
-
Integration testing with Classification Service
-
Performance benchmarking
-
Production deployment with feature flag
-
Gradual traffic migration
This service complements the FLUXX Classification Service to provide a complete document processing pipeline from classification to individual form extraction.