Implement logging in web api documents ai
Business Value
-
Structured Logging: Consistent log format across all services for easier parsing and analysis
-
Log Levels: Proper severity levels (debug, info, warn, error) for better filtering and alerting
-
Centralized Logging: Integration with Azure Monitor and Application Insights
-
Performance: Better performance compared to console.log in production environments
-
Debugging: Child loggers with request context for better traceability
-
Standards Compliance: Alignment with organizational logging standards
Current State
Existing Console.log Usage
The DocumentAI service currently uses console.log statements throughout the codebase:
Issues with Current Approach
-
No structured logging format - difficult to parse logs programmatically
-
No log level filtering - all logs output at same level
-
No request correlation - cannot trace logs across a request lifecycle
-
Poor integration with Azure Monitor and Application Insights
-
Inconsistent log formatting across the application
-
No child logger support for request context
-
Console output not optimized for production environments
Future State
Framework Logger Implementation
All logging will use the framework logger pattern established in fluxx-file-intake:
Log Levels
-
debug: Detailed diagnostic information for troubleshooting
-
info: General informational messages about application flow
-
warn: Warning messages for potentially problematic situations
-
error: Error messages for failures and exceptions
Design
Architecture Pattern
Follow the exact pattern established in the fluxx-file-intake repository:
1. Logger Type Definition
Define a Logger interface that picks specific methods from the framework logger.
2. App Container
Central app container that holds the logger instance and makes it available throughout the application.
3. Missing Logger Placeholder
Placeholder that throws errors if logger is accessed before initialization.
4. Logger Registration
Function to register the framework logger into the app container during startup.
5. Usage Pattern
Import and use app.log throughout the codebase for all logging operations.
Component Architecture
Implementation Steps
Phase 1: Setup Logger Infrastructure
Task 1.1: Create Logger Type Definition
File: src/internal/Logger.d.ts
Rationale: Define a TypeScript interface that extracts only the logging methods we need from the framework logger. This provides type safety and clear contract.
Task 1.2: Create Missing Logger Placeholder
File: src/internal/frameworkAdapter/missingLogger.ts
Rationale: Provides fail-fast behavior if code attempts to log before the logger is properly initialized. This prevents silent failures and makes initialization errors obvious.
Task 1.3: Create Register Logger Function
File: src/internal/frameworkAdapter/registerLogger.ts
Rationale: Safely extracts the framework logger and assigns it to our app container with proper validation to ensure the framework is fully initialized.
Task 1.4: Create App Container
File: src/internal/app.ts
Rationale: Central container that provides application-wide access to the logger. Uses the missing logger as default until properly initialized.
Phase 2: Update Application Entry Point
Task 2.1: Initialize Logger in index.ts
File: src/index.ts
Add after framework initialization:
Rationale: Register the logger as early as possible in the application lifecycle so it’s available for all subsequent initialization steps.
Phase 3: Migrate Console.log Statements
Task 3.1: Create Migration Checklist
Identify all files with console.log usage:
Common patterns to migrate:
| Current Pattern | New Pattern | Log Level |
|----------------|-------------|-----------|
| console.log('Info message') | app.log.info('Info message') | info |
| console.error('Error:', error) | app.log.error('Error message', error) | error |
| console.warn('Warning') | app.log.warn('Warning message') | warn |
| console.debug('Debug') | app.log.debug('Debug message') | debug |
| console.log('Starting...') | app.log.info('Starting...') | info |
Task 3.2: Migrate Controllers
Example transformation in a controller file:
Before:
After:
Task 3.3: Migrate Services
Example transformation in OpenAI service:
Before:
After:
Task 3.4: Migrate Form Recognizer Service
Before:
After:
Task 3.5: Migrate Utility Functions
Before:
After:
Task 3.6: Update Middleware
Before:
After:
Phase 4: Remove All Console.log Statements
Task 4.1: Add ESLint Rule
File: eslint.config.js or .eslintrc.json
Rationale: Prevent future console.log usage by enforcing the rule at the linting level.
Task 4.2: Verify No Console.log Remains
Testing Strategy
Unit Tests
Test 1: Missing Logger Throws Error
File: test/unit/internal/frameworkAdapter/missingLogger.test.ts
Purpose: Verify that attempting to use the logger before registration throws descriptive errors.
Test 2: Logger Registration Success
File: test/unit/internal/frameworkAdapter/registerLogger.test.ts
Purpose: Verify that logger registration correctly assigns the framework logger and validates inputs.
Test 3: Logger Usage in Services
File: test/unit/services/openAIService.test.ts
Purpose: Verify that services correctly use the logger and log at appropriate levels.
Test 4: Logger Usage in Controllers
File: test/unit/controllers/documentController.test.ts
Purpose: Verify that controllers log requests and errors appropriately.
Test 5: No Console.log Usage
File: test/unit/noConsoleLogs.test.ts
Purpose: Automated verification that no console.log statements exist after migration.
Integration Tests
Test 6: Logger Integration with Framework
File: test/integration/logger.test.ts
Purpose: Verify that the logger integrates correctly with the framework and supports all required operations.
Contract Tests
Test 7: Logger Output Format
File: test/contract/logger-format.test.ts
Purpose: Verify that the logger produces correctly formatted structured logs.
Acceptance Criteria
Functional Requirements
-
All
console.logstatements replaced withapp.log.info -
All
console.errorstatements replaced withapp.log.error -
All
console.warnstatements replaced withapp.log.warn -
All
console.debugstatements replaced withapp.log.debug -
Logger initialized in application entry point (
index.ts) -
Logger available throughout application via
app.log -
Structured JSON logs produced in all environments
-
Error objects logged with full stack traces
Technical Requirements
-
Logger.d.tstype definition created -
missingLogger.tsplaceholder created -
registerLogger.tsfunction created -
app.tscontainer updated with logger -
No direct
console.*usage in source code -
TypeScript type safety maintained for logger
-
Logger registered before any logging operations
-
Proper error handling for missing logger
Code Quality Requirements
-
ESLint
no-consolerule enabled and enforced -
All files import
app.logwhere logging is needed -
Log messages are clear and descriptive
-
Appropriate log levels used (debug for verbose, info for general, warn for issues, error for failures)
-
Sensitive data not logged (passwords, tokens, PII)
-
Log messages follow consistent format and style
Testing Requirements
-
Unit tests for
missingLoggerimplementation -
Unit tests for
registerLoggerfunction -
Unit tests verify services use logger correctly
-
Unit tests verify controllers use logger correctly
-
Integration tests verify logger framework integration
-
Contract tests verify structured log format
-
Automated test to verify no console.log usage
-
All tests pass in CI/CD pipeline
-
Test coverage > 80% for logger modules
Documentation Requirements
-
Code comments explain logger initialization
-
README updated with logging guidelines
-
Examples of proper log usage documented
-
Migration completed and documented
-
Team trained on new logging pattern
Deployment Requirements
-
Application starts successfully with new logger
-
Logs appear in Azure Application Insights
-
Log levels filter correctly in production
-
No performance degradation from logging
-
Rollback plan documented
Non-Functional Requirements
Performance
-
Logger operations should add < 1ms overhead per log statement
-
Structured logging should not significantly impact throughput
-
Application startup time should not increase > 100ms
Reliability
-
Application should fail fast with clear error if logger not initialized
-
Logger should handle errors gracefully without crashing application
-
Logging failures should not prevent application operation
Observability
-
All logs should include timestamp, level, and message
-
Error logs should include stack traces
-
Logs should be parseable by Azure Monitor
-
Log correlation IDs should be preserved across async operations
Maintainability
-
Log statements should be self-documenting
-
Log levels should be consistent across codebase
-
Adding logging should be straightforward for new developers
-
Logger interface should remain stable across updates
Migration Patterns
Pattern 1: Simple Info Logs
Pattern 2: Logs with Variables
Pattern 3: Error Logging
Pattern 4: Structured Logging with Context
Pattern 5: Debug Logging
Pattern 6: Warning Logs
Risks and Mitigation
Risk 1: Missing Console.log Statements
Impact: Medium - Some logs might be missed during migration
Mitigation:
-
Use grep/ripgrep to find all console.* usage
-
Enable ESLint no-console rule to prevent regressions
-
Code review checklist includes verification
-
Automated test to detect console.log usage
Risk 2: Incorrect Log Levels
Impact: Low - Logs at wrong severity level
Mitigation:
-
Document log level guidelines clearly
-
Code review focuses on appropriate log levels
-
Team training on when to use each level
Risk 3: Performance Impact
Impact: Low - Increased logging overhead
Mitigation:
-
Benchmark logging performance
-
Use appropriate log levels (avoid excessive debug logs)
-
Configure log levels per environment
Risk 4: Breaking Changes During Migration
Impact: Medium - Application errors during deployment
Mitigation:
-
Test thoroughly in development environment
-
Gradual migration file by file
-
Maintain backward compatibility during transition
-
Feature flag for new logger if needed
Rollout Plan
Phase 1: Infrastructure (Week 1)
-
Create logger infrastructure files
-
Update application entry point
-
Deploy to development environment
-
Verify logger initialization
-
Run integration tests
Phase 2: Core Services Migration (Week 2)
-
Migrate OpenAI service
-
Migrate Form Recognizer service
-
Migrate storage services
-
Test services individually
-
Deploy to QA environment
Phase 3: Controllers and Utilities (Week 3)
-
Migrate controllers
-
Migrate utility functions
-
Migrate middleware
-
Enable ESLint no-console rule
-
Deploy to UAT environment
Phase 4: Cleanup and Production (Week 4)
-
Verify no console.log statements remain
-
Complete all tests
-
Update documentation
-
Deploy to production
-
Monitor logs in Azure Application Insights
Success Metrics
Technical Metrics
-
Zero console.log statements in source code
-
100% of log statements use framework logger
-
All log levels used appropriately (info, warn, error, debug)
-
Log output appears in Azure Application Insights
-
No logging-related runtime errors
Quality Metrics
-
Test coverage > 80% for logger modules
-
All unit tests passing
-
All integration tests passing
-
ESLint no-console rule enforced
-
Code review approval on all changes
Operational Metrics
-
Application starts successfully with new logger
-
Logs searchable in Azure Monitor
-
Log queries work with structured format
-
No performance degradation (< 5% overhead)
-
Mean time to find logs reduced by 50%
Team Metrics
-
All developers trained on new logging pattern
-
Documentation complete and reviewed
-
Zero questions about how to log after training
-
Consistent log usage across all new code
Related Documentation
Checklist for Developers
During migration, verify each file:
-
Replace all
console.logwithapp.log.info -
Replace all
console.errorwithapp.log.error -
Replace all
console.warnwithapp.log.warn -
Replace all
console.debugwithapp.log.debug -
Import
appfrom./internal/app.js -
Use appropriate log level for message severity
-
Include context objects for structured logging
-
Do not log sensitive data (passwords, tokens, PII)
-
Test that logging works correctly
-
Verify ESLint passes with no console warnings
Story Points: 8
Complexity: Medium
Effort: 2 weeks
Dependencies: Framework-nodejs-logging, Framework-nodejs-extensions
Definition of Done
-
All acceptance criteria met
-
All console.log statements migrated
-
Code reviewed and approved
-
All tests passing (unit, integration, contract)
-
ESLint no-console rule enabled
-
Deployed to all environments
-
Documentation updated
-
Team training completed
-
Logs visible in Azure Application Insights
-
Post-deployment validation successful
-
Stakeholder sign-off received