Migrate DocumentAI Service from Environment Variables to Azure App Configuration
The DocumentAI service currently uses the following .env configuration:
Issues with Current Approach
-
Configuration values hardcoded in deployment files or environment-specific
.envfiles -
Secrets exposed in plain text in environment variables
-
No audit trail for configuration changes
-
Configuration changes require redeployment
-
Potential for configuration drift across environments
Azure App Configuration Keys
All configuration will be stored in Azure App Configuration with the DOCAI_ prefix to prevent collisions:
Design
Architecture Pattern
Follow the pattern established in the fluxx-file-intake repository:
1. App Configuration Initialization
Use the framework-nodejs-appconfig and framework-nodejs-core packages to initialize App Config with environment variable fallback support.
2. Configuration Constants
Define all configuration keys in a central constants file (constants/configuration.ts).
3. Lookup Abstraction
Use an app.lookup() abstraction layer that:
-
Resolves values from Azure App Configuration
-
Falls back to environment variables during local development
-
Handles type conversions (string → boolean, string → number)
-
Supports package.json property resolution
4. Dependency Injection
Configuration values are looked up asynchronously and injected where needed.
Component Architecture
Implementation Steps
Phase 1: Setup Dependencies
Task 1.1: Install Required NPM Packages
Task 1.2: Update package.json
Add the following to package.json:
Phase 2: Create Core Infrastructure
Task 2.1: Create Logger Type Definition
File: src/internal/Logger.d.ts
Task 2.2: Create App Container
File: src/internal/app.ts
Task 2.3: Create Configuration Constants
File: src/constants/configuration.ts
Phase 3: Create Framework Adapters
Task 3.1: Missing Logger Placeholder
File: src/internal/frameworkAdapter/missingLogger.ts
Task 3.2: Missing Lookup Placeholder
File: src/internal/frameworkAdapter/missingLookup.ts
Task 3.3: Register Logger Function
File: src/internal/frameworkAdapter/registerLogger.ts
Task 3.4: Provide App Config
File: src/internal/frameworkAdapter/provideAppConfig.ts
Task 3.5: Create Lookup Function with Type Conversion
File: src/internal/frameworkAdapter/createLookup.ts
Task 3.6: Prepare Lookup Function
File: src/internal/frameworkAdapter/prepareLookup.ts
Phase 4: Update Application Entry Point
Task 4.1: Modify index.ts
File: src/index.ts
Phase 5: Migrate Service Code
Task 5.1: Update Configuration Usage Pattern
Before (using direct environment variables):
After (using app.lookup):
Task 5.2: Update Database Configuration
Task 5.3: Update OpenAI Service
Task 5.4: Update Form Recognizer Service
Phase 6: Azure Infrastructure Setup
Task 6.1: Create App Configuration Keys in Azure Portal
For each environment (dev, qa, uat, prd), create the following keys in Azure App Configuration:
Task 6.2: Migrate Secrets to Key Vault References
Store sensitive values in Azure Key Vault and reference them in App Config:
Create Key Vault secrets:
Update App Config with Key Vault references:
Task 6.3: Configure Managed Identity Access
Grant the DocumentAI service’s Managed Identity access to:
-
App Configuration:
App Configuration Data Readerrole -
Key Vault:
Key Vault Secrets Userrole
Phase 7: Update Deployment Configuration
Task 7.1: Update Kubernetes Deployment YAML
File: k8s/deployment.yaml
Task 7.2: Create Local Development .env File
File: .env.example
Testing Strategy
Unit Tests
Test 1: Configuration Lookup
File: test/unit/internal/frameworkAdapter/createLookup.test.ts
Test 2: Missing Configuration Handling
File: test/unit/internal/frameworkAdapter/missingLookup.test.ts
Test 3: Environment Variable Fallback
File: test/unit/config-fallback.test.ts
Integration Tests
Test 4: App Configuration Connection
File: test/integration/appconfig-connection.test.ts
Test 5: Key Vault Secret Resolution
File: test/integration/keyvault-secrets.test.ts
Contract Tests
Test 6: Configuration Schema Validation
File: test/contract/config-schema.test.ts
Acceptance Criteria
Functional Requirements
-
All configuration values are retrieved from Azure App Configuration in deployed environments
-
Local development works with
.envfile fallback -
Secrets (API keys) are stored in Key Vault and referenced through App Config
-
All configuration keys use the
DOCAI_prefix -
Boolean values (
DB_SQL_MANAGED_IDENTITY) are correctly typed as booleans -
Numeric values (
APPLICATION_PORT) are correctly typed as numbers -
String values are retrieved without type conversion
Technical Requirements
-
app.lookup()abstraction is used throughout the codebase -
No direct
process.envaccess for configuration values -
Configuration constants defined in
src/constants/configuration.ts -
Framework initialization happens before any configuration lookup
-
Proper error handling for missing configuration
-
TypeScript type safety maintained for all configuration values
Testing Requirements
-
Unit tests cover configuration lookup logic
-
Unit tests cover type conversion (string → boolean, string → number)
-
Integration tests validate Azure App Config connectivity
-
Integration tests validate Key Vault secret resolution
-
Contract tests validate configuration schema
-
All tests pass in CI/CD pipeline
-
Test coverage > 80% for configuration modules
Security Requirements
-
Sensitive values removed from
.envfiles in version control -
.envcontains only non-sensitive local development values -
Secrets stored in Azure Key Vault
-
Managed Identity used for authentication (no connection strings in code)
-
App Configuration connection string stored as Kubernetes secret
-
No secrets logged or exposed in error messages
Documentation Requirements
-
README updated with App Config setup instructions
-
.env.examplefile created with all required keys -
Migration guide documented for team members
-
Architecture diagram updated
-
Runbook created for App Config management
Deployment Requirements
-
App Configuration provisioned in all environments (dev, qa, uat, prd)
-
All configuration keys created with appropriate labels/environments
-
Managed Identity configured with correct RBAC roles
-
Helm charts updated with App Config connection string
-
Deployment pipeline validates configuration availability
-
Rollback plan documented
Non-Functional Requirements
Performance
-
Configuration lookup should add < 100ms to application startup time
-
Cached configuration values should not require repeated App Config calls
-
Application should start successfully within 30 seconds
Reliability
-
Application should fail fast with clear error messages if configuration is missing
-
Fallback to environment variables should work seamlessly in local development
-
Service should handle App Config temporary unavailability gracefully
Maintainability
-
Configuration keys should follow consistent naming convention (
DOCAI_*) -
Type conversions should be centralized in
createLookup.ts -
Adding new configuration should require updates in only 2 files (
ENV_KEYSand App Config)
Risks and Mitigation
Risk 1: App Configuration Unavailability
Impact: High - Service cannot start
Mitigation:
-
Implement caching at application level
-
Use environment variable fallback in deployment YAML as backup
-
Monitor App Config availability with alerts
Risk 2: Key Vault Access Issues
Impact: High - Cannot retrieve secrets
Mitigation:
-
Validate Managed Identity permissions during deployment
-
Implement retry logic with exponential backoff
-
Monitor Key Vault access patterns
Risk 3: Configuration Drift Between Environments
Impact: Medium - Inconsistent behavior across environments
Mitigation:
-
Use Infrastructure as Code (Terraform/Bicep) to manage App Config
-
Implement configuration validation tests in CI/CD
-
Document configuration schema
Risk 4: Breaking Changes During Migration
Impact: High - Service downtime
Mitigation:
-
Implement feature flag to toggle between old and new config methods
-
Deploy to non-production environments first
-
Maintain backward compatibility during transition period
Rollout Plan
Phase 1: Development Environment (Week 1)
-
Implement configuration infrastructure code
-
Deploy to development environment
-
Run integration tests
-
Validate logging and monitoring
Phase 2: QA Environment (Week 2)
-
Deploy to QA environment
-
Run full test suite
-
Performance testing
-
Security validation
Phase 3: UAT Environment (Week 3)
-
Deploy to UAT environment
-
Stakeholder validation
-
Load testing
-
Documentation review
Phase 4: Production Environment (Week 4)
-
Deploy during maintenance window
-
Monitor application health
-
Validate all services
-
Conduct post-deployment review
Success Metrics
Technical Metrics
-
Zero configuration-related incidents post-migration
-
Application startup time < 30 seconds
-
Configuration lookup latency < 50ms (p95)
-
100% of secrets stored in Key Vault
-
Zero secrets in version control
Operational Metrics
-
Configuration change time reduced from 30 minutes to 5 minutes
-
Zero environment-specific configuration files in repository
-
100% configuration audit trail in App Config
-
Mean time to update configuration < 2 minutes
Team Metrics
-
All team members trained on App Config management
-
Zero deployment failures due to missing configuration
-
Configuration documentation maintained and up-to-date
Related Documentation
-
fluxx-file-intake Configuration Pattern
Story Points: 13
Complexity: High
Effort: 2-3 weeks
Dependencies: Azure App Configuration, Azure Key Vault, Managed Identity setup
Definition of Done
-
All acceptance criteria met
-
Code reviewed and approved
-
All tests passing (unit, integration, contract)
-
Deployed to all environments
-
Documentation updated
-
Team training completed
-
Monitoring and alerts configured
-
Post-deployment validation successful
-
Stakeholder sign-off received