Skip to navigation
Research and SpikesMemory and Caching Strategy

Memory and Caching Analysis

Purpose

Evaluate platform-wide memory and caching options that support performance, scalability, and operational stability in a horizontally scaled AKS environment, and identify the recommended approach for the Fluxx Platform

Architectural Requirements

Any selected strategy must:

  • Support horizontal pod scaling
  • Preserve stateless service design
  • Prevent unbounded memory growth
  • Avoid cross-tenant data leakage
  • Degrade safely under cache failure
  • Minimize operational overhead within the CLA tenant

Options Considered

  1. In-Pod Memory Caching Only
    Each service maintains its own local in-memory cache with no shared distributed cache layer

  2. Redis Hosted Inside AKS Cluster
    Deploy and manage a Redis cluster within the AKS cluster using Terraform/Helm

  3. Azure Cache for Redis (Managed Service)
    Use Azure Cache for Redis as an external managed distributed cache integrated via the CLA Framework caching package

Option 1 - In-Pod Memory Caching Only

Description

Each service maintains its own in-memory cache within the pod process. No shared distributed cache is used.

Advantages

  • Simple to implement
  • No additional infrastructure dependencies
  • Lowest latency access

Limitations

  • Cache is not shared across pods
  • Horizontal scaling introduces cache inconsistency
  • Pod restarts cause cache loss
  • Cannot support distributed idempotency or rate limiting
  • Risk of unbounded memory growth if not tightly controlled

Assessment

Does not meet multi-pod consistency requirements
Not suitable for idempotency, deduplication, or shared configuration
High risk of correctness issues in event-driven workflows

Recommendation: Not recommended

Option 2 - Redis Hosted Inside AKS Cluster

Description

Deploy Redis using Helm within the AKS cluster and manage it as part of platform workloads

Advantages

  • Shared cache across pods
  • Full control over configuration
  • Avoids external managed dependencies

Limitations

  • Requires cluster-level operational management
  • Must configure persistence, failover, backups, and scaling
  • Adds operational complexity to the Fluxx team
  • Competes for compute resources with application workloads

Assessment

Technically viable but increases operational burden
Introduces unnecessary infrastructure ownership

Recommendation: Not recommended

Option 3 - Azure Cache for Redis (Managed Service)

Description

Use Azure Cache for Redis as a managed distributed cache service, integrated via the CLA Framework caching package

Advantages

  • Shared across all pods and services
  • Fully managed by Azure
  • Built-in high availability and scaling
  • Supports Managed Identity and private networking
  • Operational separation from AKS workloads
  • Enterprise-ready security posture
  • Native Azure monitoring integration

Limitations

  • External dependency
  • Additional Azure cost

Assessment

Meets all horizontal scaling and statelessness requirements
Reduces operational overhead
Aligns with enterprise SaaS architecture standards
Supports idempotency and cross-service coordination safely

Recommendation: Recommended strategy for the Fluxx Platform

Recommended Strategy for the Fluxx Platform

Adopt Azure Cache for Redis as the shared distributed caching layer.

Guidelines:

  • Use cache-aside pattern
  • Require TTL for all cache entries
  • Enforce tenant and application scoped keys
  • Version key prefixes
  • Never treat cache as source of truth
  • Limit in-pod memory caching to micro-TTL and strictly bounded usage

Supporting Controls

Memory Management

Memory management controls will be enforced through Kubernetes configuration and monitored using Azure Monitor, Application Insights, and Prometheus/Grafana

  • Define container memory requests and limits per service (Kubernetes resource configuration)
  • Validate resource limit compliance via Azure Monitor or Prometheus metrics
  • Monitor heap usage, allocation rate, and GC pressure using Application Insights runtime metrics
  • Track container memory usage and working set via Azure Monitor for Containers or Prometheus
  • Alert on:
    • Container memory utilization exceeding defined thresholds (for example, 80 percent of limit)
    • OOMKilled events
    • Pod restarts caused by memory pressure
    • Node-level memory pressure conditions
  • Detect abnormal heap growth trends to identify potential unbounded in-memory structures

These controls ensure early detection of memory leaks, unsafe in-memory caching patterns, and resource exhaustion risks

Statelessness Enforcement

  • Services must operate correctly with empty cache
  • Pod restart must not impact correctness
  • Redis outage must degrade safely by falling back to source of truth

Conclusion

Azure Cache for Redis is the recommended caching strategy for Fluxx Platform due to:

  • Horizontal scalability support
  • Stateless service alignment
  • Multi-tenant isolation capability
  • Reduced operational complexity
  • Enterprise-grade security compliance

This strategy provides a stable foundation for performance optimization without compromising correctness or scalability