Skip to content

Infrastructure cache expires during deployment, causing unnecessary rebuild #485

@seanspeaks

Description

@seanspeaks

Description

The infrastructure definition cache has a 60-second TTL that expires during typical deployments (which take 5+ minutes), causing the infrastructure to be rebuilt unnecessarily during post-deployment health checks.

Current Behavior

// packages/devtools/infrastructure/create-frigg-infrastructure.js (lines 42-52)
if (fs.existsSync(cachePath)) {
    const cached = JSON.parse(fs.readFileSync(cachePath, 'utf-8'));
    // Verify cache is still valid (less than 60 seconds old)
    if (Date.now() - cached.timestamp < 60000) {
        console.log('✓ Using filesystem-cached infrastructure definition');
        return cached.definition;
    } else {
        console.log('⚠️ Cache expired (> 60s), recomposing...');
        fs.removeSync(cachePath);
    }
}

Cache TTL: 60 seconds (60000ms)

The Problem

Typical deployment timeline:

  1. T=0s: Infrastructure composed, cache written (timestamp: 0)
  2. T=0-333s: Serverless deployment runs (5 minutes 33 seconds is common)
  3. T=333s: Deployment completes ✅
  4. T=334s: Post-deploy health check starts
  5. T=334s: Cache check: 334000ms - 0ms > 60000msEXPIRED
  6. T=334s: Infrastructure rebuilt from scratch:
    • Re-runs AWS resource discovery
    • Re-runs all domain builders (VPC, KMS, Integration, etc.)
    • Re-composes entire serverless definition
  7. T=334-338s: Unnecessary rebuild completes

Impact

  • Performance: Adds 10-30 seconds to every deployment
  • AWS API Calls: Redundant discovery calls (CloudFormation DescribeStacks, EC2 DescribeVpcs, KMS ListKeys, etc.)
  • Cost: Unnecessary API usage
  • Confusion: Logs show "Composing serverless definition" twice, making deployments appear to run twice

Real-World Example

From a production deployment log:

🏗️  Composing serverless definition with domain builders...
[... builds infrastructure ...]
✔ Service deployed to stack create-frigg-app-production (333s)
✓ Deployment completed successfully!

⚠️ Cache expired (> 60s), recomposing...
🏗️  Composing serverless definition with domain builders...
[... rebuilds exact same infrastructure ...]
✅ Serverless definition composed successfully

Proposed Solutions

Option 1: Increase Cache TTL (Simple)

Change cache TTL to 10-15 minutes to cover most deployments:

// Increase from 60000ms (60s) to 900000ms (15min)
if (Date.now() - cached.timestamp < 900000) {
    console.log('✓ Using filesystem-cached infrastructure definition');
    return cached.definition;
}

Pros: Simple one-line fix
Cons: Fixed duration might still be too short for very complex deployments

Option 2: Configurable Cache TTL (Flexible)

Make cache duration configurable via environment variable:

const CACHE_TTL = parseInt(process.env.FRIGG_CACHE_TTL || '900000', 10); // Default 15min

if (Date.now() - cached.timestamp < CACHE_TTL) {
    console.log('✓ Using filesystem-cached infrastructure definition');
    return cached.definition;
}

Pros: Flexible, users can set based on their deployment time
Cons: Requires documentation

Option 3: Deployment-Aware Caching (Robust)

Don't expire cache during active deployment:

// Track deployment state in cache
const cached = {
    timestamp: Date.now(),
    deploymentActive: true,
    definition
};

// In cache check:
if (cached.deploymentActive || Date.now() - cached.timestamp < 60000) {
    return cached.definition;
}

Pros: Most robust, works for any deployment duration
Cons: More complex, requires state management

Option 4: Pass Definition Directly (Elegant)

Avoid cache check entirely by passing infrastructure definition from deploy to health check:

async function deployCommand(options) {
    const appDefinition = loadAppDefinition();
    const definition = await composeServerlessDefinition(appDefinition);
    
    await executeServerlessDeployment(definition, options);
    
    if (!options.skipDoctor) {
        await runPostDeploymentHealthCheck(stackName, {
            ...options,
            infraDefinition: definition  // ← Pass directly
        });
    }
}

Pros: Eliminates cache issues, most efficient
Cons: Requires API changes to health check

Recommendation

Short-term: Option 1 or 2 (increase/configure TTL) - minimal change, immediate fix
Long-term: Option 4 (pass definition) - cleanest architecture

Files Affected

  • packages/devtools/infrastructure/create-frigg-infrastructure.js (line 46)
  • packages/devtools/frigg-cli/deploy-command/index.js (calls infrastructure composition)

Environment

  • Frigg version: 2.0.0--canary.482.bda352c.0
  • Node version: 20.x
  • Deployment: GitHub Actions / AWS Lambda
  • Typical deployment time: 5-10 minutes (first deploy with Prisma layer build)

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingperformanceImprove performance of an existing feature

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions