GCP Cloud Run Cold Starts at the 99th Percentile: What Actually Fixed Them

We run a mixed Node and Python API fleet on Cloud Run for a client with spiky traffic — think 20 req/s baseline, 400 req/s during campaign pushes. For months, p50 was fine (~80ms) but p99 would spike into multi-second territory whenever autoscaling kicked in. This is the ranked list of what actually moved the needle, and what turned out to be theatre.

The shape of the problem

Cloud Run's cold start isn't one thing. It's a stack: container pull, container start, language runtime boot, framework init, your app's own warmup (DB pools, secret fetches, SDK clients), then the first request handler. Any of these can dominate, and the mix changes as you optimize.

We instrumented startup with a tiny helper that logged timestamps at each phase and shipped them as structured logs. Nothing fancy:

const t0 = process.hrtime.bigint();
const mark = (label) => {
  const ms = Number(process.hrtime.bigint() - t0) / 1e6;
  console.log(JSON.stringify({ severity: 'INFO', startup_phase: label, elapsed_ms: ms }));
};

mark('process_start');
await import('./config.js');
mark('config_loaded');
await initSecrets();
mark('secrets_loaded');
await initDbPool();
mark('db_pool_ready');
const app = await buildApp();
mark('app_built');
app.listen(process.env.PORT, () => mark('listening'));

Once you have this, Log Explorer with a startup_phase filter gives you a flame-graph-shaped mental model within an hour. Do this before you touch anything else. We skipped it the first time and burned a week guessing.

What our baseline looked like

For the Node service, a cold instance took roughly 2.8–3.4 seconds to serve its first request. Breakdown, approximately:

• Container pull + start: 400–600ms
• Node runtime + import graph: 900–1200ms
• Secret Manager fetches (3 secrets, serial): 500–700ms
• Firestore + Pub/Sub SDK first-call init: 400–600ms
• First handler execution: 100–200ms

Python (FastAPI + a few ML-adjacent libs) was worse: 4–6 seconds, dominated by import time and a lazy model loader.

What actually worked, ranked

1. Startup CPU boost (the cheapest real win)

Cloud Run has a flag called startup CPU boost that temporarily allocates extra CPU during container startup. We'd ignored it as marketing until we A/B tested it. On the Node service, enabling it cut cold start from ~3.1s to ~1.9s. On Python it went from ~5.2s to ~3.3s.

It costs nothing extra beyond the normal CPU allocation model, and it's a one-line change:

gcloud run services update my-api \
  --cpu-boost \
  --region=europe-west1

This is the first thing we'd try on any CPU-bound startup. If your bottleneck is network (secrets, image pull from a far-away region), it won't help as much.

2. Collapsing Secret Manager calls

We were pulling three secrets serially using the SDK, each a round trip. Two fixes stacked here:

• Parallelize: Promise.all the fetches. Saved ~250ms on average.
• Mount as volumes: Cloud Run can mount Secret Manager secrets as files. The platform caches them with the instance, so first-request latency drops to a filesystem read.

Mounting won. We moved all three secrets to volume mounts and killed ~500ms off cold start, plus removed a failure mode where Secret Manager rate-limited us during scale events.

gcloud run services update my-api \
  --update-secrets=/secrets/db-url=db-url:latest,/secrets/api-key=api-key:latest

3. Trimming the container image

Our Node image was 480MB (Debian slim base, full node_modules including dev deps that snuck in). A multi-stage build with node:20-alpine and a production-only install dropped it to 110MB. Container pull time fell from ~500ms to ~180ms on cold start.

FROM node:20-alpine AS build
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build && npm prune --production

FROM node:20-alpine
WORKDIR /app
COPY --from=build /app/node_modules ./node_modules
COPY --from=build /app/dist ./dist
COPY --from=build /app/package.json ./
USER node
CMD ["node", "dist/server.js"]

Alpine has its own tradeoffs (musl, native module rebuilds), so test thoroughly. For the Python service we stuck with python:3.12-slim and used uv to install, which also meaningfully cut layer size and install time at build.

4. Lazy-loading heavy SDKs and ML code paths

The Python service imported a model-serving library at module load. It was only used by two endpoints out of fourteen. We moved it inside those handlers:

# before: top-level import added ~1.2s to every cold start
# from heavy_lib import Model

def get_model():
    from heavy_lib import Model  # imported on first use only
    global _model
    if _model is None:
        _model = Model.load("/models/v3")
    return _model

This is unglamorous but it's often the single biggest Python win. Audit your import graph. python -X importtime locally will show you the offenders.

For Node, the equivalent is dynamic import() for route-specific heavy modules. The Google Cloud SDKs in particular are chunky; only load the clients you need, and avoid importing the umbrella package.

5. Min instances — useful, but not a silver bullet

Setting --min-instances=2 eliminates cold starts for your baseline traffic. We do this on auth and checkout services. But:

• You pay for idle CPU/memory 24/7.
• It does nothing for scale-out cold starts during a traffic spike — those still land on cold instances.
• It can mask real startup regressions. A dev adds a 2s import, nobody notices, until the service scales.

We use min instances conservatively — only for latency-critical paths — and we alert on startup time regardless, so we catch bloat.

What didn't work, or barely did

• Switching to Cloud Run's second-generation execution environment: slightly faster for some workloads, slightly slower for others. In our case, roughly a wash. Worth benchmarking yours.
• Bumping CPU from 1 to 2 vCPU without startup boost: helped startup by maybe 10%, cost us 40% more. Startup boost is a better lever.
• Pre-warming via a scheduled Cloud Scheduler job hitting /healthz: this keeps one instance warm. It does not help p99 during scale events, and it conflicts with min instances if you're not careful.
• Switching from Firestore client to raw REST: saved ~100ms of client init but added code complexity and lost retry/backoff logic we actually wanted. Reverted.

Measuring the right thing

If you only look at service-wide p99, you'll be confused. Cold starts are rare by definition — maybe 0.5–2% of requests depending on traffic shape. They can be hidden by averaging.

Two things we do now:

Separate the cold-start p99 from the warm p99

Our middleware tags every request with whether the instance has served a request before. In OpenTelemetry terms, we add a span attribute instance.cold_request=true for the first request on an instance. Then we have two SLO burn rate alerts: warm p99 (<200ms) and cold p99 (<2s). Mixing them hides problems.

Track startup duration as its own metric

Startup time (listening timestamp minus process_start) is a leading indicator. If a PR adds 400ms to startup, we want to know in CI, not after a traffic spike. We publish startup duration as a custom metric and alert on week-over-week regression.

Rough cost framing

For the Node service, the full set of changes (CPU boost, secret volumes, smaller image, no min instances) cut cold-start p99 from ~3.2s to ~1.1s. Monthly cost went up by roughly 8% because CPU boost adds compute during startup. For the Python service, lazy imports plus the same changes took cold-start p99 from ~5.5s to ~1.8s, with no meaningful cost delta.

These are our numbers on our workloads. Yours will differ. The point is that most of the wins are free or near-free, and min instances — the obvious answer — isn't necessarily the best first move.

Where we'd start

If you're staring at a Cloud Run service with a bad cold-start p99 tomorrow morning:

1. Instrument startup phases. Ship it to logs. Look at the breakdown.
2. Turn on startup CPU boost. Measure.
3. Move secrets to volume mounts.
4. Audit your import graph. Lazy-load anything used by fewer than half your routes.
5. Slim the container image with a multi-stage build.
6. Then consider min instances for the handful of endpoints that genuinely need sub-second cold recovery.

If you want help wiring this into a CI gate or an SLO dashboard, our DevOps and cloud team has done this on both Cloud Run and comparable AWS Lambda setups. The patterns transfer; the specifics don't.