Measuring Cold Starts Without Misleading Yourself

Measuring cold starts without misleading yourself

“Cold start” often appears as one number with no definition. That number may include DNS, a TLS handshake, a queue, artifact retrieval, runtime initialisation, application setup and ordinary request work. Change any one of those and the comparison changes.

A useful experiment begins by defining the lifecycle state, not by starting a stopwatch.

Name the states

Use terms that can be reproduced:

• cold: no eligible execution instance or cached application state exists under the stated platform policy;
• warm: an eligible instance has already completed at least one request with the same artifact and configuration;
• restored: an instance or compiled artifact was recovered from a platform cache or snapshot; and
• unknown: the experiment cannot prove the state.

Do not label an idle request cold merely because several minutes passed. The platform’s eviction and caching policy is part of the evidence.

Measure the pieces you control

From an external client, record DNS and connection reuse, region, request start, response headers, status and total duration. Inside the runtime, use safe timestamps for admission, artifact readiness, initialisation, handler start and handler end where the platform exposes them.

The external result represents user experience. Internal spans help explain it. Neither should silently replace the other.

Use distributions and retain failures

Run enough samples to report a distribution such as p50, p95 and p99, alongside the sample count and failure rate. An average can hide a small but important slow tail. Dropping timeouts from the dataset makes an unhealthy system appear faster.

Keep concurrency controlled. A cold-start study at one request at a time answers a different question from a burst that also tests queueing and capacity. Run those as separate experiments.

Hold the artifact and workload steady

Record the artifact digest, byte size, language/runtime version, dependency set, region, memory class and request payload class. A larger artifact is not automatically slower, and a fast no-op handler does not predict a database-backed workload. It simply isolates one part of the system.

Comparing languages across different machines, regions or lifecycle definitions produces a leaderboard, not evidence.

A small experiment

1. Deploy one identified artifact.
2. Define how a cold state will be established and verified.
3. Send one bounded request from a named client region.
4. retain every success and failure;
5. repeat for the defined sample count;
6. run a separate warm sequence with connection policy recorded; and
7. report distributions with the complete experiment label.

The analyser below accepts one duration per line and calculates simple percentiles in your browser. It deliberately does not infer whether the samples were truly cold.

Optimise the right layer

After measurement, improvements may include smaller artifacts, fewer eager imports, precompiled modules, connection reuse, regional placement, cache policy or minimum warm capacity. Each has a different cost and reliability effect. Optimising handler code will not fix DNS or queueing.

BlinkHost keeps runtime lifecycle labels and evidence scope visible in public compatibility material. That restraint matters more than a flattering number: buyers and engineers need measurements they can reproduce.

Primary references

• OpenTelemetry traces
• Prometheus: histograms and summaries
• IETF HTTP Semantics
• Coordinated omission explained by Gil Tene

Reviewed 2 September 2026.