Backend Development

Production Checklist

Review correctness, security, operability, deployment, and recovery before serving real traffic.

What you will learn

By the end of this lesson, you will be able to:

  • conduct a structured production-readiness review;
  • identify missing limits, failure handling, and recovery plans;
  • distinguish build checks from runtime safeguards;
  • prepare safe schema and application rollouts;
  • write a release and rollback plan that another engineer can follow.

Mental model: production readiness is evidence

“It works locally” proves one path in one environment. Production readiness requires evidence that the service behaves acceptably during expected load, invalid input, dependency failure, cancellation, deployment, and recovery.

Use this checklist as a conversation and test plan, not as a badge produced by ticking boxes blindly.

Correctness and tests

  • Critical business rules have unit tests.
  • Repositories are tested against the actual database engine used in production.
  • HTTP contract tests cover success, malformed input, validation, authentication, authorization, missing resources, conflicts, and internal failures.
  • Concurrency-sensitive code is exercised with go test -race on a supported platform.
  • Fuzz targets cover parsers or other input-heavy boundaries where useful.
  • Tests are deterministic and do not depend on public network services.
  • A failed production case becomes a regression test when practical.

Build and dependency hygiene

  • go.mod and go.sum are reviewed and committed.
  • The supported Go toolchain version is documented and used in CI.
  • Formatting, tests, static analysis, and builds run in CI.
  • govulncheck scans reachable dependencies and the result is reviewed.
  • Release artifacts are reproducible enough to trace source revision, build inputs, and configuration expectations.
  • Third-party licences and operational dependencies are understood.

HTTP and resource limits

  • Request headers and bodies have appropriate limits.
  • Server read-header, idle, and other timeouts are selected for the deployment and streaming requirements.
  • Outbound HTTP clients have deadlines and reuse configured transports/clients.
  • Retries are bounded, cancellation-aware, jittered where needed, and limited to safe operations.
  • Queues and worker pools are bounded.
  • Rate limits protect expensive or abuse-prone operations without becoming the only security control.
  • File descriptors, memory, CPU, goroutines, and connection pools have monitored capacity.

There is no universal timeout value. Derive limits from service objectives, dependency behaviour, network path, and request type.

Security

  • All external traffic uses correctly configured TLS across untrusted links.
  • Authentication validates the complete policy; authorization covers resource and tenant boundaries.
  • SQL values are parameterized and dynamic identifiers are allow-listed.
  • Secrets are stored, delivered, rotated, and audited through an approved mechanism.
  • Logs and traces exclude passwords, tokens, cookies, private keys, and unnecessary personal data.
  • Cookie, CSRF, CORS, proxy-header, upload, and redirect policies match the real clients and deployment.
  • Error responses do not expose internal details.
  • Security headers are chosen and tested for the application's browser behaviour.
  • Dependencies and toolchains receive supported security updates through a reviewed process.

Database and data lifecycle

  • Schema migrations are versioned, reviewed, and tested on production-like data volume.
  • Application and schema changes use backward-compatible rollout stages when zero-downtime deployment is required.
  • Transactions protect multi-step invariants.
  • Unique, foreign-key, and check constraints enforce critical integrity rules.
  • Pool limits account for all service replicas.
  • Backups are encrypted where required, monitored, and restored in drills.
  • Retention and deletion requirements are implemented and verified.
  • High-risk data changes have a rollback or forward-fix plan.

Observability and operations

  • Logs use stable structured events and useful correlation identifiers.
  • Metrics cover request outcomes, latency, dependencies, queues, and saturation with bounded labels.
  • Traces preserve context across important boundaries when tracing is used.
  • Dashboards show user-impacting behaviour, not only machine utilisation.
  • Alerts connect to an actionable runbook and avoid predictable noise.
  • Liveness and readiness have tested, distinct meanings.
  • Diagnostic and profiling endpoints are protected.
  • Operators can identify the running version and relevant safe configuration without exposing secrets.

Deployment and shutdown

  • The process runs as a non-privileged identity with the smallest required permissions.
  • The artifact contains only required files and trusted certificates.
  • Readiness prevents traffic before startup completes.
  • Deployment waits for readiness and honours a tested termination grace period.
  • Graceful shutdown drains requests and coordinates background work within a fixed deadline.
  • Rollout strategy limits blast radius through staged, canary, or similar deployment when appropriate.
  • Rollback steps are tested and account for database compatibility.
  • A second deployment while the first is failing cannot silently make recovery harder.

Performance and capacity

  • Expected traffic, payload size, concurrency, and growth assumptions are written down.
  • Load tests represent important production paths and failure modes.
  • Profiles and benchmarks identify actual bottlenecks before optimisation.
  • Capacity includes headroom for bursts and dependency slowdown.
  • Autoscaling signals measure useful pressure and account for startup and shutdown behaviour.
  • Expensive endpoints have bounded work and pagination.

Incident readiness

  • Ownership and escalation paths are known.
  • Runbooks explain diagnosis, mitigation, rollback, and recovery.
  • Feature flags or operational controls fail safely and are auditable.
  • Data repair procedures are reviewed and avoid untracked manual changes.
  • Post-incident learning results in code, test, documentation, or operational improvements.

Release review template

Before a release, record:

Change:
Expected user impact:
Source revision and artifact:
Configuration changes:
Schema changes:
Pre-deployment checks:
Success signals:
Failure signals:
Rollback trigger:
Rollback steps:
Data compatibility notes:
Owner:

Common mistakes

  • Treating the checklist as universal configuration: timeouts, pools, and limits require workload evidence.
  • Testing backup creation but not restoration: an unusable backup provides false confidence.
  • Making schema changes that old code cannot tolerate: rollback becomes impossible after migration.
  • Alerting on every exception or CPU spike: operators learn to ignore alerts.
  • Depending on graceful shutdown for correctness: machines and processes can disappear without warning.
  • Keeping the runbook only in one person's memory: recovery slows when that person is unavailable.

Practice

Apply this checklist to the lesson API. Produce a one-page readiness report with evidence, unresolved risks, owners, and a staged release/rollback plan. Do not mark an item complete without a test, configuration reference, dashboard, runbook, or other inspectable evidence.

Summary

  • Production readiness combines correctness, security, operability, deployment, and recovery.
  • Every important checklist result should point to evidence.
  • Limits and timeouts come from workload and dependency behaviour, not copied constants.
  • Backward-compatible migrations and tested rollback protect deployments.
  • Crashes still happen, so durable recovery and rehearsed incident response remain essential.

Official reference: Security best practices for Go developers.

Ready to wrap up?

Mark this complete when you can explain the main idea and finish the practice task.

Copyright © 2026