Most teams know they need CI/CD. Most teams also have a CI/CD pipeline that doesn’t actually gate anything — it runs tests, produces a green checkmark, and ships whatever was pushed. The gap between “we have a pipeline” and “the pipeline prevents bad releases” is where the engineering lives. This post covers the patterns that survived a real engagement: a Series B fintech that went from four-hour manual deployments to fifteen-minute automated ones, with zero failed deployments in the first thirty days. No theory, no vendor pitch — what was built, what it gates, and where it breaks when you scale.
The runner setup
Gitea Actions is a CI/CD runner that integrates directly with Gitea repositories. The architecture is straightforward: a runner process registers with the Gitea instance, picks up workflow jobs from a queue, and executes them in an isolated container. No per-seat licensing, no per-build pricing, no external service to depend on.
The runner configuration that works in production is not the default. Three changes matter:
Dedicated runner hosts, not shared. Running the CI runner on the same machine as the Gitea instance works for a handful of repositories and breaks at scale. A test suite that spikes CPU to 100% on the runner starves the Gitea web process, and developers start seeing timeouts on push. The production setup is a separate runner host — or a small fleet of them — with the Gitea instance on its own machine.
Container isolation per job. Each workflow job runs in a fresh container. This is not just security hygiene — it is correctness. A test suite that passes in a clean container and fails on the runner host means a dependency leaked from a previous job, and that flaky failure will cost hours to reproduce. Container-per-job eliminates the entire class of “works on the runner, fails in CI” bugs.
Secrets in the runner, not in the repo. Gitea supports repository and organization secrets, encrypted at rest and injected as environment variables at job start. The production pattern is to keep secrets at the organization level and scope them down to specific repositories — not to store them in the repository, not to pass them through workflow files in plaintext, and not to bake them into container images.
Quality gates that mean something
A pipeline without a quality gate is a deployment accelerator, not a deployment safeguard. The gate is what turns “the tests passed” into “the release is safe.” The gate that worked in the engagement had three layers:
Test coverage above baseline. The pipeline measures coverage on every push and rejects builds where coverage drops below the project baseline. The baseline is not aspirational — it is the current measured value, ratcheted upward when the team agrees. A build that adds code without tests does not deploy.
SonarQube analysis as a hard gate. Code quality, security hotspots, and duplication are checked on every push. A new critical security hotspot blocks the build — not a warning, not a suggestion, a block. The team can override in an emergency, but the override is logged and reviewed.
Vulnerability scanning on dependencies. govulncheck for Go, npm audit for Node, SBOM generation for everything. A new high-severity finding blocks the build unless it is explicitly triaged and suppressed with a VEX statement. The suppression is documented, time-bounded, and reviewed.
The gate is only as good as its blocking behavior. A gate that warns but does not block is decoration. The team will learn to ignore the warnings within a week, and the pipeline becomes a green-checkmark generator again.
Blue-green deployment and rollback
The deployment strategy that eliminated failed releases was blue-green: two identical production environments, with traffic switching between them. New code ships to the inactive environment, health checks run for five minutes, and traffic switches over only if everything is within thresholds. The old environment stays warm — if the health checks fail after the switch, traffic reverts automatically. No human has to wake up and decide whether to roll back.
The mechanics:
- Deploy to the inactive slot. The pipeline builds the container image, pushes it to the registry, and deploys it to the environment that is not currently serving traffic.
- Health checks for five minutes. The pipeline polls the inactive environment’s health endpoint, checks error rates, and verifies that key endpoints respond within latency thresholds. If anything drifts, the deploy is flagged and traffic does not switch.
- Traffic switch. If health checks pass, a load balancer rule flips traffic to the new environment. The switch is atomic from the user’s perspective.
- Automatic rollback. If health checks fail after the switch, traffic reverts to the previous environment. The failed deployment stays in the inactive slot for inspection — it is not destroyed, so the team can debug it.
The five-minute health-check window is the part that most teams get wrong. Too short, and a slow memory leak or a connection-pool exhaustion slips through. Too long, and the deployment cycle becomes sluggish. Five minutes was the number that worked for this workload — measured against the failure modes the team had actually seen, not guessed.
The failure modes at scale
The pipeline that works for five repositories develops new failure modes at fifty. Three are worth naming:
Runner contention. When multiple teams push simultaneously and the runner fleet is too small, jobs queue. A pipeline that took twelve minutes starts taking forty, and developers start bypassing it — pushing directly to production “just this once.” The fix is monitoring the queue depth and scaling the runner fleet before it becomes a bottleneck, not after.
Test suite drift. As the test suite grows, it gets slower. A suite that ran in three minutes at launch takes fifteen at six months, and the team starts skipping the long integration tests locally because “CI will catch it.” The fix is splitting the suite — fast unit tests on every push, slower integration tests on pull requests, full end-to-end on release branches. The pipeline should run the right tests at the right stage, not all tests on every push.
Secret sprawl. As the number of environments grows (dev, staging, prod, plus per-customer environments), secrets multiply. Organization-level secrets help, but the real discipline is auditing which repositories can access which secrets, and rotating on a schedule. A secret that has not been rotated in a year is a liability.
What the team owns afterward
The engagement ended with the customer engineering team running the pipeline day-to-day — adding repositories, tuning test suites, extending the blue-green logic. The runbooks and decision records in their knowledge base let them self-serve routine changes. The part that stayed with us was the architecture and the hard calls around rollback policy and quality-gate thresholds — the decisions that are cheap to get wrong and expensive to fix after a bad release.
That division is the point. A pipeline your team cannot operate is a pipeline your team will bypass. The pipeline that works is the one your team owns, tunes, and trusts — because they were there when it was built, and the runbooks explain why each gate exists, not just how to configure it.