Every commit or pull request on a team that automates delivery sets off the same workflow. That workflow is the build pipeline, and it turns each source code change into a tested, packaged build artifact by running ordered stages on every trigger.
A typical run checks out the source and compiles it, then tests and scans it before packaging and publishing the result.
Vendors don’t use the term the same way. Microsoft Learn treats build and release as two separate pipeline types, while Jez Humble and David Farley’s deployment pipeline (Addison-Wesley, 2010) is wider, covering every step from check-in to release.
Go with the second wording and the build stages become only the first segment of that larger process.
What Is a Build Pipeline?
None of it is run by hand. Teams treat the pipeline as the build half of a continuous integration setup, so it fires whenever code is pushed or proposed for merge.
It needs very little to get going. The pipeline configuration file says what to do, and the source code and dependency manifests (pom.xml or package.json, for example) supply the material.
What comes out depends on the stack.
- JAR or WAR files from a Java build
- Docker images
- npm packages
- NuGet packages or published DLLs from .NET
- Test reports, logs and a pass or fail status
A successful run always ends with a build artifact, meaning a versioned file or image that later stages deploy unchanged.
Artifacts don’t stick around forever, though. As of November 2026, GitHub’s upload-artifact action keeps them for 90 days by default, and the retention-days input shortens that window. I’d shorten it on most projects.
It’s easy to mix the pipeline up with the things around it. A build script is one step inside it, and a build server only hosts the run.
The deployment pipeline sits further along, since that one starts where the artifact is stored.
Build Pipeline Stages in Execution Order
Most pipelines follow the same order, give or take.
- The runner begins with a source checkout, fetching the triggering commit from the Git repository.
- Compile comes next. Maven, Gradle or npm resolves dependencies and builds the code.
- Tests run unit first, integration second.
- A scan follows, covering linting, static analysis and a quality gate such as SonarQube.
- Whatever survives gets packaged as a JAR, a binary or a container image.
- Publish is the final stage, where the artifact is uploaded to a repository such as JFrog Artifactory or Sonatype Nexus Repository.
Linting often moves ahead of compile because it fails in seconds, and I’d do the same on a new setup. Order shifts by project, though.
Unit tests go ahead of integration tests for the same reason. They finish fastest, so a broken commit gets rejected early.
The gate logic matters more than the order. GitLab runs stages in sequence and the jobs inside one stage in parallel, and a later stage starts only after the earlier jobs succeed (GitLab CI/CD documentation).
A red test stage therefore means no package stage and no artifact. That’s the whole point of a quality gate.
Inside a stage, the unit of work is called a job in GitLab and a step in a Jenkins stage. When the run finishes, it leaves behind a build status and logs, and the committer gets a notification.
What Triggers a Build Pipeline?
A push to a watched branch is the obvious one, and every such push starts a run. Opening or updating a pull request does the same, and it builds the proposed merge before anyone reviews it.
Nightly or weekly builds come from a cron expression. Then there’s the manual run, where a person kicks things off from the CI interface or API (workflow_dispatch in GitHub Actions).
Behind the scenes, the Git host tells the CI system about a push or pull request through webhooks, and the CI system checks that event against the triggers in the pipeline file.
Azure Pipelines has one more, a pipeline completion trigger, which lets one pipeline finishing start another.
Per Microsoft Learn (as of November 2026), a YAML pipeline in Azure Pipelines gets a CI trigger on all branches by default, with pull request triggers switched on by default as well.
That means a pipeline file with no trigger block builds every branch push, unless the Disable implied YAML CI trigger setting is turned on. Setting trigger: none switches CI builds off entirely.
The IREE project splits things sensibly, per its developer documentation. Jobs closest to everyday developer work run on pull_request, and nightly jobs sit on a schedule.
Scheduled runs are the least predictable of the bunch, since GitHub notes the schedule event can be delayed during periods of high load.
To push a commit without starting a run, add [ci skip] or [skip ci] to the commit message, as GitLab documents.
How a Build Pipeline Is Defined and Where It Runs
The definition sits in a configuration file stored in the repository, which is what people mean by pipeline as code. Execution happens on a build agent or runner, a virtual machine or a container.
| Platform | Config file | Executor name |
|---|---|---|
| Jenkins | Jenkinsfile | Agent |
| GitHub Actions | Workflow YAML in .github/workflows | Runner |
| GitLab CI/CD | .gitlab-ci.yml | Runner |
| Azure Pipelines | azure-pipelines.yml | Agent |
Pipeline as code files
Keeping the file next to the source means pipeline changes go through the same review as code. Microsoft Learn names that as the main benefit of YAML pipelines in Azure DevOps.
GitHub Actions workflows live in .github/workflows, while GitLab looks for .gitlab-ci.yml in the project root unless a custom path is set.
Secrets stay out of the committed file. GitLab holds passwords and API keys as CI/CD variables set in project settings, and the pipeline only references them.
The catch with a file in the repository is that anyone with write access can change what the pipeline does, even though the file is easy to review.
Agents, runners and build servers
On GitHub’s hosted runners, a new virtual machine is provisioned for each job (GitHub Docs). Self-hosted runners are machines you run yourself, and their usage limits differ from GitHub’s.
Jenkins agents are declared with the agent block and pinned by label when a stage needs specific tooling (CloudBees refcard).
The build server is whatever machine or service hosts those agents and queues the work.
Hosted runners do have a ceiling. According to GitHub’s Actions limits page (as of November 2026), a job that reaches 6 hours of execution time is terminated and fails.
Stay on hosted runners until a job needs hardware or network access they can’t provide, or a run time they don’t allow. Then self-host.
Writing a minimal first pipeline file
A first file doesn’t need much.
- Commit the pipeline file to the repository
- Trigger it on pushes to the main branch and on pull requests
- Run install, test and build, in that order
- Upload the build output as an artifact
Add scans, matrices and caching only after the first green run.
Build Pipeline vs CI/CD Pipeline vs Deployment Pipeline

The build pipeline’s job stops once the artifact is stored. From there, a deployment pipeline takes that artifact and carries it through environments until it reaches production.
Jez Humble and David Farley introduced the deployment pipeline, and they describe it as an automated process that manages every change from check-in to release (publisher description of their book Continuous Delivery, Addison-Wesley, 2010).
By that definition, the build pipeline is the first segment of a deployment pipeline, not a separate thing.
Azure Pipelines draws the line somewhere else. Microsoft Learn treats build and release as two pipeline types with separate interfaces, and its release pipelines can deploy artifacts from Azure Pipelines builds as well as from Jenkins or TeamCity.
Both vocabularies are fair. Just check which one a team means before comparing tooling or metrics.
Microsoft Learn’s capability table (as retrieved in November 2026) shows where the two Azure pipeline types part ways.
| Capability | Classic build | Classic release | YAML |
|---|---|---|---|
| Stages | No | Yes | Yes |
| Approvals | No | Yes | Yes |
| Gates | No | Yes | No |
| Caching | Yes | No | Yes |
Classic build has caching and none of the rest. Classic release brings stages, approvals and gates, but no caching.
YAML pipelines carry stages and approvals, which is why a single file can hold both halves. Gates are the exception.
Once you’re past the artifact, the real question is who approves production. Continuous delivery waits for a person, whereas with continuous deployment every passing change goes out on its own.
The artifact is the handoff point. Per Microsoft Learn, a release can redeploy its artifacts to any stage defined in it, so each environment receives the same files instead of a fresh build.
My rule of thumb: keep build and deploy stages in one YAML file while there’s a single deploy target. Split them once approvals, gates or several builders feeding one release need separate owners.
How to Speed Up a Build Pipeline
Caching is the first thing to look at, meaning reuse of downloaded dependencies and task outputs between runs. Running independent test suites at the same time comes second. A build matrix tests several versions or operating systems with each one as its own job, and fail-fast cancels the remaining jobs after the first failure.
Gradle makes the caching idea easy to see. Build automation tools such as Gradle compute a cache key from a task’s inputs and load stored outputs instead of rerunning the task, according to the Gradle build cache documentation.
HCL Software’s VersionVault documentation says parallel builds can cut overall build time significantly, and HCL’s OneTest guidance makes the same case for running tests in parallel across several nodes.
A matrix multiplies jobs. Two operating systems and four Node versions make 8 jobs, each on its own runner.
GitHub Actions cancels the remaining matrix jobs after the first failure unless fail-fast is set to false. Turn it off when you need the full result set, not just the first red job.
The Terraform AWS Provider repository, maintained by HashiCorp, runs 8 or more workflows that compile Go code across 500+ active pull requests, according to its contributor docs.
A few numbers from those docs are worth having in one place.
- GitHub Actions gives each repository 10 GB of default cache storage, and entries unused for 7 days are removed (actions/cache documentation).
- One matrix can generate at most 256 jobs in a workflow run (GitHub Docs).
- The Terraform AWS Provider docs list build time at 10-15 minutes before and 2-3 minutes after its caching strategy (HashiCorp contributor docs).
- Under naive cache keys, that same project would need an estimated 32,000 GB of cache (8 workflows x 500 pull requests x 8 GB), against GitHub’s repository limit of 10 GB (HashiCorp contributor docs).
Older write-ups call 10 GB a hard cap. GitHub’s November 20, 2025 changelog lets repositories on a paid plan exceed it on a pay-as-you-go basis, while the default stays at 10 GB and seven days.
Read together, the numbers say cache key design matters more than cache size. HashiCorp keys its Go build cache on the runner OS and a date, and leaves go.sum out of that key on purpose, so workflows and pull requests share entries. Only the smaller module cache is keyed on go.sum.
Caches fail quietly. The same docs name a different runner OS and eviction at the size limit as causes of a miss. A major dependency update that invalidates most of Go’s internal cache is the cause of a hit that still builds slowly.
No standard sets a target duration. Pick one, track it per stage, and treat a creeping median as a defect.
Every one of these levers adds state (cache entries, matrix cells, cancelled runs) that someone has to debug later, and that someone is usually whoever turned the lever on.
When a Build Pipeline Breaks or Falls Short
The failures worth planning for are flaky tests, dependency drift, builds nobody can reproduce, and tampering with the build itself.
Flaky tests and dependency drift
A flaky test passes and fails on the same code, which makes a red build hard to trust.
Google’s Testing Blog (2016) reported that about 84% of the pass-to-fail transitions it observed involved a flaky test.
Size drives the rate. Google’s 2017 follow-up found 0.5% of small tests flaky against 14% of large ones, a 28-fold gap.
Google’s own mitigation is fairly blunt. A test marked flaky only reports failure after failing 3 times in a row. These remain the most widely cited public figures, and they date from 2016 and 2017.
Quarantine known flaky tests in their own job instead of retrying the whole pipeline.
Unpinned versions are how two runs of the same commit end up with different packages. npm ci installs the exact versions in the lockfile and exits with an error when package-lock.json and package.json disagree, per the npm CLI documentation. A plain npm install can rewrite the lockfile as it resolves, and inside a pipeline that’s the wrong behavior.
Non-reproducible builds
The two main sources don’t define a reproducible build the same way.
reproducible-builds.org says the same source code, build environment and build instructions let any party recreate bit-by-bit identical copies of the specified artifacts. The OpenSSF glossary is looser. It names the same source code as the only stated condition, though environment and instructions are usually needed to get there.
The stricter wording has practical weight. The reproducible-builds.org project lists dependency versions, build flags and environment variables such as locale as relevant environment attributes, so a pipeline has to pin them.
Verification is a bit-by-bit comparison, usually done with cryptographic hashes. Build the same commit twice on clean runners and compare hashes, because a mismatch is the whole diagnosis.
Supply chain tampering
The SLSA build track grades how well a build resists tampering. Version 1.2 is current, and its build levels are the same as in v1.0.
| Level | Requirement | Focus |
|---|---|---|
| Build L1 | Provenance showing how the package was built | Mistakes, documentation |
| Build L2 | Signed provenance from a hosted build platform | Tampering after the build |
| Build L3 | Hardened build platform | Tampering during the build |
SLSA’s own guidance points teams that want moderate benefit toward a hosted build platform at L2, and notes that L3 usually needs significant changes to the platform itself.
For most teams, L2 on a hosted runner is the realistic target.
Signing covers the other half. Sigstore’s cosign uses keyless signing, where the Fulcio certificate authority ties a certificate to an identity and the Rekor transparency log records the signing event.
For a container image, cosign uploads the signature to the OCI container registry next to the image, so the signature travels with the artifact.
Signing proves who built an artifact. It does not prove the build was clean, which is what the SLSA levels grade.
Build Pipeline FAQ
Is a Jenkins pipeline the same as a build pipeline?
No. A Jenkins Pipeline is one way to define a build pipeline, written in a Jenkinsfile and run on a Jenkins agent.
The term build pipeline names the workflow itself, which GitHub Actions and GitLab CI/CD also run, and so does Azure Pipelines.
What is the difference between a build pipeline and a build server?
The pipeline is the defined sequence of stages, and the build server is simply the machine or service that executes it.
One server usually hosts many pipelines, and moving a pipeline to another server changes where it runs, not what it does.
Does a small project need a build pipeline?
Yes, a small one. Three stages (install, test, build) take minutes to write and reject broken commits before they reach the main branch.
Scans, matrices and caching can wait until the project grows.
Can a build pipeline run on a developer machine?
Yes, the commands inside it run locally, but the pipeline itself runs on a runner or agent.
That separation keeps results independent of one person’s installed tools and dependency versions, a common cause of builds that pass locally and fail in CI.
How much does it cost to run a build pipeline?
Standard GitHub-hosted runners are free for public repositories, and self-hosted runners carry no GitHub usage charge, according to GitHub Docs. You still pay for the machines.
Private repositories draw on included minutes first, and usage beyond that is billed. Check the platform’s current price list, because rates change.
What to Add to a Build Pipeline First
Trust comes before speed. Gate pull requests first, then quarantine flaky tests, then add caching, and sign artifacts last.
Each step protects the signal of the one before it.
Matrices have a hard ceiling. GitHub’s 6-hour job limit and 256-job matrix cap together allow 1,536 runner-hours in one workflow run, so a matrix that large belongs on a nightly schedule, not on every push.
The cost is a longer wait for feedback with each stage added, which is why the last two steps wait for evidence of need.
This order was checked in November 2026. SLSA v1.2, published in November 2025, added a Source track alongside the Build track, which covers how source code is produced and reviewed.
With a stored, signed artifact in hand, the next decision is how it reaches users through app deployment.
- Who Answers Your Users After Launch? - November 18, 2026
- How to Create a Folder in GitHub From the Browser - November 17, 2026
- 6 Best Cookie Consent Apps for Shopify in 2027: Features and Setup Compared - November 17, 2026



