Scoutman
Scoutman
M365 and Azure MVP,
Enterprise Architect 🟔🟔🟔
thoughts on Architecture, Automation, Development and Technical Leadership
M365 Azure SPfx PowerShell DevOps keep it simple. keep it honest. keep it real.

āšœļø DevOps Concern: Artifact Immutability

DevOps Besides architecture , M365 ands Azure Stuff i do a lot of DevOps work and something keeps popping up in some of projects I’m helping with : misuse of the artifact immutability concept.

We build an solution in our dev environment that goes through the different stages:test,stating and finally production. Usual stuff right?

But somewhere along the road, the pipeline starts doing some tweaks: it replaces something, swap or rewrite text or copies a file into the package. Hey, since its the pipeline doing it automaticly it feels repeatable , controlled. Safe right? Wrong!

A good pipeline is one of the best thing we, as a team can have . The problem starts when the pipeline stops being the yellow brick road that promotes the solution and become a place where the solution is changed. ugh!

Packages built, tested and approved ALWAYS should be treated as immutable assets!

Dev, test, staging, acceptance, production, beta stages -»> same package.

Are we allowed to have different configuration on each one of them? Absolutely. Are we allowed to have different secrets? For sure!

Now… different package contents? That is in fact a show stopper.

Build Once, Promote Many

The release model we all follow is indeed a bit boring

  1. Solution is build once.
  2. A versioned artifact is created and stored with a version
  3. Same artifact us promted through the different environments.
  4. From outside the package, each environment provides its own configuration .

Its simple right? Funny enough simple ideas have a way to becoming complex\disputable once they start hitting real projects, old habits … and deadlines.🤠

If the artifact\package deployed to production is not the same that we tested, what exactly did the testing proved?

Now the package is a slightly different gremlin.

Perhaps is a just a tiny , waning , penny thing that changed but … the defined trust model changed.

This Is Not Just Me

I wanted to check if this was only my stubborn head being stubborn, so I’ve search at how this appears in common delivery guidance.

Here’s the findings:

The Twelve-Factor App separates build, release, and run . Then code and dependencies become a build,build is combined with configuration and a realease is created and executed.

That separation matters : the build stage should not be altered later in the deployment path.

Microsoft’s own Azure DevOps documentation points out the same direction. In Azure Pipelines artifact documentation, its explicit explained that Azure Artifacts packages are immutable:once a package is published, that version is permanently reserved. That is a product rule,and a good release habit.

The AWS Well-Architected Framework also explicit recommends immutable deployment practices to enforce reliability, consistency, and reproducibility.

AWS is mostly talking about infrastructure there, but the concept applies just as well to application artifacts: DONT PATCH THINGS in place

And if we look at software supply chain work like SLSA, the emphasis on build integrity is unmistakable..

The more we care about knowing where an artifact came from, the less comfortable we should be with deployment tooling modifing it after the build.

So no… thats not being stubborn :P

If a DevOps process changes code, scripts, manifests, bundled assets, templates, packaged files differently per environment, IT IS DRIFTING AWAY FROM IMMUTABLE DELIVERY, it still automated,still familiar, … but it is not clean.

What Should Be Different

Environments are indeed allowed to be different, in fact they usually have to be.

Different environments can have different settings, flags,values… no issues with any of that : the important is where those differences are.

They should live in configurations, not inside a modified package.

The moment the package changes, the pipeline is no longer only deploying, it becomes a maker, and editor.

… and that is a risky place to be.

Smells Like Teen Spirits

DevOps Here’s are a few patterns that immediately make me shiver:

  • Replacing tokens inside JavaScript bundles after the build
  • Modifying PowerShell scripts per environment
  • Rewriting manifests during deployment
  • Injecting environment-specific JSON or XML into the package
  • Patching compiled or bundled assets in transit
  • Building dev, test, uat, and prod versions of the same package
  • Using deployment tasks to alter files that were already tested
  • Overwriting a package version with different content
  • Treating CI output as raw material instead of a finished artifact

I do understand why some teams end up here1: sometimes the pipeline was inherited from three projects ago and nobody wants to touch it , an urgent release and replacing a token in a deployment task feels like the fastest way out.

I’m not judging anyone who make those choices under pressure: im just stating that this pattern should not be normalized.

The Trap Of Environment-Specific Builds

Environment-specific builds can look nice, tidy and pretty …

  • myapp-dev.zip
  • myapp-test.zip
  • myapp-uat.zip
  • myapp-prod.zip

… where the naming is clear and the values are already inside… where deployment feels straightforward.

But production did not received the same artifact that went through the earlier environments : it has the artifact built for production.

The more often we rebuild or transform, the more distance we create between “tested” and “shipped” :that distance is where release risk hides.

A safer pattern is a single package identity:

  • myapp-1.4.7.zip
  • myapp:1.4.7
  • MyCompany.MyProduct.1.4.7.nupkg
  • runbook-package-2026.05.10.3

One package. Many environments. period Let the environments bring their own values. Let the package remain itself.

Scripts And Assets Count

This is not only about compiled application code: scripts,json filestemplates, yaml,runbooks, any asset count

If a file affects production behavior, then that file is part of production : a PowerShell script that performs operational work, a JSON file that changes how the app behaves, a static asset that contains an endpoint, tenant value, or feature behavior, it matters.

Calling it “just a file” does not make it less important.

We should not let the deployment pipeline become the place where production “code” is quietly edited.

Same Model. Any Tool

Same Model

This principle applies if we use Azure Devops. GitHub Actions, GitLab CI, Jenkins, Octopus Deploy, Argo CD, Terraform, Bicep, PowerShell, Bash, containers, NuGet, npm, ZIP files, or a deployment tool that has survived longer than anyone expected.

The tool does not matter as much as the model.

  1. Build once.
  2. Version it.
  3. Store it immutably.
  4. Promote the same artifact.
  5. Inject configuration from outside.
  6. Keep provenance.
  7. Use approvals and gates.
  8. Deploy without rewriting the package.

Tooling should support the release model and it should not quietly redefine it.

Why I Care

This may sound strict, but it is really about making operations calmer: immutable packages reduce the number of mysteries. It makes audits less painful because source commit, build logs, tests, artifact version, and deployment history all connect to the same thing.

Mutable deployments create the opposite feeling.

They leave teams asking:

  • Was production really running the tested version?
  • Did the deployment task change more than expected?
  • Did one environment receive a different asset?
  • Was a token replacement missed?
  • Can we reproduce the package that failed?
  • Can we prove which source produced the running files?

Those are not questions we want to answer for the first time during an incident call.

Final Notes

Final Notes

DevOps should be a trustworthy bridge between development and operations where the artifact needs to keep its identity while it crosses.

When the pipeline starts rewriting the package, the pipeline becomes a second development surface, and the release becomes harder to explain.

So my position is simple:

  • Use DevOps to promote packages through environments.
  • Use it to control approvals.
  • Use it to manage configuration.
  • Use it to make delivery repeatable.

But do not use it to rewrite artifacts.

Immutable packages are not an DevOps fashion or a tooling preference. They are part of reliable delivery:

  • Build once and promote the same artifact
  • Configure environments from outside the package
  • Preserve source lineage for audit and provenance
  • Keep approval gates meaningful
  • Make rollback point to a real, known version

Hope it helps! 🤠


See also