Avoid Vendor Lock-in for your Build Server/Continuous Integration provider

One of the best steps a developer (especially in a team or company) can do is to setup a continuous integration solution. Either by running an actual build server (using e.g., Jenkins, TeamCity, Bamboo, Azure DevOps Server (formally Team Foundation Server/TFS), or by using one of the many cloud providers that have sprung up in recent years (Travis CI, CircleCI, Azure DevOps, GitHub Actions).

I maintain a .net Library to create Excel Sheets (Simplexcel), and have used several of these services over the years - from Jenkins to Travis to Azure DevOps to GitHub Actions. And the number one lesson that I learned is... well, the number one lesson would be "You should absolutely use a CI Server, don't just build and publish your code locally". But the number one lesson I've learned from actually using a CI Server is to avoid Vendor lock-in as much as possible.

It's tempting to look at the automation offerings of each platform and create a massive .yaml file with instructions on how to build things and do all sorts of stuff, but if you switch providers, you now basically have to start setting up your builds from scratch. Also, if you do need to do a local build, how would you do that if you depend on the build server to do a bunch of stuff?

In my specific case, I've decided to have an actual build script that runs on PowerShell Core, so it works on Windows, macOS, and Linux. The build script does all the heavy lifting to create an artifact:

  • Determine version information
    • Using a base version specified via build_version, and setting a minor revision by counting git commits (so that I'm not depending on some counter maintained by an external service)
  • Determine if this is a development/pre-release or a production/release build
  • Update files accordingly if needed (e.g., string-replace version information)
  • Compile the code (in this case, dotnet restore and dotnet pack)
  • Run Tests

It's driven by a properties file, but also reads environment variables which can override things.

The advantage is that I can run this locally on my machine, and get the desired artifacts created. But I can also run the same on the build server and have the artifacts created there using the exact same build script.

My GitHub Actions build script handles some things that are not applicable or practical locally. It does:

  • Get a Linux VM
    • Checkout the applicable git revision
    • Run the build script to make sure it compiles on Linux
    • As part of the build script, run tests
      • A failed test will fail the build
  • Get a Windows VM
    • Checkout the applicable git revision
    • Run the build script to create the nuget packages and run tests
      • A failed test will fail the build
    • If we succeed, push the Nuget Package either to GitHub Packages for development builds, or Nuget.org for production builds.

Now, because this is only a single library and not e.g., a suite of interconnected services that need to be orchestrated and treated as a single unit, the build process is fairly simple.

But it does serve as an illustration on what I try to keep in my vendor-specific build scripts, and that's basically only the things that HAVE to be specific to their environment - if I was to switch from GitHub Actions to another provider, chances are that "runs-on: ubuntu-latest" will not be supported there and I need another magic word to get a Linux VM.

But everything that happens once I have a machine to build on, and my source code checked out should be in an independent build script that I can run locally. This includes Compilation, Tests, but could also extend to things like building a Docker Container (probably using Buildah). Even the publishing of Nuget packages could be in my build script, since it only relies on credentials provided via the environment, but for this I've decided that the .yaml was a good place for it - GitHub's Package Registry is GitHub specific anyway, and for Nuget.org I can just upload a locally built .nupkg via their Web UI.

Build Servers change, and sometimes, your provider shuts down or changes their pricing. Or maybe something new comes along that's better for your circumstances.

You cannot avoid some vendor lock-in, but it's in your own best interest to try to minimize how much vendor-specific functionality you use and how much you can keep generic, runnable on any system.