Commit’s peer-to-peer bot helps community members tap into the vast expertise of our peers and colleagues to ask questions that span the spectrum from technical to personal. Here’s the latest question and answers.
Alex Gogan: What recommendations do you have for creating automated changelogs with a trunk-based branching strategy and a build-once, release-many-times mindset? I’m interested in the detailed process rather than the tooling.
For reference, we currently use the Nx Workspace as a monorepo along with Azure DevOps and Google Cloud Build for CI/CD and semantic-release-plus for changelog generation.
Our current workflow:
- We have one long-lived branch called “main”
- At any given time we will merge new features from a feature branch into main via pull requests (squashed and rebased) using standardized commit messages (conventional commit)
- Each new commit on the “main” branch triggers a new automated build to create the project artifacts for deployment
- Each new build artifact will be released continuously into a staging environment
- At some point in time we “promote” a set of features from staging to a sandbox environment (integration)
- At some point in time we promote a set of features from the sandbox environment to production
- As releases are detached from commits, my big question is: at what point of time do you generate a changelog and how do you bring that changelog back into your code base?
Is it best to:
- Create a new branch for each release from the point of time the build artifact was created?
- Generate the changelogs?
- Merge those back into “main” with a flag to skip CI?
It seems to be a fairly common topic but I don’t seem to find very helpful material out there.
David Cheung: I’m trying to understand your workflow. When you say “promote a set of features,” do you mean in staging environments you would promote individual services (artifacts) to the next stage once that artifact is deemed worthy? And because of that, when you go to production you would have:
- Service 1: Promoted from [commit: A]
- Service 2: Promoted from [commit: B]
- Frontend: Promoted from [commit: C]
- Gateway: Promoted from [commit: A]
Its hard to tell what the combination of artifacts (and diff) are, since its monorepo and commit hash are shared across all artifacts. And hard to build a changelog between one production build and another.
I don’t think our approach is helpful in a monorepo scenario. Maybe someone with more monorepo experience can chime in, but this is what has been working for us in a multi-repo setup.
The way we’ve done changelogs is by tags. It will regenerate the changelog between all tags since inception all the time, so it will be idempotent and always the same whenever triggered. assuming the branch is FF only.
For repositories that users would not directly pick a version to consume (e.g., internal tools or services we host), we occasionally tag commits with a version number, then use GitHub to compare between tags. Since the our commit messages are also standardized, this approach has been adequate to quickly tell the difference between tags.
For repositories that are consumer facing, where consumers would pick a version to use (e.g., CLIs, SDK clients, terraform modules), we follow semver and cut releases and use tools like (goreleaser/release-it(npm)/semtag+gitchglog) to generate changelogs when we create a release, along with tagging a commit.
In terms of CI/CD, we only run stage/prod, we put an approval step before production build+deploy (via circleci/github actions) to give us the flexibility, so every commit (main) has the option to be promoted to production, but depending on the product, some would go to production almost every commit and some repos can go to production quite rarely.
Alex Gogan: Great feedback. The challenge is actually less around a monorepo but rather a trunk-based development strategy.
This chart shows one of our applications. Through CI/CD there is a direct link between commit (feature merged into main) and a release into staging.
But, a sandbox as the second layer on top is selective when we as developers “deem” the changes to be ready for a pre-integration. What happens is that we take the artifacts from that specific point in time and release these artifacts into the next environment by promoting it. Of course development doesn’t stop, so by the time we test, other new features make their way into the main branch.
Tagging is a great way to mark releases and determine changes based on those tags. But as I now have two tags (in the past), I would like to have a permanent changelog written back to code. How would I do that? It feels like I would always create the changelogs (in a CHANGELOG.md) with a time delay.
I’m wondering what is good common practice to do it in a clean way. Overall we’d like to leverage semver for versioning, but I’d be relying on clean build and release IDs, with buildId being when the artifacts were generated and releaseId being when a deployment into an environment happened.)
Bill Monkman: If you can make sure people are consistent with their git comments you could use something like git-chglog to auto-generate the changelog file.
You could even put that into your build pipeline and set up a step where, for example, each commit that goes to staging gets tagged with an incremented patch version and each one that goes to production gets a minor version. So the pipeline itself would create a tag, run git-chglog, then commit that back to the repo.
Basically, using this example it would be running [make changelog] [make release] [git commit] [git push].
David Cheung: Oh, so the artifacts are tied to the environment and not long lived (like docker images) and stored somewhere, but instead are replaced as commits go into staging? Or you’re trying to get changelog delta from selected buildID -> buildID?
Or is the challenge that you ONLY want to “cut releases” when things have been tested all the way, and when its tested all the way there are commits ahead of that hash? If this is the case you could do something like the excerpt below—the changelog is entirely regenerated based on history every single time, and based on tags and tagging old commits, so your changelog generation is not time-depended at all, it’s just an update of records.
David Cheung: The way we’ve done change-log is by tags, so it will regenerate the changelog between all tags since inception all the time, so it will be idempotent and always the same whenever triggered. (assuming branch is FF only)
Alex Gogan: @Bill, that’s the thinking. I just found it a bit odd to do a commit when a new release happens and that new commit is ahead of the tagged build artifact, but I guess in a continuous integration environment that would do just fine.
My main goal is to know the delta between tags so I can use it downstream to send, such as with Slack messages about all the features and fixes that were just deployed on an environment.
For technical implementation, we’re using conventional commit messages (e.g., [feat(auth): add Google Provider]) and comparing based on release tags between releases to auto-generate the changelog.md and leverage the semantic-release package.
Bill Monkman: If you’ve standardized your messages you’re already in a really good spot.
✅ ✅ ✅