Changelog automation, and its limits
Changelog automation projects fail in one of two directions, and both are predictable from the first design meeting.
Automate too little and the changelog is a document someone is supposed to update, which means it is updated in bursts, before releases, by whoever drew the short straw. Automate too much and it becomes a formatted git log: complete, accurate, and read by nobody.
The useful question is not how much to automate. It is which parts.
Four steps, and only three of them should be automatic
Collection. Getting changes out of the place they happen (commits, PRs, tickets) and into a list. Automate this completely. Humans are bad at it, it is tedious, and it is the step that gets skipped under deadline.
Classification. Deciding whether something is Added, Fixed, Changed, Deprecated, Removed or Security. Automate the first pass from commit type or PR label, and let a human override. Accuracy here is around eighty percent from metadata alone, and the wrong twenty percent is concentrated in exactly the entries that matter, because ambiguity correlates with significance.
Selection and wording. Deciding what a reader should be told and how to say it. Do not automate this. It is the entire value of the artifact. Everything else is logistics.
Publishing. Getting the finished entries to a page, a feed, an email, an in-app widget, a Slack channel. Automate completely, and from one source. This is where most of the manual effort actually goes, and almost nobody counts it.
That last point is worth sitting with. Teams tend to think of the changelog as a writing problem, then spend most of their time on distribution: copying entries into an email tool, reformatting for in-app, pasting into Slack, updating a docs page. The writing is an hour. The copying is an hour every release, forever, and it is the part a machine should have.
What happens when the line moves
Move it up and you get a git dump. Full automation from commits produces bump deps,
fix flaky test, wip and address review comments in front of customers. Every team that has
done this has then added a filter, and the filter is a selection step reintroduced under another
name, with worse ergonomics.
Move it down and you get bursts. Fully manual collection means entries are written from memory at release time. That is the mode Keep a Changelog opens by warning about, and it degrades quietly: the changelog looks maintained right up until the week nobody had time.
A pipeline that holds
- On merge, derive a draft entry from the PR: type from label or commit prefix, title as a first draft, link back to the PR, author recorded. Land it in an unreleased bucket.
- Anyone can edit any draft at any time, and edits are cheap. Most get one line rewritten.
- Cutting a release requires every entry in the bucket to be either edited or explicitly marked internal. This gate is the whole design. Without it, drafts ship unedited on the busy week.
- Publishing is a fan-out from the released set: the public page, the feed, the email, the widget, the Slack post. One source, several renderings, no copying.
Step 3 is the only place a human is required, and it takes about ten minutes per release once the drafts are decent.
What this requires of your data
None of the above works if the changelog is a Markdown file, because a file cannot be rendered into five surfaces without parsing it back out, and parsing prose is how you end up with a widget that displays half a heading.
Entries need to be structured: a type, a date, a version or release identifier, an audience, a body and a link. Then the file, the page, the feed and the email are all views. That structural point is the one thing worth getting right before you pick a tool, because it is the thing you cannot retrofit cheaply.
We build changeloop, where the changelog is a feed first and a page second, so treat that as an interest rather than an impartial recommendation. Changelog tools is our roundup of what else is out there, including the products we compete with, and the changelog generator does the collection and classification steps in the browser if you want to see the shape before committing to a pipeline.
The test
Count the minutes between a change being merged and that change being visible to a customer who does not read your repo. If most of those minutes are someone copying text between tools, the automation you need is in publishing, not in writing.
The technical claims in this article have not been independently reviewed. If something here is wrong, tell us and we will correct it.