Changelog automation, and its limits

Changelog automation works when it automates collection, classification and publishing, and stops at selection and wording. Automate the whole thing and you ship a formatted git log; automate none of it and the changelog is written in bursts, from memory, before releases. The useful question is which parts to automate, not how much.

Changelog automation projects fail in one of two directions, and both are predictable from the first design meeting. Automate too little and the changelog is a document someone is supposed to update, which means it is updated in bursts, by whoever drew the short straw. Automate too much and it becomes a formatted git log: complete, accurate, and read by nobody.

Which parts of a changelog should be automated?

Three of the four steps. Collection and publishing completely; classification as a first pass with a human override; selection and wording never.

Collection. Getting changes out of the place they happen (commits, PRs, tickets) and into a list. Automate this completely. Humans are bad at it, it is tedious, and it is the step that gets skipped under deadline. Conventional commits or PR labels are the usual raw material.

Classification. Deciding whether something is Added, Fixed, Changed, Deprecated, Removed or Security. Automate the first pass from commit type or PR label, and let a human override. Accuracy here is around eighty percent from metadata alone, and the wrong twenty percent is concentrated in exactly the entries that matter, because ambiguity correlates with significance.

Selection and wording. Deciding what a reader should be told and how to say it. Do not automate this. It is the entire value of the artifact. Everything else is logistics.

Publishing. Getting the finished entries to a page, a feed, an email, an in-app widget, a Slack channel. Automate completely, and from one source. This is where most of the manual effort actually goes, and almost nobody counts it. It is also the step that can tell the person who asked for the change that it shipped, which is the whole of closing the feedback loop from the changelog side. The email half of that step has its own shape, in the product update email template.

That last point is worth sitting with. Teams tend to think of the changelog as a writing problem, then spend most of their time on distribution: copying entries into an email tool, reformatting for in-app, pasting into Slack, updating a docs page. The writing is an hour. The copying is an hour every release, forever, and it is the part a machine should have.

What happens when the line moves?

Move it up and you get a git dump. Full automation from commits produces bump deps, fix flaky test, wip and address review comments in front of customers. Every team that has done this has then added a filter, and the filter is a selection step reintroduced under another name, with worse ergonomics.

Move it down and you get bursts. Fully manual collection means entries are written from memory at release time. That is the mode Keep a Changelog opens by warning about, and it degrades quietly: the changelog looks maintained right up until the week nobody had time.

What does a changelog automation pipeline look like?

Four steps, with exactly one human gate, placed where a draft becomes public.

1. On merge, derive a draft entry from the PR: type from label or commit prefix, title as a first draft, link back to the PR, author recorded. Land it in an unreleased bucket.
2. Anyone can edit any draft at any time, and edits are cheap. Most get one line rewritten.
3. Cutting a release requires every entry in the bucket to be either edited or explicitly marked internal. This gate is the whole design. Without it, drafts ship unedited on the busy week.
4. Publishing is a fan-out from the released set: the public page, the feed, the email, the widget, the Slack post. One source, several renderings, no copying.

Step 3 is the only place a human is required, and it takes about ten minutes per release once the drafts are decent. Where a request from a customer is involved, the draft also carries the issue it closes, which is what lets step 4 tell the requester; the feature request template is designed so that link survives. Where this step sits in the wider release flow is the subject of the release management process.

What does automation require of your data?

None of the above works if the changelog is a Markdown file, because a file cannot be rendered into five surfaces without parsing it back out, and parsing prose is how you end up with a widget that displays half a heading.

Entries need to be structured: a type, a date, a version or release identifier, an audience, a body and a link. Then the file, the page, the feed and the email are all views. That structural point is the one thing worth getting right before you pick a tool, because it is the thing you cannot retrofit cheaply. None of it runs, though, unless an entry actually gets created for every change that needs one; enforcing a changelog entry in CI covers making the pipeline refuse a merge without one, instead of leaving that step to memory.

We build changeloop, where the changelog is a feed first and a page second, so treat that as an interest rather than an impartial recommendation; the pricing is one free repository with no card, which is enough to see the shape. Changelog tools is our roundup of what else is out there, including the products we compete with, and the changelog generator does the collection and classification steps in the browser if you want to see the derivation before committing to a pipeline.

The test

Count the minutes between a change being merged and that change being visible to a customer who does not read your repo. If most of those minutes are someone copying text between tools, the automation you need is in publishing, not in writing. If the writing side is where the minutes go, GitHub release notes shows how far a generated draft gets you.

FAQ

Can AI write the changelog? It can draft one. A model given the merged pull request produces a usable first draft of the title and body most of the time, which is the collection and classification steps done better. The selection, whether a reader should be told at all, and the final wording still need the person who knows the audience, and a pipeline that publishes drafts without that gate has automated the wrong step.

What is the difference between a changelog generator and changelog automation? A generator turns commits into a formatted list once, on demand. Automation runs on every merge, keeps an unreleased bucket, gates the release on human review, and publishes to every surface from one source. The generator is the first step of the pipeline, run by hand.

Should the changelog be automated from commits or from pull requests? From pull requests, where the unit of change is the PR: the title and description are written once, for the whole change, and the PR links the issue it closes. Commit-based derivation works when commits are the unit and are written to a convention.

How do you stop automation publishing internal changes? Classify chore, ci, test, refactor and dependency bumps as internal by default, and make promotion to public a deliberate act. The reverse default, public unless someone hides it, is how bump deps reaches customers.

The technical claims in this article have not been independently reviewed. If something here is wrong, tell us and we will correct it.

Related on changeloop: Changelog tools compared, Changelog generator