Closing the customer feedback loop from the changelog side

A customer feedback loop is closed when the person who gave the feedback is told what happened to it. Not when it is filed, not when it is prioritised, not even when it ships. When they are told. Most teams run the first three steps well and the last one not at all, and then wonder why the people who send feedback stop sending it.

This article is about that last step, and about a specific claim: the changelog is the right place to close the loop from, because it is the one artifact that already exists at the moment the loop can be closed.

What is a customer feedback loop?

A customer feedback loop is the path from a user telling you something to that user learning what you did about it. It has four steps: collect the feedback, decide what to do with it, ship the result, and tell the person who asked. The loop is open until the fourth step happens. A team that collects feedback and ships fixes but never tells anyone has an inbox, not a loop.

The fourth row is the one this article is about. It breaks for a structural reason, not a cultural one: by the time a feature ships, the request that caused it is in a different system from the thing that shipped, and nobody’s job is to connect them. The loop starts earlier, with how the request is asked for in the first place; how to ask for customer feedback covers the wording and the timing.

Why do feedback loops stay open?

Feedback loops stay open because the request and the shipped change live in different places and the link between them is made by hand, if at all. The request is in a feedback tool, a support inbox or a spreadsheet. The change is in a pull request. The announcement is in a changelog or an email. Three systems, three owners, and the link from the third back to the first is a person remembering, months later, who asked.

There is a second reason. The tell step is usually framed as a marketing task (“announce the feature”) rather than a support task (“reply to the person”). Announcements go to everyone and reach no one in particular. The person who asked for the feature in March reads the June announcement, if they read it, as news, not as a reply. The loop closes only if the message is addressed to them.

Why close the loop from the changelog?

Because the changelog entry is the one artifact that exists at exactly the right moment, contains exactly the right words, and is written by exactly the right person. It exists when the change is live and not before. It says what changed in the reader’s terms, which is the message the requester needs. And it is written by someone who has just read the pull request, which is the only moment the link to the original request is still visible.

Compare the alternatives. Closing the loop from the feedback tool means the feedback tool has to know when the feature shipped, which means someone updating a status by hand. Closing it from the pull request means telling the customer at merge, before the change is live, which is a broken promise with a timestamp when the deploy is delayed. Closing it from the marketing announcement means waiting for one, and most shipped changes never get one.

The changelog sits in the middle: after the merge, at the moment of release, with the wording done.

How the loop closes, step by step

This is the mechanism we run. It is described here as a specification rather than a product tour, because every step can be done by hand or with other tools; what matters is the order.

1. Feedback becomes an issue in the repository that will fix it. A widget submission is filed as a labelled GitHub issue (feature-request or bug, a priority, and from-widget), with the submitter’s email address kept out of the issue body. The issue lives next to the code, so step three can find it. An issue filed by hand, for example from a feature request template, is outside this path: step five does not comment on it, so close that loop yourself.
2. The fix references the issue. The pull request says Fixes #142, GitHub’s own closing keyword. Nothing new to learn, and it is the same sentence developers already write.
3. The changelog entry is drafted from the merged pull request and carries the link. At merge, the draft is created and the #142 is read from the PR body and attached to the draft. The link is made while it is still cheap, by a machine, from data that is already there.
4. A human reviews the entry. Wording, audience, whether it should be published at all. A discarded draft closes nothing, which is correct: an internal refactor that happened to reference an issue is not news.
5. On approval, the requester is told. A comment is posted on the issue their feedback became, “Shipped —” followed by the entry’s title and a link to the published entry, and the widget shows the submitter the same shipped entry. Once, never twice, and only after a person has released the entry. The same entry goes out through the feed and widget to everyone who did not ask.

The order in step five is the whole design. Telling the requester at merge would be earlier and easier, and it would be wrong roughly as often as deploys are delayed. A feature flag breaks even this order, because approved and published can happen while the feature is still invisible to the requester’s own account; feature flags and feature requests covers the extra check that step needs once a flag is involved.

What does a closed loop look like to the customer?

It looks like a reply. The customer sent a request through a widget, and one day the widget shows it as shipped, with a link to an entry that describes it in their terms; on GitHub, the issue gets the same news as a comment. They did not subscribe to a newsletter, check a roadmap, or search the changelog. They were told.

That is the experience that makes the second piece of feedback happen. People send feedback to products that answer. The changelog examples page includes entries from teams whose users visibly keep coming back with requests, and the common thread is not the tooling; it is that the entries read as replies.

How do you measure a feedback loop?

Measure the fraction of shipped changes that told at least one requester, and the time from ship to tell. Two numbers, both easy once the link exists and impossible before.

• Close rate: of the changelog entries published this month, how many linked to at least one request, and of those, how many notified the requester. If the second number is much lower than the first, notifications are failing; if the first is low, requests are not being referenced from pull requests, and the fix is a sentence in the PR template.
• Ship-to-tell time: how long between the entry going live and the requester being told. With the mechanism above it is seconds. By hand it is typically weeks, or never, and “never” is the number that matters.

Do not measure the loop by volume of feedback collected. Collection is the easy step, and a team that measures it will optimise it, which produces more open loops.

Where does the roadmap fit?

A public roadmap is a way of closing the loop early: it tells requesters their request was heard before it ships. It is useful, and it is not a substitute for the last step. “Planned” is a promise about the future; “Shipped” is a fact about the present. Run the public roadmap from the same issues, with a label per column, so that the same request moves from planned to shipped without being re-entered anywhere. The move to shipped is a label change (roadmap:shipped) that nothing makes for you when the entry is approved, so do it in the same review.

FAQ

What are the four steps of a customer feedback loop? Collect, decide, ship, tell. The loop is open until the fourth step. Most frameworks add analysis and prioritisation steps in the middle; they are refinements of “decide”, and none of them closes anything.

Should you tell customers when you decline a request? Yes, and it is the most neglected message in the loop. A clear “we are not going to do this, and here is why” ends the wait. Silence leaves the loop open forever and the customer checking.

How is closing the loop different from announcing a feature? An announcement goes to everyone. Closing the loop is a reply to the people who asked, on the channel they asked through. Do both; they are different messages to different readers.

What if the requester is not on GitHub? Most are not, and that is fine. The widget keeps showing them the status of what they sent, including the shipped entry and its link, so they need nothing beyond the page they wrote from. The comment on the issue is for the people who can see the repository.

Does this loop work on GitLab or Bitbucket instead of GitHub? The widget and the changelog do; the automatic comment in step five does not, today. A team on GitLab or Bitbucket still gets every submission, still files it as an issue, and still shows the requester a status in the widget, but closing that specific loop back onto the issue itself is a step you do by hand until that integration exists.

The technical claims in this article have not been independently reviewed. If something here is wrong, tell us and we will correct it.

Related on changeloop: Developer docs, Changelog examples