Feedback loop

Alpha vs beta testing: differences and when to run each

7 min read

Alpha testing is the first round of testing, done inside the company on a build that is still missing pieces and known to be unstable. Beta testing comes next: a build that is feature-complete, or close to it, goes to real users outside the team, who use it in their own environments and report what breaks or confuses them. Alpha answers whether the software works at all. Beta answers whether it works for the people it was built for, on their machines, with their data and their habits.

Alpha testingBeta testing
Who testsDevelopers, QA, other staffCustomers and volunteers outside the company
Build stateFeatures still being added, many known bugsFeature-complete or nearly so, bugs expected
EnvironmentLab, staging or the team’s own accountsReal devices, real data, real workflows
Main questionDoes it work as designed?Does it work for real users?
Typical feedbackPrecise bug reports with stepsSymptoms, confusion, missing features
What ends itThe build is complete and stable enough to show outsidersNo blockers left and feedback has slowed

What is alpha testing?

Alpha testing is internal testing of an early build. The people testing it know the product, can read logs, and accept that things will break, so the goal is to find the large problems before anyone outside sees them.

Wikipedia’s article on the software release life cycle describes the alpha phase as the first phase of software testing: developers test with white-box techniques, and a separate testing team then validates the build with black-box or gray-box techniques. Many teams add a third layer, which is staff using the product for their own work. That practice has its own name, dogfooding, and it finds the friction a scripted test never touches.

What is beta testing?

Beta testing puts a nearly finished build in front of real users before the general release. It catches the problems that only show up outside the building: other devices and browsers, messy data, unusual workflows and people who have never seen the product’s internals.

The same Wikipedia article says a beta phase generally begins when the software is feature-complete but likely to contain known or unknown bugs. A closed beta invites a chosen group, often customers who asked for the feature or design partners. An open beta lets anyone join. Some products stay in what is called a perpetual beta, updated continuously and never declared finished.

What is the difference between alpha and beta testing?

Four things change between the two phases: who tests, how finished the build is, where it runs and what happens to a report. Alpha testers are insiders working on an incomplete build in a controlled setup, while beta testers are outsiders using a complete build for real work.

The last difference matters most for the people who read the reports. An alpha report usually comes from someone who knows the code, names the component and includes steps. A beta report comes from someone who saw a symptom, and the same root cause can arrive as five different complaints. Beta feedback therefore needs triage before it reaches developers: separating bugs from requests, merging duplicates and asking follow-up questions while the tester still remembers what they did.

Should you run a closed or an open beta?

Start closed, and open the beta only when you need scale or variety you cannot recruit by hand. A closed group is easier to support and less likely to leak an unfinished feature.

The app stores build this sequence into their tooling. Google Play offers three testing tracks: internal testing for up to 100 testers, closed testing for a wider invited group, and open testing that anyone can join. New personal developer accounts created after November 13, 2023 must run a closed test with at least 12 testers, opted in continuously for at least 14 days, before they can apply for production access. Apple’s TestFlight allows up to 100 internal testers from the development team and up to 10,000 external testers, and the first build sent to external testers must be approved by App Review. All three figures were checked on 16 October 2026.

How do you version alpha and beta builds?

Give every build a version that says which stage it belongs to and sorts in release order. Pre-release suffixes do both.

Semantic Versioning marks a pre-release with a hyphen and dot-separated identifiers after the version you are working towards. The spec’s own precedence example runs 1.0.0-alpha < 1.0.0-alpha.1 < 1.0.0-alpha.beta < 1.0.0-beta < 1.0.0-beta.2 < 1.0.0-beta.11 < 1.0.0-rc.1 < 1.0.0, so the stages sort correctly without any special handling. Tag each build in git as described in git tags and releases, and when the beta ends, the next stage is a release candidate.

How do you collect and triage beta feedback?

Give testers one obvious place to report, inside the product if you can, and ask for what they did, what they expected and what happened. Then sort every report into bug, feature request or question, merge duplicates and reply to the tester.

Email threads and chat channels feel friendly at the start of a beta and become unsearchable within a week. A form or widget in the product captures the page, the version and the account without asking the tester to type them. The distinction between a feature request and a bug report is worth making at intake, because the two go to different owners, and duplicate feature requests need merging with their counts preserved, since ten testers asking for the same thing is a finding in itself. In Changeloop, a report sent through the feedback widget becomes a GitHub issue labelled bug or feature-request, plus a priority label and from-widget, so beta feedback lands in the same tracker as the work it describes.

When is a beta test finished?

A beta is finished when the remaining problems are known, none of them blocks the release, and new reports have slowed to a trickle. Write the exit criteria down before the beta starts, or it tends to end on the date someone promised a customer.

A workable set of exit criteria:

  • No open bug is rated a blocker.
  • Crash and error rates are at or below the level you set for the release.
  • New reports per day have fallen, and most of them repeat known issues.
  • The most requested changes are either done or explicitly deferred, with testers told which.
  • Support documentation and release notes are drafted from what testers actually struggled with.

When those hold, freeze features and cut the first release candidate. Bugs found from then on are fixed only if they block shipping.

How do you close the loop with beta testers?

Tell testers what happened to what they reported, in the same channel they used to report it. People who see their feedback turn into fixes keep testing, and people who hear nothing stop.

Send a short note with each new beta build listing the fixes that came from tester reports, and say plainly which requests will not make the release. When the final version ships, the release notes should credit the changes testers drove; the release notes template gives a structure for that. Changeloop does this per report: when a changelog entry linked to a widget issue is approved, the issue receives one comment saying the change shipped, with a link to the entry on the public changelog. The wider practice is covered in closing the customer feedback loop.

FAQ

Is beta testing the same as user acceptance testing? They overlap, and they answer different questions. User acceptance testing checks the software against agreed requirements, usually by the customer or business owner before sign-off, while beta testing observes how a wider group uses the product in practice.

Can a small team skip alpha testing? It can shorten it, and skipping it entirely is risky. Even a few days of the team using the build for real work, before anyone outside sees it, removes the crashes and broken flows that would otherwise waste testers’ first impressions.

What comes after beta testing? A release candidate, then general availability. The candidate is a feature-frozen build that ships unchanged unless testing finds a blocker, and the final release is that same build relabelled.

How many beta testers do you need? Enough to cover the platforms, account types and main workflows the product supports. Some stores also set a floor, such as the 12-tester, 14-day closed test Google Play requires of new personal developer accounts.


The technical claims in this article have not been independently reviewed. If something here is wrong, tell us and we will correct it.

Related on Changeloop: Developer docs, Release notes template

changeloop
The team building a closed-loop changelog. Your users ask, your team ships, the person who asked gets told.