Plenty of Play Store listings get changed because someone on the team likes the new screenshots better. Store listing experiments, the Play Console feature most people mean by Google Play A/B testing, let you check that instinct against real visitors. Published apps can use it for free.
Play Console splits listing visitors between your current listing and the variants, then reports which version did better on a target metric such as unique user install clicks. The icon, feature graphic and screenshots can all be tested, and so can descriptions if you run a localized experiment.
Older guides get one detail wrong. Play Console Help caps each experiment at 2 variants against the current listing (as of November 2026), one fewer than guides still repeat.
What is Google Play A/B testing?
Play Console shows each store listing visitor either your current listing or an experimental variant, then measures which version earns more of the target metric you chose. Google’s name for it is store listing experiments, and it describes them as a free way to test store listing text and graphics.
The feature used to carry a different label. Global experiments were renamed default graphics experiments and kept the same functionality, according to Play Console Help.
Testing starts from a published app. When a listing is not fully rolled out, Play Console asks you to finish the rollout before it allows a test.
Custom store listings get mistaken for experiments a lot. They are separate pages for audience segments, up to 50 per app (Play Console Help), and experiments can run on them, but they aren’t experiments themselves. Price experiments are another Play Console tool, built for testing in-app product prices across markets. Firebase A/B Testing tests changes inside the app and never touches the store page.
Mixing these up wastes setup time. Only store listing experiments split live store traffic between versions of the listing.
Default graphics or localized: which experiment type fits?
Default graphics experiments cover the icon, feature graphic and screenshots in your default listing language. Localized experiments cover those same graphics plus descriptions, in up to five languages you pick.
Each app runs one default graphics experiment or up to five localized experiments at the same time, according to Play Console Help (as of November 2026).
Default graphics experiments
You can swap the icon, the feature graphic or the screenshots here.
On a single-language listing, every visitor can land in the experiment. Add localized graphics in another language and that language drops out.
Google’s own example: a French feature graphic on an English-default listing excludes French viewers from the default experiment, even when the icon is the asset under test.
Localized experiments
Pick the languages first. Variants appear only to visitors viewing the listing in those languages.
Descriptions are testable here and nowhere else. If the listing exists in one language only, a localized experiment reaches just the people viewing it in that default language, so it behaves like a default test with text added.
How to choose between them
- Graphics only on a single-language listing? Go with default graphics.
- Any description change needs a localized experiment.
- Graphics for one market or a few also fit localized, with one language set per experiment.
- If localized graphics are already live in several languages, localized again, since the default test skips those visitors.
A localized test on each priority language beats a default test whose audience quietly shrinks. Run the default type only when your default language carries most of your traffic.
What can you test in a Google Play experiment?
Icon, feature graphic and screenshots can be tested in either experiment type, and descriptions only in a localized one. Google recommends changing one asset per experiment so any shift in results traces back to a single cause.
Graphics
SGN lifted Cookie Jam conversions by 8 percent by changing only the background color of its app icon (Android Developers Blog, 2015).
Small change, measurable result. That is the argument for isolating one asset at a time.
Google’s Play Console page names icons, videos and screenshots as the tests with the biggest impact, and tells developers to revisit assets as audiences, locations and seasons change.
Descriptions
Descriptions take part in localized experiments only. Play Console Help names descriptions in general, and the asset selector inside your own console shows what a test accepts.
Guides split on the long description. Gummicube’s 2020 guide says both descriptions are testable in localized experiments, while MWM’s glossary says the long description cannot be tested.
Promo video and other disputed elements
Several claims in popular guides don’t match the documentation. The table sets them side by side.
| Claim in guides | Where it appears | Play Console Help, November 2026 |
|---|---|---|
| Up to 3 variants | Asomobile guide, December 2025 | Up to 2 variants |
| Long description cannot be tested | MWM glossary | Descriptions testable, localized experiments only |
| Promo video is testable | MWM glossary | Not named in either asset list |
| Results use a fixed 90% confidence interval | Gummicube guide, 2020 | Confidence level is a setting you choose |
The promo video row is the messiest. Google’s marketing page for store listing experiments names videos as a high-impact test, yet the Help page lists no video in either experiment type.
Where official pages conflict, the Help page and your own console win.
How to set up an experiment in Play Console
The path through Play Console differs from older guides. It now runs through Grow users, Store presence, Store listings (Play Console Help, November 2026).
- Open Play Console, pick the app, and go to Grow users, then Store presence, then Store listings.
- Click Set up in the Experiment column of the listing you want to test. Alternatively, open its Details page and click Set up experiment.
- Name the experiment in 50 characters or fewer. Users never see the name.
- Choose the experiment type: graphics on the default listing, or a localized listing with its languages selected.
- Pick the target metric: unique user install clicks, unique user open clicks or unique user pre-registration clicks.
- Review the advanced settings. Change them only for a reason, because the defaults give the fastest results.
- Select the one asset type to test and upload the variants.
- Review the variants, save, then submit the changes for review from the publishing overview page.
The creation page estimates the time and the number of installs, opens or pre-registrations a statistically significant result needs. Read that estimate before launching, not after.
If the Set up experiment button is missing, Play Console wants the listing fully rolled out before it allows a test.
Running experiments, and their results, live in the Experiments tab on the same page.
How to choose variants, audience share, minimum detectable effect and confidence level
Leave the advanced settings alone unless traffic is high. Google says the defaults deliver results in the shortest time with reasonable accuracy, and any change can lengthen the run before results become useful.
Variants and audience share
You can run up to 2 variants against the current listing, and fewer variants finish sooner. Audience share is the percentage of visitors who see a variant instead of the current listing, split equally across variants.
Phiture’s April 2022 analysis favors one control and one test variant, or an A/B/B layout, because extra variants stretch the run and cut reliability. Equal splits also keep the math simple.
Minimum detectable effect
Phiture’s worked example: a 20% baseline conversion rate with a 10% relative minimum detectable effect flags only moves outside 18% to 22% (Phiture, 2022).
Google defines the setting as the smallest difference between a variant and the control that counts as a result. Anything below it ends as a draw.
A smaller value needs more visitors, and a low baseline conversion rate needs more still, per Phiture. My rule of thumb is to set it to the smallest lift worth shipping, not the lift you hope to see.
Confidence level
Raising the confidence level lowers the chance of a false positive and adds runtime. At a 90% confidence level the significance level is 10%, so roughly one in ten experiments can still report a false positive (Phiture, 2022).
The console offers a range from 90% to 99%, according to Yellowhead’s 2022 write-up. Phiture suggests a level above 90% paired with 80% power.
Google ties confidence to false positives, and Phiture ties minimum detectable effect to sample size. Both settings draw on the same pool of traffic, and so does the variant count.
Spend it on confidence when the winner goes live on a high-traffic listing. Keep 90% for tests you can reverse.
Target metric
Unique user install clicks counts installers who did not already have the app on another device. Unique user open clicks counts installed users who opened the app, and unique user pre-registration clicks counts the users who pre-registered.
Google’s marketing page still describes acquisition and 1-day retention rates per listing version, but the Help page names only these click-based metrics. Plan around the Help page.
How long does a Google Play experiment need to run?
Plan for at least one week, and longer when traffic is thin. Google’s Play Console page recommends a full week to cover the difference between weekday and weekend visitors.
Traffic volume sets the rest. Experiments stop by themselves after 6 months, apply no variant and return all traffic to the current listing (Play Console Help, as of November 2026).
Asomobile puts the typical completion window at 7 to 14 days (Asomobile, December 2025). Phiture’s numbers show how wide the spread gets: a calculator estimate of 22 days for a market with 19% conversion and 600 daily visitors, against about 14 days observed at 20% conversion and 1,000 daily visitors (Phiture, 2022).
Guides and Google differ on the minimum. Asomobile points out that Google Play sets no formal minimum duration and ends a test once significance arrives, while Google itself advises a week (Google Play Console).
Both hold. The tool enforces nothing, and the recommendation still stands.
External calculators make poor planning tools here. Phiture’s 2022 review found two popular sample size and duration calculators inconsistent predictors for Google Play experiments, and expected the built-in estimate to replace them.
Reading results and applying a winner
Play Console labels each experiment with a result state and adds a recommended action when one applies. Every result carries an explanation based on its confidence interval and minimum detectable effect.
Reading the result states
- When one variant performed better, Play Console can recommend applying it.
- Several better variants, or a draw, leave the decision to you after you review the results.
- “More data needed” means come back later. Google flags it when less than a week remains before a result is expected.
- If the current listing performed best, click Keep current listing.
A draw is not a failed test. It tells you the change was smaller than the detectable effect you set.
Choosing the action
A declared winner is not final.
Phiture reported a test that showed a winner, so the team waited a few more days. The result slipped back to needing more data and then pointed the opposite way.
Their rule from the same 2022 analysis: let a short-lived positive result run a few more days, apply it, then run a reverse (B/A) test to measure the conversion change.
Turn on notifications before launch. Google’s Play Console page recommends opting in to email alerts that announce a declared winner, so a finished experiment never sits unread.
On a draw, rerun with a bolder change. Phiture favors large differences in graphics over minor tweaks.
When Google Play A/B testing does not work
Store listing experiments can’t answer every testing question, and a few situations make them worse than useless.
Audience volume has to stay stable during a test, according to Phiture’s 2022 analysis, so a paid campaign that starts or stops mid-test muddies the data. Campaigns in the tested market can add sample size, but only when they run steadily.
Questions that live inside the app sit beyond the store page. Onboarding screens, in-app UI and feature changes belong to Firebase A/B Testing, and Play’s price experiments cover in-app product prices.
Wide concept sweeps are another weak spot. Phiture’s 2022 advice reserves A/B/C/D tests for early concept testing, but the console’s variant cap sits below that. Sweeps take sequential rounds or a third-party platform.
Then there’s the temptation to leave a losing variant running. Phiture advises stopping an experiment whose first 3 days look off, to avoid a drop in conversion rate. Early reads justify stopping a loser, never applying a winner.
How Google Play A/B testing compares with other testing options
Store listing experiments suit changes to the live Google Play listing. The other tools answer different questions, like who sees which page, what happens inside the app, how the iOS page performs and what works before launch.
| Option | What it covers | Where it runs | Fits when |
|---|---|---|---|
| Store listing experiments | Icon, feature graphic, screenshots, descriptions | Play Console | Changing the live Google Play listing |
| Custom store listings | Separate pages per audience segment | Play Console | Targeting countries, keywords or campaigns |
| Apple Product Page Optimization | Icon, screenshots, app previews | App Store Connect | Testing the iOS product page |
| Firebase A/B Testing | App UI, features, notification wording | Firebase console | Changing in-app behavior |
| Third-party platforms | Simulated store pages | External tool | Pre-launch checks, wide concept sweeps |
Experiments split one audience across versions of a listing. Custom store listings do the opposite and send different audiences to different listings, targeting segments such as countries, pre-registered users, search keywords, lapsed users or ads traffic.
A country can belong to only one custom listing at a time (Play Console Help, as of November 2026).
Apple’s version gives one more treatment than Play, but no concurrency. It allows up to three treatments with alternate icons, screenshots and app previews, runs one test at a time for up to 90 days (Play allows up to five localized experiments at once), and locks a test once it starts (Apple Developer, as of November 2026).
Apple treatments can go to review without a new app version, but any alternate icons must already be in the app binary live on the App Store.
Firebase A/B Testing tests app UI, features and engagement campaigns through Remote Config and Cloud Messaging, measured against metrics such as revenue and retention (Firebase documentation, November 2026). It never touches the store page.
Price experiments allow at most two price variants and a control per experiment (Play Console Help). Space Ape reported purchases up 20% to 40% by market after running them (Android Developers Blog, 2023).
SplitMetrics and StoreMaven test simulated store pages outside the store, which is where pre-launch checks and wide sweeps belong.
Start native for any change to the live Play listing. Step outside it only for work before launch or for sweeps wider than the console allows.
Google Play A/B testing FAQ
Do experiment variants go through review before they run?
Yes. After you save the variants, Play Console Help sends you to the publishing overview page to submit the changes for review.
Some guides claim traffic splits instantly with no review, which contradicts that step.
Is Google Play A/B testing free?
Yes, Google describes store listing experiments as free A/B tests on store listing text and graphics. Third-party platforms such as SplitMetrics and StoreMaven are separate products with their own pricing.
Can you A/B test in-app product prices?
Not with store listing experiments, which cover store assets only. In-app product prices belong to Play Console price experiments, which require a merchant developer account.
A repeat run on overlapping products and countries has to wait at least 30 days after the original ends (Play Console Help).
Does a description experiment change search rankings?
Play Console Help documents no ranking effect. Results are reported on the target metric, such as unique user install clicks.
Track keyword positions separately before and after applying a new description.
Which target metric fits an app in pre-registration?
Unique user pre-registration clicks. The creation page then estimates the pre-registrations a statistically significant result needs.
How do you get notified when an experiment finishes?
Set your preferences on the Email notifications page in Play Console. Google then emails you when an experiment completes.
Its Play Console page also recommends opting in to alerts that announce a declared winner.
What to Test After the First Experiment Ends
Google Play A/B testing pays off as a fixed sequence of single-asset experiments, ordered by how many visitors each asset reaches. Start with the icon in a default graphics experiment, move to screenshots and then the feature graphic in the same type, and leave descriptions for last, with a localized experiment per priority language.
Graphics go first because a default graphics experiment on a single-language listing reaches every visitor, while a description test reaches only the chosen languages.
Google’s one-asset recommendation and one-week minimum combine into a floor: with one default graphics experiment running at a time, a first pass over the 3 graphic assets takes at least 3 weeks.
That pace is the trade-off. Bundling assets finishes sooner and explains nothing.
Move on after consecutive draws on one asset. Verified against Play Console Help in November 2026; recheck variant limits, asset lists and target metrics first, since older guides already disagree with the documentation on all three.
- Go Cheat Sheet - November 9, 2026
- Build or White-Label Software? How Software Teams Decide What to Resell in 2026 - November 9, 2026
- Notepad++ Column Mode: Edit 100 Lines at the Same Time - November 7, 2026
