Green CI runs can create a false sense of completeness. An Android build may pass automated regression, open cleanly on an emulator, and still behave differently once it reaches a phone with a manufacturer skin, limited memory, an aging battery, or an unstable mobile connection.
That gap changed the way we think about mobile QA. We still rely on emulators every day, but we no longer treat them as a stand-in for the devices our clients’ customers actually use. Real hardware now has a defined place in the release process, especially around cameras, background execution, network changes, memory pressure, and device-specific UI behaviour.
Why our emulator-first pipeline looked fine
There was nothing unreasonable about relying heavily on emulators. They start quickly, reset cleanly, fit neatly into CI, and let several test jobs run in parallel without maintaining shelves of hardware.
They are predictable too. A clean Android image does not arrive with an OEM battery manager, years of updates, carrier settings, or a collection of background apps fighting for memory. When the job is to isolate a regression, that consistency is useful.
The problem starts when a controlled environment becomes evidence that the release is ready for every environment. An emulator can answer “does this build work here?” remarkably well. It is much less reliable at answering what will happen on the phones people already have in their pockets.
The bugs that only showed up in users’ hands
Some of the hardest mobile defects are not complicated coding mistakes. They appear where the application meets the hardware and operating system around it.
Background work is a familiar example. A sync task or push flow can behave normally on a clean test image and become unreliable on a handset whose manufacturer applies more aggressive battery restrictions. Nothing has necessarily changed in the application code, yet the user sees late or missing updates.
Camera features expose a different gap. A barcode or document scanner tested against a simulated feed does not reproduce autofocus hunting, glare, movement, poor indoor light, or the quirks of a specific camera stack. A flow that appears finished at a desk may behave very differently when somebody points a real phone at a receipt under supermarket lighting.
Network transitions can cause equally awkward failures. A development machine normally gives the emulator a stable connection. Users leave Wi-Fi coverage, switch to LTE or 5G, lose packets in lifts, and reconnect halfway through a request. Poor state handling can turn that moment into a failed submission or duplicate action.
| Bug class | What real hardware exposes | Possible client impact |
| Background sync and push | OEM battery rules | Missed notifications and support tickets |
| Camera and scanning | Autofocus, lighting and camera hardware | Broken scanning flows |
| Network changes | Real handoffs and latency | Failed or repeated requests |
| App restoration | Memory pressure and process death | Lost form state |
| UI compatibility | Cutouts, keyboards and unusual screens | Covered controls and broken layouts |
What do those bugs cost our clients
The cost is rarely just the hours needed to patch the code.
A production defect interrupts the sprint that was supposed to deliver something else. QA has to reproduce the issue on the right configuration, engineering has to prepare a fix, and the release process starts moving again. Support may meanwhile be collecting reports from devices the team cannot reproduce immediately.
There can also be pressure to slow a rollout or delay entry into a region where the device mix is different. Negative store feedback can arrive long before the investigation is finished.
We do not attach invented dollar figures to those situations. The cost is already visible in diverted engineering time, delayed planned work, support load, and the additional release cycle needed to repair something that would have been cheaper to find before launch.
What we changed in our QA process
We did not replace emulators with a room full of phones. We narrowed the job each testing environment is expected to do.
- Build the device matrix from client analytics: common models, Android versions, manufacturers, screen sizes, and relevant hardware.
- Keep emulators in CI for fast regression, repeatable UI checks, and early debugging.
- Run release candidates on selected physical devices that represent the actual user base.
- Test interruptions deliberately: backgrounding, network changes, low battery, system prompts, and returning after a long pause.
- Use staged releases and crash monitoring so one device-specific issue does not immediately reach the whole audience.
- Log the device model, OS version, network conditions, and reproduction steps with hardware-dependent bugs.
Getting real hardware without building a warehouse
There are several practical ways to expand physical-device coverage.
An in-house lab gives the team full control and makes repeated debugging easy. The downside is obvious over time: phones age, batteries deteriorate, operating systems need maintenance, and yesterday’s useful device collection gradually stops reflecting the market.
Cloud device farms are easier to scale and work well for automation. They can give a team access to many models without physically storing them, although available configurations and network conditions depend on the provider.
A third model is temporary remote access to consumer hardware hosted by individual owners. DroidDesk uses this approach: owners make money with their phone by keeping an eligible spare handset online at home and renting out remote access to it, while teams control the device from a browser by the hour. For a QA team, that creates another route to a physical model without purchasing a device that may only be needed for a short testing window.
None of these approaches needs to replace the others. An office phone may be best for repeated debugging, a device cloud for broad automated coverage, and temporary access for an uncommon model that appears in client analytics but is not worth buying.
Final takeaway
We did not stop using emulators. We stopped asking them to prove things they were never designed to prove.
They remain fast and useful for controlled, repeatable testing. Physical devices answer a different set of questions about memory, manufacturer software, cameras, networks, screens, and interruptions.
For teams building a real-device QA strategy, the first device matrix should come from actual client analytics rather than a list of the newest flagship phones. Test where the users are, not where the specification sheet looks best.
- How to Restart VS Code Without Losing Your Work - November 5, 2026
- QA on Real Devices: What We Changed After Shipping Mobile Apps for Clients - November 5, 2026
- How Businesses Can Build a Smarter and More Secure File Storage Strategy - November 4, 2026
