Perfecto Mobile released the Mobile Test Coverage Index, a data-driven benchmark to help organizations determine the appropriate mix of mobile devices, operating systems and form factors to test their apps and responsive websites and ensure optimal app quality for end users.
Today’s mobile market is extremely fragmented and constantly changing, challenging organizations with device mix obstacles, future releases, and the pressure to deliver flawless customer engagement. Currently, most enterprises have no standard approach for testing mobile apps and responsive websites, and the business impact that non-tested or poorly tested apps have is devastating to a brand. To help begin this testing process to ensure continuous quality and develop a comprehensive mobile strategy, enterprises need to define a set of mobile devices to test apps and responsive websites against, a strategy that best represents their target market and end users. This should be part of a larger definition of the user profile, such as the environment that the app is operating in (network conditions, GPS location, etc.). To answer the vital question “what devices should I begin testing on,” the industry benchmark report provides crucial information and insight into market trends and the correct device mix to ensure coverage.
Perfecto Mobile’s Quarterly Mobile Test Coverage Index aggregates 360,000 hours of exclusive Perfecto Mobile customer usage data, market data, and future device release timelines to create an extensive list of the 32 most relevant devices. It serves as a recommended starting point for organizations beginning the test coverage process.
“The first question both business and technical executives alike ask when building a mobile testing strategy relates to coverage. A relevant device and environment mix offers strong test coverage, getting the organization much closer to the true end user experience,” said Roi Carmel, SVP of Products and Strategy, Perfecto Mobile. “It is challenging to achieve desired test coverage for mobile end-user profiles in today’s rapidly expanding ecosystem, especially since a mobile device’s shelf-life is around 9 months. Our goal with the quarterly Mobile Test Coverage Index is simple: Help enterprises better understand their customers and automate app testing to deliver a high-quality digital experience.”
The report provides an index for reaching 30 percent, 50 percent and 80 percent test coverage in North America, France, Germany, Italy, Spain and the UK. The device selections address five key sectors: device model, screen size, GA date, hardware system and recommended operating systems. The number of devices in each test bracket may vary from quarter to quarter to reflect the changes in the mobile market. The report will be continuously updated to meet these market changes.
Organizations can use the index as a benchmark to take advantage of the three levels of mobile market coverage and scale as their needs grow.
Key takeaways from the research include:
- 30 percent of the market can be covered with 10-16 devices, based on market adoption, market device leaders, reference devices and device characteristics.
- 50 percent market coverage can be covered with 17-25 devices. This level of test coverage yields an increased number of devices from manufacturers such as Sony, LG, Samsung, Apple, HTC and more.
- 80 percent market coverage can be achieved with an index of 26-32 devices. This is the final group of devices to test on that will guarantee the most recommended coverage.
- While Apple pushes its users to upgrade to the latest iOS version, there are earlier iOS 8 versions that should be included in test plans. Additionally, iOS 7 still accounts for 17 percent of total users, and should be included at the same level of iOS 8 test plans.
- Including the most widely adopted Android versions KitKat and Jelly Bean is important to an Android test plan, Lollipop proves to be very important as well. Lollipop, the latest OS from Google, is running on the Google Nexus devices (considered reference devices).
This report analyzed and aggregated a wide range of public, private and proprietary data from 4,000+ device profiles and 360,000+ user hours across U.S. and EU markets, which includes Spain, UK, France, Italy and Germany.
The Latest
In the heat of the holiday online shopping rush, retailers face persistent challenges such as increased web traffic or cyber threats that can lead to high-impact outages. With profit margins under high pressure, retailers are prioritizing strategic investments to help drive business value while improving the customer experience ...
In a fast-paced industry where customer service is a priority, the opportunity to use AI to personalize products and services, revolutionize delivery channels, and effectively manage peaks in demand such as Black Friday and Cyber Monday are vast. By leveraging AI to streamline demand forecasting, optimize inventory, personalize customer interactions, and adjust pricing, retailers can have a better handle on these stress points, and deliver a seamless digital experience ...
Broad proliferation of cloud infrastructure combined with continued support for remote workers is driving increased complexity and visibility challenges for network operations teams, according to new research conducted by Dimensional Research and sponsored by Broadcom ...
New research from ServiceNow and ThoughtLab reveals that less than 30% of banks feel their transformation efforts are meeting evolving customer digital needs. Additionally, 52% say they must revamp their strategy to counter competition from outside the sector. Adapting to these challenges isn't just about staying competitive — it's about staying in business ...
Leaders in the financial services sector are bullish on AI, with 95% of business and IT decision makers saying that AI is a top C-Suite priority, and 96% of respondents believing it provides their business a competitive advantage, according to Riverbed's Global AI and Digital Experience Survey ...
SLOs have long been a staple for DevOps teams to monitor the health of their applications and infrastructure ... Now, as digital trends have shifted, more and more teams are looking to adapt this model for the mobile environment. This, however, is not without its challenges ...
Modernizing IT infrastructure has become essential for organizations striving to remain competitive. This modernization extends beyond merely upgrading hardware or software; it involves strategically leveraging new technologies like AI and cloud computing to enhance operational efficiency, increase data accessibility, and improve the end-user experience ...
AI sure grew fast in popularity, but are AI apps any good? ... If companies are going to keep integrating AI applications into their tech stack at the rate they are, then they need to be aware of AI's limitations. More importantly, they need to evolve their testing regiment ...
If you were lucky, you found out about the massive CrowdStrike/Microsoft outage last July by reading about it over coffee. Those less fortunate were awoken hours earlier by frantic calls from work ... Whether you were directly affected or not, there's an important lesson: all organizations should be conducting in-depth reviews of testing and change management ...
In MEAN TIME TO INSIGHT Episode 11, Shamus McGillicuddy, VP of Research, Network Infrastructure and Operations, at EMA discusses Secure Access Service Edge (SASE) ...
