AppDynamics announced the debut of the AppDynamics/Mobile Strategies 360 Performance Index, a weekly snapshot of the performance of 30 top mobile websites.

The index will be published weekly in Mobile Strategies 360, the new digital publication introduced by Vertical Web Media, the publisher of Internet Retailer magazine. The new mobile performance index and Mobile Strategies 360 both made their debut on June 3.

The AppDynamics/Mobile Strategies 360 Performance Index provides the market with a powerful new ‘real world’ benchmark of mobile site performance, and shines a bright light on current issues, challenges and best practices in mobile performance today. The new index tracks the mobile performance week over week of leading sites in a cross-section of industries including retail, consumer products, government, financial, healthcare, fashion, food and more, from weekly measurements taken with the AppDynamics Synthetic Browser Monitoring solution, currently in beta. Critical mobile performance metrics including time to first render, time to visually complete, number of elements loaded, total page weight, and an overall “Speed Score” provides the market with a comprehensive and extremely useful view on how well major mobile websites perform relative to each other, and on mobile performance as a whole. Results for both 3G and 4G networks are included in the new index.

“The new AppDynamics/Mobile Strategies 360 Performance Index is a marquee weekly feature for Mobile Strategies 360, and a valuable source of data for our audience of mobile industry professionals,” says Katie Evans, Editor of Mobile Strategies 360. “Nowhere else will readers find such a useful benchmark for mobile performance, updated every single week. Enterprises and their mobile site owners will have a clear and ongoing picture of how the best sites perform, and where the bar should be set relative to the performance of their own sites. This is an extremely beneficial index that AppDynamics and Mobile Strategies 360 are providing to the mobile web community.”

“Speed Score” Rates Real User Experience

The multiple metrics gathered weekly are crunched into a “Speed Score” for each site, ranking the perceived speed and quality of experience for the end user. For example, site A may have a “visually complete” time of three seconds, loading 90 percent of its content in the first second and the remaining 10 percent in seconds two and three. Site B may have the same three-second visually complete time, but present zero percent of its content in the first two seconds and 100 percent in the third second. The sites have identical visually complete times, but Site A will clearly be perceived as much faster, and will earn a better Speed Score as a result.

“Most mobile quality rankings present total time to load, a typical synthetic measurement,” says Ian Withrow, Group Product Manager at AppDynamics. “But total time to load is not necessarily what you and I as users see. The Speed Score in the AppDynamics mobile index is immensely helpful for understanding what the real user experience of a site is relative to the others in the index. Sites get points for loading more content faster.”

The AppDynamics/Mobile Strategies 360 Performance Index is built using open-source WebPageTest technology, which is considered by many to be the best real browser measurement technology available; until now, though, it has not been widely accessed as a source of data outside the hardcore performance community. The AppDynamics index makes this best-in-class data available every week in a form that is useful to marketers, executive management, and others beyond the IT department.

“Following on the heels of the recent Google’s search algorithm, nicknamed the ‘mobilegeddon,’ the AppDynamics/Mobile Strategies 360 Performance Index is going to be especially useful to site owners,” says Peter Kacandes, Senior Product Marketing Manager, Mobile Application Management and Monitoring for AppDynamics. “With Google now factoring mobile site speed into its search algorithm, it’s not inconceivable that a site could drop significantly in search results if it is not built to be mobile-friendly, putting in jeopardy all the time and money spent on search engine optimization.”

Data-Defined Best Practices

The weekly flow of accurate performance data generated by the index is expected to help define best practices for mobile websites as a whole. Data that AppDynamics gathered in the course of beta testing the index indicate that, on average, each megabyte of page size adds 6.8 seconds of page load time over 4G speeds and 9.87 seconds over 3G, as seen by the user. But how the page is architected can have a tremendous impact on how much time it takes that megabyte to load, with the best sites taking as little as three seconds and the slowest as long as 18 seconds.

“A site with ten 100-k objects is not the same as a site with one 1,000-k object,” Withrow says. “And that is factored into the Speed Score. But as a general best practice, if you want a 4G load time of three seconds, your page can’t be much bigger than one megabyte. And if you’re just starting out, you should aim for half of that.”

More insights on mobile best practices and performance trends will be reported in editorial accompanying the index each week.