Crowd Testing for Mobile App Helps E-commerce Leader Gain Competitive Edge

The success of e-tail businesses depends on the seamless performance of their e-commerce platforms. A global e-commerce leader was struggling to achieve the Quality of Service (QoS) benchmark and wanted to enhance its app’s performance and functionality for better customer experience. Know how Cigniti utilized crowdsourced testing to benchmark the app quality while reducing their costs.

The Challenge

A UK based global e-commerce leader offers a global electronic platform to its consumers that lets them choose and buy from a wide range of consumer goods using a mobile app. The mobile app was very popular and was being installed in large numbers by consumers across the globe using multiple devices and browsers. This made it critical for the organization to provide the best customer experience in terms of functionality and performance of the app.

The organization was facing challenges in having its mobile app get benchmarked for Quality of Service (QoS). It wanted to baseline its app against that of its competitors in-terms of functionality and UX, as also validate the behavior and performance of the app across multiple browsers, carriers, & devices spread across geo-locations having varied signal strength of the mobile carriers. Realizing that the app needed extensive concurrent testing across timezones, the organization conducted a thorough research, and finally reached out to Ismart for its expertise in Crowdsourced Testing.

How we utilized crowdsourced testing to benchmark the quality of the app

Based on the detailed discussions with the client, we approached our crowdsourcing partners to identify testers spread globally who possessed different mobile variants (2G and 3G), different OS variants across Android and IOS, and used different carrier networks.

Our team of specialized crowdsourced testers traveled across the globe and positioned themselves at different geo-locations covering different time zones, and validated the application end-to-end, including its payments functionality. We also positioned our team on the client premises round the clock to monitor, support and log defects/issues reported by the testers across the locations. Additionally, data related to competitors apps was collected and analyzed internally to provide benchmarking.


Eden Office Park, 64 Macrae Road, Pill, Bristol


We also helped lower costs & proactively mitigate risks

While the benchmarking activity helped the client to validate future enhancements & releases and adopt corrective measures during the early QA phases, by implementing crowdsourcing, the organization was also able to cut down its cost of mobiles and mobile carriers.

The results for each ISP and Mobile carrier were shared with the client in the form of an exhaustive report that covered different combinations of mobile OS versions and mobile carriers. The client leveraged these reports to proactively suggest changes to prevent and mitigate risks or potential issues.