Case Study: How MyBluebird Apps Improves Its Registration Success Rate
Bluebird has a long history to offer transportation solutions in Indonesia. The organization is famous for its trusted taxi service, which is currently transforming its value into mobility solutions with multiservice, multichannel, and multi-payment options. One of the company’s portfolio highlights is its mobile apps, MyBluebird (MyBB Apps). This platform offers online ride-hailing, premium rental, delivery service, and soon intercity travel on its platform. From a product and data point of view, this consumer-based platform produced quite interesting data to “play with”, which is still untapped. Our team realizes that building a strong cross-functional relationship in reliable data can empower product leaders to deliver bigger business impact; thus we start its journey by building an experimenting culture in our product management. Not only we harvest the power of data analytics into our product development process, but also we disrupt the way we gather our product requirements, from business needs/requests to product discovery based on user interactions. Below is the elaboration of its practical approach to improving registration success rate on MyBluebird acquisition funnel.
Part I Situation
MyBluebird becomes one of the key products enabling the company’s core services on mobility solutions. It’s a mobile app-based service that provides online taxi ride-hailing, renting and delivery services in Indonesia. Despite covid-19 situation having a notable impact on the business, we noticed that MyBluebird onboarding key metrics are not growing. The highest number of users registered happened on May 21, yet the trends are declining within the last 3 months, and hit its lowest number on Aug 21. Among hundred thousand acquired users (from both new and reinstall), on average only ~33% are successfully registered on MyBluebird apps. Although there’s an increase in #user acquired users, the trends in success rate have a constant gap (~65% to 70%). This situation is concerning since registration and onboarding are one of the key metrics in the acquisition funnel.
Part II Action
Improving the acquisition funnel is not solely impacted by product features and, but also by digital marketing activities including campaigns. In this experiment, we will only focus on improving registration success rates from the product side by re-engineering its onboarding process. This is how we solve the riddle.
#Key no 1. Bring close collaboration to the table within Product, Data, Tech and UI/UX Teams
Interestingly, our organization at that time was in the middle of a transformation in which each function was still finding its equilibrium to work in harmony. We noticed that the key to pulling these initiatives and building a solid foundation on product experimentation was to bring close collaboration between the product, tech, data, and product design teams. Thus, we embraced a co-design approach led by the product team so that each function was on the same page for the goals that we were trying to achieve.
Co-design : *) Co-design: is a process where design professionals empower, encourage, and guide users to develop solutions for themselves.
#Key no 2. Experimental approaches with 3 execution steps: data gathering, benchmark, and validation steps.
Step 1: Data gathering
In this step, we evaluate our onboarding flow from multiple PoV. Implementing appropriate trackers to enable data gathering from existing performance and user experience which includes success & failure rate by both events & #users through product flow. Below is the current Registration Flow :
With the current design, each user needs to “remember” whether they are registered users or sign up for new ones. When the user chose the registration button, it was pretty much shown as a direct form to fill in and then followed by verification steps. Moreover, we run current Onboarding-Funnel Analysis, to evaluate drop-off rate from the steps. The evaluation was done with 2 PoV Registration Events (attempt) and Registered #Users. Please note that in this case study, all data presented will be evaluated only by number of#registered users.
Following up no #3 highlight regarding the “intimidating registration” process, we zoomed in the funnel between step 2 Click_create_new_acc and step 3 click_button_register to discover drop-off reasons for the step. Among users from the previous funnel (step2), 35 % of registration failed from the email used, 28% of Registration failed as a result of incorrect email, 18% of registration failed from the phone number used, and the rest 19% fell under other drop-off reasons such as an invalid password or even didn’t feel encouraged to press registration button at all. These findings give us a baseline hypothesis for what we should do differently as a proposed solution.
Step 2: Benchmark
Push forward, we also did a competitive analysis to learn how our competitors handle their product flow and user interactions including crafting both error responses as well as CTA (Call to Action).
We picked 2 supper apps (Grab and Gojek) as our benchmark with consideration that both platforms offer mobility solutions for ride-hailing services in the same market as MyBluebird Apps, especially for Ride and Delivery services. We defined comparison points as follows.
We found that each platform has its own unique approach to handling its acquisition process, while Grab is more emphasis on first validation using OTP and breaks its registration process into several steps, yet Gojek reduces its experience stage to be pretty straightforward in one registration page.
Step 3 Validation
During the validation phase, we design experimental plans including which platform, duration, and defined rules. This experiment was run on both platforms (Android and IoS) within a 1-month experimenting period in which we closely monitored its daily performance. The experimenting period may vary and needs to be defined case-by-case. The longer we run the experiment, the more confidence in the results will increase. Keep in mind there’s a trade-off. Experiments are time-consuming, so we need to manage it doesn’t take longer than necessary. Furthermore, learning from the discovery process, both the data gathering and benchmarking process, our team decided to execute 5 solutions to answer the design challenge. Here are what we do differently to improve the registration success rate on MyBluebird Apps :
1# Re-engineer current registration process to be more efficient
2# Redesign registration form to encourage faster validation
3# UX improvement with onboarding Information and automation
4# Clear error message and error prevention
Other than the changes we made to the product flow, we also “disrupted” existing product development approaches in our organization; changing how our team’s mindset and how they collaborate to work on new ideas or manage the product itself. Considering this approach is new to our organization, our objective was not only to improve acquisition funnels but also to put a good baseline for a data-driven product management approach.
5# Harvesting “data analytics” power into product management
In the delivery phase, we decided to execute it with a split-testing approach (commonly A/B testing). It’s a live experiment conducted on the MyBluebird user base with a random sample of users seeing one version (variant) and the other seeing another variant. By using this approach, we can be sure that any differences between the groups are due to the product changes we made, instead of launching a change to all users and trying to compare its performance. This approach also helps to reduce the risk for resource investment and eliminates guesswork with comparable results to inform certain product decisions.
Part III Result
Defining what success looks like is essential to evaluate how successful a product/feature launch has been and whether you need to adapt your product strategy. As for this experiment, we defined that registration success rate was the key metric that we were overseen.
The trends continue within the next month: the registration success rate improves from an average of ~30% before experimentation to ~70% after post-implementation.
Apart from registration success rate, we also run a funnel analysis to review the drop-off rate from each funnel.
Zooming in into the funnel between step 2 Click_create_new_acc and step 3 click_button_register to discover drop-off reasons for the step, Fig. 5. We noticed that changes on the registration form (both product flow and overall user experience) contribute to a higher succession rate. Moreover, existing and new users are validated in the first place, as seen in Fig. 7, which gives a more precise product flow to each user.
We also noticed that changes we made to product flow have an impact not only on the registration and onboarding process but also on login succession rate, in which variant B (new) gives a higher succession rate up to 15%, compared with variant A (existing).
Although we brought higher traffic to the platform, interestingly, we also discovered potential hidden costs from our current design. As you may notice, from the current design as seen in Fig. 18, when users register, log in, or just open the apps, they will be redirected to the homepage with a map that automatically recognizes the user’s current location. Such product behavior requires certain API calls which become a hidden cost when trying to bring more traffic to the platform if the product also doesn’t improve its conversion rate. This finding will become a further product improvement to craft a better user experience and improve its conversion rates (i.e. improve service visibility, reduce unnecessary product steps, or offer quick actions with related recommendations).
Last but not least, what our team was doing with these experiments was just the beginning for many to come. The success ingredient to pull out this initiative not only comes from team collaboration but also openness to embracing an experimenting culture within the product development process.
Special mention to those who made this initiative possible. My partner in crime Maulana Bhara from the data team, to our mentors Pak Ifnu Bima and Pak Ferrizal Azwar to tech team and engineering leader Fikry Al Farisi Muslim, Alfian Maulana Malik, Risman Zainuri, Wira Setiawan, Hamzah Tossaro, Fahreza Fauzi and UI/UX team Sindy Larasati.
