“Mobilification” of customer satisfaction at Orbitz TMRE, October 2014 About Orbitz: A Global Travel Leader • One of the largest and most sophisticated e-commerce websites in the world, with a fully developed in-house global technology platform • Three main businesses: 1) Direct-to-consumer travel product booking, 2) Corporate managed travel solutions to businesses of all sizes including IBM, Burger King and GameStop, and 3) Private label technology and distribution services for some of the most recognizable brands in the world like American Express, Capital One and Southwest • Headquartered in Chicago, IL with major offices in the UK, France, Switzerland, Sweden, Hong Kong, Mexico and Australia, and advanced analytics and technology teams in Sunnyvale, CA and India 2 About Knowledge Wave International: • Boutique BI firm specializing in … ‒ Customer experience optimization ‒ Healthcare ‒ Mobile customer experience • Broad customer base – KWI supports more than 1mm C-SAT surveys monthly ‒ Analysis and reporting on structured and unstructured data 3 ‒ Social customer sentiment ‒ Significance analysis and predictive modeling Why adapt surveys for mobile? What are the trends in device ownership? Device Ownership among Online Travel Bookers 100% 80% 2012 2013 2014 60% 40% 20% 0% 5 PC Smartphone Source: Proprietary Orbitz Mobile Research, 2012-14 Tablet ereader How often are consumers booking hotels on a mobile device? 6 *Source: Internal Orbitz transactional data How has email readership changed due to this channel shift? 100% 80% Email Opens by Device Type 39% 27% 60% 40% 20% 26% 35% 0% December 2012 7 23% Source: Proprietary Orbitz data, 2012-14 50% June 2014 Desktop Webmail Mobile At Orbitz, we survey our customers at various touchpoints to understand opportunities for improvement How survey is delivered Site Experience Today’s focus Overall Customer Experience Customer Service 8 Booking Experience We have evolved our satisfaction program over time to comprehend these channel shifts… 2007 Begin post-service satisfaction tracking 2011 Begin post-sales satisfaction tracking Survey designed for big screen Survey designed for big screen Functional on small screen but not optimized 9 2013-14 Analyze effect of mobile adapted survey experience Test, validate, and migrate to mobileoptimized survey experience So we tested two versions – mobile-adapted and mobileoptimized and compared to the original desktop survey Device Detec(on Survey Rendering Desktop only; Not mobile adapted/ optimized Mobileadapted Mobileoptimized Assumes Desktop þ þ þ Desktop version only Touch-‐ friendly No Ques(on types 10 All types þ General by device type Responsive; Specific to individual device þ þ All types, just as desktop, but readability may be compromised Fewer and shorter question and answer text, fewer open-ends, less complex questions types Original survey renders fine on a desktop browser, but mobile-adapted can be hard to read Desktop 11 Mobile The mobile-optimized survey improves readability and ease of response 12 In our testing, we measured both response and survey variables to look for significant differences Did 50/50 split tests to compare results on: • Survey completion rates • Mix of mobile vs desktop completions • Survey answers like Net Promoter Score and ease of use • Profiles of responders …and now for the results! 13 So, what happened when we “mobilified” our surveys? How did completion rates change? 100% 95% Survey Completion Rates % completed out of all starts 90% 85% 80% 76% 82% 75% 70% 65% 60% 55% 50% 15 Original (Q4 2013) Optimized (Q1 - Q2 2014) What was driving the increase in completion rates? Survey Completion rates by Device Type % completed out of all starts 100% 95% 90% 85% 80% 75% 70% 65% 60% 55% 50% Desk top Mobi le Not optimized August 2013 16 Mobileadapted Q4 2013 Mobileoptimized Q1 2014 How did our mix of completions change by device type? Device used to complete survey 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 23% 2013 Q4 30% 33% 2014 Q1 2014 Q2 Desktop 17 Mobile What happened with Net Promoter Scores? Net Promoter Score May be some impact on tracking, but trade-off for better experience and data Desktop Mobile Not Optimized August 2013 18 Mobileadapted Q4 2013 Mobileoptimized Q1 2014 Were there any changes in perceived ease of use of Orbitz? % Very/somewhat easy to use site/app Original Mobile survey for Mobile booking 19 Original Desktop survey for Desktop booking Optimized Mobile survey for Mobile booking Optimized Desktop survey for Desktop booking We see differences in who uses mobile for surveys, which likely is improving representativeness Gender by Survey type 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% Mal e Age by Survey type 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 55 + 35 54 18 34 Desktop Mobile 20 In conclusion, after our testing… • Changes in response metrics were neutral to slightly positive and survey changes mainly neutral which gave us confidence to shift all of our satisfaction surveys to offer the mobile-optimized experience ü Took opportunity to simultaneously optimize desktop survey experience to be more focused and engaging • Now we design most other research efforts with “mobile first” perspective as that is where customers are ü Standard use of sniffer for most studies to deliver right experience to respondents ü Starting with shorter mobile survey then recruit to finish on desktop for more complex questions, led to increased survey engagement and completion ü Although still some exceptions – conjoint, eyetracking, large format stimuli, etc which are not easily converted to mobile-optimized but anticipate development here as well 21 THANK YOU AND QUESTIONS
© Copyright 2024 ExpyDoc
