Tracey Reed - Air New Zealand - Creating The Airline Of The Future With Experimentation

Episode 023 - Tracey Reed - Air New Zealand - Strategic Experimentation

 

Gavin Bryant    00:05

Hello and welcome to the Experimentation Masters Podcast. Today, I would like to welcome Tracy Reed to the show. Tracy is currently a senior conversion optimization specialist at Air New Zealand, with over 12 years of experience in the industry. She has been instrumental in implementing and scaling the experimentation program, leading key optimization initiatives, and advancing program maturity. Welcome to the show, Tracy. 

 

Tracey Reed  00:34

Thank you. Glad to be here. Thanks, Gavin for having me.

 

Gavin  00:38

A little bit of a sidebar for our audience Tracy is one of the speakers at the 2024 Asia Pacific Experimentation Summit this year. So really looking forward to learning a little bit more A/Bout experimentation at Air New Zealand today. Air New Zealand has long had a reputation as providing an exceptional customer experience in the airline industry, so they're really excited and keen to learn a little bit more A/Bout how experimentation feeds into developing and nurturing this fantastic customer experience. So Tracy, if you just give us a little bit of an overview of your experimentation, journey, and background to date, please.

 

Tracey  01:23

Sure, thanks. I never set out to become a CRO specialist. I'm not sure if anybody specifically does, but I started in digital. So digital marketing, back in the day where you had a website master, someone just looked after the website, and it's naturally grown through there. I think it's really built off curiosity. So I've been through pretty much all of the big sectors, insurance and finance, travel, airline, education. I think as CRO really started was when I was working for Cabarbone Travel Insurance, which is well known in Australia and provides the insurance for Air New Zealand and a few other brands in New Zealand, and they just wanted me to optimize travel insurance. So that's where I really probably learned the art. I knew digital experience. I knew analytics, but that was my first foray into A/B testing, which becomes somewhat addictive, right? When you find a win or a giant loss and you're like, but why? So from there, moved into in New Zealand, probably A/Bout eight or nine years ago, when they first built their analytics team, and CRO was part of that. So got to establish the program there with Simon woods, and it's been quite a journey. I took a few break, quite a few years, break from Air New Zealand, and ran my own consultancy, working with lots of customers, both big enterprise and small startups, all of which had different challenges different customers. So I have a lot of interesting and probably less interesting knowledge across huge amount of industries. And then I've come back to Air New Zealand, it's just great to be able to work somewhere that has a program of its size, and you can really make a difference within the customer journey through CRO.

 

Gavin  03:57

So given you’re really diverse experience across a lot of different types of customers and a lot of different types of industries. What are some of the universal truths that you now know about experimentation? What are some of those things that really stick out you that are absolutely true about experimentation?

 

Tracey  04:19

I think the number one that applies to every company, every brand, every digital experience is, you have to understand the customer pain points and deeply understand them, to solve them, to increase conversion. Sometimes those can be known pain points, and sometimes they can be unknown pain points. I'm sure you've all seen an ad on Instagram or TikTok, and you're like, I never knew that I needed that, but now I can't unneeded. So those are those unknown customer pain points, but really focusing on, might see a drop off in an area of a website or, I guess, getting a lot of customer feedback about one experience within a journey. And you can solve it, but you have to deeply understand it to solve it in the right way. So I think that the universal truth is, unless you really understand the customer, and know that they won't always tell you the exact problem to be able to increase conversion.

 

Gavin  05:27

Let's talk about that one for a little moment. I've been conducting some other podcast interviews with the other speakers for the APAC Experimentation Summit, and I think universally across all of the discussions that I've had, that everyone has suggested one of the key struggles that all experimentation teams have, and many product teams too, is having clarity around those pain points and Identifying those customer pain points. Why do you think organizations and teams tend to struggle so much with that? From your experience

 

Tracey  06:10

I think it's probably relying on, normally, two points of data to try and understand that problem. So when we have a look at analytics, and we see there's a problem, and then we take that exact drop off, or that exact conversion point, and then we go and ask customers why, or we have a look at other people and see how these other companies and see how they're solving it, and then try and test on that. But you have to have multiple data points to really understand what's going on and even, how do I explain this? So even while that might, even while you can see a drop off on one point of the funnel, the problem might be in a completely different part of the funnel. So you're always trying to fix it right there without having the holistic view of what is important to the customer and why that is not working for them?

 

Gavin  07:06

Yes, that's one of the conclusions that I've had as well that and potentially the area of struggle, is that to be able to do that, to be able to connect multiple disparate pieces of information across multiple information sources, qualitative and quantitative, it really requires a different type of thinking, and much of our thinking in day-to-day work is linear thinking, where we're making very simple interconnections and links to pieces of information, but to be able to formulate really strong and well-reasoned hypotheses and to be able to identify those strategic opportunities and pain points. It really requires systems thinking to be able to piece together all this information, make sense of it, identify the patterns and themes, and then truly make meaning from it. So I think, teams being able to switch from that linear thinking into systems thinking mode, which is not really our dominant mode in business can be challenging for lots of people.

 

Tracey  08:06

Yes, completely agree. We're very lucky in New Zealand that New Zealand is very passionate about Air New Zealand, and so probably one of our best data sources is summer barbecues. So everyone comes back after the weekend and you say, Well, what did everyone talk about at the barbecue? And they'll give you the customer pain points. So that's one of our richest data sources. Overlay that with actual data and some other analytics, and you've got your pain points really.

 

Gavin  08:36

You've got this continuous stream of qualitative feedback coming in from customers?

 

Tracey  08:43

Yes, 100%.

 

Gavin  08:46

Hey, I've never asked this next question of any podcast guests before, so I'm really interested to get your perspective on this. What's an epiphany moment that you've had with experimentation recently, something that's come up?

 

Tracey  09:07

I've had some very specific ones, and I don't know if they would apply to others, but sometimes it's probably around retailing and even it might be more prominent right now coming off the back of COVID and New Zealand's economy has had bit of a hit, and so a lot of retail brands are heavily relying on Sales and everything else to basically keep their doors open. And so the population in New Zealand has been taught or educated to buy on sale, which means you basically have to live in sale area right now to continue business. But what can we do for those people that we're going to purchase anyway, and so starting to really understand those different audiences and then personalize towards those? I think everyone's always personalized towards presenting the best sale or the best price. But actually, what does retailing look like for those that already had that high intent? So yes, I'm not sure if that's an epiphany moment, but it's definitely something I've spent a lot of time diving into more recently, just especially with the way that the economy is and how we've trained our retailers really well our purchases.

 

Gavin  10:41

So just getting a lot clearer on, the behaviors and the motivations of different cohorts and segments, and to your point, the level of personalization is required across those different customers.

 

Tracey  10:58

Also the psychology of it as well. We think that everybody wants the cheaper price point, and you think about traditional retailing like clothing. But actually, there's a segment of people that, if you don't show a sale to more likely to purchase because they see the quality or the consistency as a better selling point. So yes, I think if you spoke to a customer, they're always going to tell you the price point, but we know that what customers tell us and what they do aren't always parallel.

 

Gavin  11:36

Yes, I've done quite a lot of experimentation around price over the years, and I find it a really interesting space to operate in pricing incentives and discounts because customer behaviors that sometimes are apparent through these types of experiences are definitely not what you expect. And there's lots of unexpected findings where providing to your point, a cheaper offer. Sometimes people can be skeptical of the offer. And at the other end of the spectrum there providing, spades and spades of value can often be perceived as too good to be true. So it seems like oftentimes there's a sweet spot in the middle where the customer perception of value seems to be more closely aligned to their needs and desires. So a really interesting space to experiment in.

 

Tracey  12:29

Yes, agree, and I think it's just taking the rest of stakeholders and business and business owners on that journey, because I think we reach for sales and discounts first, when maybe that shouldn't be what we're reach for all the time.

 

Gavin  12:47

Let's shift the focus a little and talk a little more specifically now around the Air New Zealand experimentation program, one of the things that I'm really interested to explore with you, Tracy, is that through the line experience? So you've been there right from the beginning of day one and through to current day. So thinking back to those really early days when you were establishing and getting the program off the ground with Simon Wood, what did those early days look like? What did you experience? What did it feel like?

 

Tracey  13:23

Good question, I guess that's taking back a few years. It was exciting that you had to I think the one thing that I've always found in CRO is, that you've got to find your influences, so finding your stakeholders that you can help champion the program. Help tell your story. You need them to be on your side and truly believe in it, to get buy-in with their peers. Yes, and it's a slow journey. Sometimes I've found that I can say all of the things in the world, but when we've got the biggest amount of buy in at Air New Zealand is when I got some of the senior leaders and took them to Las Vegas to Opticon. Optimizely said the same things, and we came back and everyone had drunk the Kool-Aid of CRO and the program could have accelerated from there. So sometimes it's just bringing in outsiders to say the same thing as you, but also finding your alliance partners internally to help champion so it's not just you speaking about, it's all the time. But yes, I think Air New Zealand has had its ups and downs. You know, being an airline, it was heavily affected over the last few years with the economy and generally not being able to fly. So, yes, it had a big amount of buyin and it pulled back over COVID years, and we're slowly building it back up. But internally New Zealand's completely changed the way of working as well. So they've moved to a fully agile business, and so re-finding those alliance partners and stakeholders working with new areas of the business and training them up to get the program back up to speed again. It's always interesting. The amount of influencing that you do within CRO is probably equal to the amount of analysis that you do.

 

Gavin  15:36

Yes, I agree. I've experienced the same thing with starting and scaling a program that after I've been through that experience that my little mental model was 50% designing and executing experiments, and then 50% on engaging communications and influencing. I think, over time, as I've reflected on that further. I've maybe even gone further the other way that maybe 30% is around executing experiments, and potentially 70% on engagement and influencing. Hey, one of the things I wanted to quickly touch on there with you, you mentioned that when you're getting the program up and thinking about those early partners and use cases, those influences you referred to those being the people that were most likely to experience the most benefit of experimentation is that how you targeted teams early on.

 

Tracey  16:35

I think you'd have to influence at multiple levels. So yes, commercial impact is probably going to be the easiest way to start. When people see financial gains from a program, they're more likely to buy into it. There's many other positive reasons why you'd be doing experimentation, but that's probably the easiest way to get by in the beginning. So understanding where improving customer experience is going to have the biggest financial impact is probably the place I start. Whenever I start CRO with in New Zealand or any other brand, when it's within a larger company, it is firstly, you'd need to start with the person that's running that area to get them on board to see what some of those gains could look like. And then it's getting the actual doers to champion the tests themselves. So they become passionate about it. They want to build A/B testing into their work. So whether it's a developer and they help you solve a customer pain point. I usually say to them this is your test. I will help you through it. I'll do the analysis, but I want you to present back the results. Then they feel that win, and then next time they want to test their experience. So it's about facilitating the conversations, but usually starting with revenue.

 

Gavin  18:09

So thinking A/Bout that early stakeholder landscape, you've mentioned there that using a bottom-up approach, so targeting leaders and the doers, people are going to immediately experience potential commercial gains. But then you also mentioned engaging the senior leadership team as well, taking them on the trip to Opticon, so approaching it from the top down too. So you were using this simultaneous, bottom-up, top-down approach. Do you feel that's what works the best in the organization?

 

Tracey  18:50

Yes, you need your senior leaders to prioritize CRO. It's not always a cheap game to play. It requires tools and time investment. So you need to take them on the journey. They need to have that buy-in, and then they need to see the results. So it's getting the buy-in, and then working down to be able to get those results, and getting them consistently is working with the doers and making them care A/Bout it. Making them do more, making them speak about the results so that the senior leadership then sees and hears that value. So yes, both, the top and bottom are very important. You can't just be a CRO or an A/B tester sitting in the corner anymore. It’s needs to be a holistic company-wide approach.

 

Gavin  19:45

Has that been consistent across the other organizations you've worked with and in over the years as well?

 

Tracey  19:52

Yes, even in medium level enterprises have generally had to sell my services to C suite, MD, CEOs, and then you work with the marketing team and the developers to actually execute on it. So it is always, I think, always, both senior leaders and the doers that make a program successful.

 

Gavin  20:23

Let's move along a little bit from those earlier days, day zero, establishing and starting the program. So thinking about that journey as you're moving along, what were some of the early growing pains that you experienced in the organization? Trying to grow out and scale the function in the business.

 

Tracey  20:48

I think one that was true in New Zealand and has been true elsewhere as well if you've always got your product teams who are developing digital products, and whether they're in waterfall or agile, they all have their deadlines. They want to deliver and ship their products, and getting them to care about A/B testing upfront and then A/B testing those experiences into production, making them feel like it's a must-have, or that they want to do it, instead of just shipping and moving on to the next. So that's always a pain point when you've got development teams that are under pressure, and adding an extra step. Normally, when it is two weeks to setup the A/B test, a new experience can be seen as really slowing development and product down. So make sure it takes a lot of effort to maintain your relationship with those stakeholders, to make sure that they do care about it and you support them in the right way to add value, instead of being seen as a slower downer.

 

Gavin  22:12

That's a really good point. Thinking about experimentation culture, you've mentioned, a lot of engagement with the teams who are doing the execution, in addition to the executive and leadership teams, when you were building out the function in the business, what were some of the strategies and things that you did to try and develop, and grow and nurture this culture of experimentation?

 

Tracey  22:42

I think a CRO was only one piece of the insights analytic and insights function. So I worked with my team chapter at the time very closely, where we would report on the performance of how a product was working. And we would present that back each month. Say this is how it was this month, against last and last year. But we would always pull out two or three. If we can improve these areas, or this is what we think some pain points or opportunities might be, we invite anyone and everyone to those sessions. So we had, for the first time, developers and UX strategists lining up to try and get into this meeting room every month to understand how their product was performing and what the next best opportunities were. So that was probably the catalyst of becoming customer-obsessed, really. People were like, cool, that's the data. That's how it's performing, and we think these are the pain points. But how do we solve this? So we really saw both developers and UX and insights working together almost in the three amigos to try and really understand, and solve those problems.

 

Gavin  24:09

And you found that your public forums and communicating this within key meetings and sessions was really instrumental in spreading the efficacy and the benefits of experimentation to customer experiencing commercial return.

 

Tracey  24:30

Yes, and we still do that today. There was a bit of a gap over a few years, but we've just put that back in place in New Zealand. So every month, everyone across the business comes and shares the experience experiments that they've done, some of the key insights, and then also what's coming up as well. So everyone's aware of what's going on within the business and able to take those insights away and apply them to the area as well.

 

Gavin  24:59

Fantastic. So thinking A/Bout operating structure, in those early days, when you were starting off the program, was that like a small centralized function that you were operating?

 

Tracey  25:15

We started, it was centralized, but it was more hub and spoke. So we had a centralized team of analysts and CRO specialists that we would go out and work with the key areas of the business. So we're very specific about which areas we worked with. I said before about identifying where we could make the biggest impact, and saying we've only got enough team or support to work with these areas right now. And while we would love to do CRO everywhere, we just don't. We can't spread ourselves thinly enough. So we made a very conscious decision of where we would focus the efforts and what teams of the business we would be working with,

 

Gavin  25:59

and how does that look today?

 

Tracey  26:03

We are now, I would say embedded. We still have the COE. We've got our Center of Excellence which is in an agile world. Centers our chapter, and then we are actually embedded within the tribes and within commercial squads. I move around depending on where the biggest focus is at any time. But we've also got a major and minor program as well. So probably taking that a little bit further, we will work with our digital performance marketers who have picked up that minor CRO, so within their day-to-day roles, they're also championing and running A/B tests to align with their squad's goals for that quarter, for that year. And we support them and then additionally run our own experiments alongside them as well.

 

Gavin  27:04

So what are some of the unique challenges that you've seen experimenting within the airline industry, that can be such a high volume transactional industry. Also seems like there's two parts here that there was the pre-COVID, where the function was growing or flourishing. There was some somewhat of a hiatus for obvious reasons, and now it's regeneration and reinvigoration post-COVID now.

 

Tracey  27:39

Yes, I mean, it was a shame that there was COVID and there was no travel, not just for the airline, but for everyone who likes to travel. But I think we've picked the program up pretty quickly again post. So I don't think that's really a challenge. It's just nice to see that it's come back from an airline point of view is probably how global we are, which means that you've got so many different types of customers every travel around the world, and the different region has different needs out of travel. While we can think of the New Zealander as our customer traveling out of New Zealand. They all have to come back. So those airlines the airplanes, have to come back, and they have to have passengers on them. You have to have a global view and understand the driver's needs and pain points of travelers from other regions. It's complex, to say the least. So I think that's something a lot of if you think about retailing, you would normally just have your target regions, and you might have your prominent region and two or three others. So you need to understand the customer needs for those regions. But when on a single flight you've got multiple motivators, yes, that adds a fair amount of complexity.

 

Gavin  29:14

How do you stay on top of that so that you've got a really up-to-date and relevant understanding of all these customers coming in from all over the world?

 

Tracey  29:26

I think it's something that we've always been challenged with and trying to get better the way that we work now within the Agile world. We've got regional teams which sit in each of those main regions. So Australia, US, Japan, China, and they know their markets really well. So it's about opening the communication lines with them. They now actually sit within one larger tribe. So if you think about international it's not just a team sitting in New Zealand, it's a global team, and they're very good at opening the table and listening to each other's needs and supporting each other. So yes, just knowing that we don't know it all from New Zealand and reaching out to those teams to understand what is important within their regions, from a cultural point of view, from a needs or a market, what's going on with an industry in their region and overlaying that lens.

 

Gavin  30:34

Yes, there are a lot of different cultural idiosyncrasies that exist across all those markets that can be challenging.

 

Tracey  30:43

Well, think about even in New Zealand, we see an elite travel experience as being known by your name, knowing what your preferred seat is or what your meal is, that makes you feel good. But from America, Elite is more hands off. You don't want to be spoken to, everything should happen. So they don't want that personalized experience. They want your personalized experience without the engagement. So, yes, just understanding what people perceive to be a good experience by region.

 

Gavin  31:23

So looking forward over coming years, what is the future hold for experimentation at Air New Zealand?

 

Tracey  31:32

Good question. I think we're at a very exciting point, not just as an airline, but in digital as a whole. I feel our speed of learning right now is a very steep curve. With AI coming in, we're only just starting to scratch the surface of what that can do. I know within travel there are some AI experiences which are very visual, engaging, and you can see that it's AI-driven. But I think where the real impact of it is going to be is behind the scenes, and how we learn quicker, how we extrapolate big data quickly to analyze it. Yes, I think, that's going to change a lot in a very short period of time. And I don't know exactly what that is, but we're open to it, and we're learning quickly, and we're focused on it, yes, but I think a lot of it, it won't be obvious. It will have a lot of AI-driven experimentation and experiences that feel natural because we've been able to learn quickly.

 

Gavin  32:46

Yes, I think airline is one of the most exciting industries and businesses to be involved in, from an experimentation perspective, there's a lot of experimentation occurring digitally at the moment. But yes, it would be really interesting to see that expand right out through the whole customer experience and value chain, right from customer awareness to arriving at a destination and everything in between. So a lot of offline experimentation is supported by online too. So, yes, super interesting. I don't know if it was in New Zealand, but I was reading an article recently where AI was being used to scan all of the food trays coming off lights when they reached their destination. So then there could be a profile that can be established around food preferences and then menus adjusted more accurately and quickly to reduce food waste as well. So there are just so many applications of AI within the industry.

 

Tracey  33:53

Yes, and I think airports as a whole are starting to understand the impact that it can have on the day of travel, so not just from a customer preference point of view, but also how can you turn around those airplanes quicker, baggage handling, a lot of different areas. So yes, we have a look at how we can apply AI and different ways to solve our internals, our internal customer problems, that then will impact the external customer. Day of travel is always stressful. No matter if you get it perfect, there's something about it that you've just got so many things that have to happen in a succinct order that it just stresses humans out. So hopefully we could make it a lot more predictable using AI and reduce the stress of day or travel. I'm sure all humans would benefit from that.

 

Gavin  34:51

You've been in the experimentation game for a long time now. You've seen a lot. You've learned a lot. What's some of your key pieces of advice that you give to other experimentation programs?

 

Tracey  35:06

I think growing experimentation and opening it up to more users is important than it being perfect. We all started somewhere. I've probably run some terrible tests and so annoying that a minor program allowing people to experiment and learn, and become curious, is what's going to grow experimentation, you can finesse and perfect things as you go on, but I think get allowing more to happen with it being a little less perfect. Probably gets that buy-in and reduces the barrier for more people to experiment.

 

Gavin  35:58

Yes, I don't know who this comment was attributed to, but I always remember it, direction first, speed second, quality third. So I'm not sure if that one came from Lucas Vermeer, maybe, but it seems to be a good little heuristic to think about for experimentation teams.

 

Tracey  36:18

Yes, I'd agree on that. I still think quality is quality and speed hand-in-hand. But I think it's just about allowing people to have a go and making it not feel so scary, like you need to know statistics but don't talk in statistics language. So make everything to reduce the barrier. We've got a lot of acronyms, A/B testing, CRO, experimentation, and personalization, everyone gets a little bit overwhelmed by it. So just making sure that you use plain English and allow people to be involved is probably. Yes, my main advice.

 

Gavin  36:56

Fantastic points. Hey Tracy, let's finish off with our closing, fast four, just four quick fun questions to conclude our chat. What's an experimentation belief that was strongly held that you've since changed your perspective on?

 

Tracey  37:22

That everything has to be A/B tested. I would, yes, perfect world. I'd love everything to be A/B tested but just understanding when experimentation and CRO is important, and sometimes when things just need to be shipped.

 

Gavin  37:43

So thinking A/Bout those scenarios where sometimes we wouldn't look to test, what are some of those examples that have popped up that you've said sometimes you just need to ship?

 

Tracey  37:57

Well obviously, when there's business critical. A lot of the time when if it's a big brand play, if it's got all of the right insights in it, and its high level, it just needs to go out, monitor it and understand the performance. But A/B testing is just one of the tools in what I'd call the experimentation or optimization toolkit, so you can definitely optimize things while they're in production, or knowing when to use just the right insights to make sure you've got your best experience going out. But yes, A/B testing is just one of the tools and the optimization toolkit.

 

Gavin  38:49

And in that example you provided there with a big brand play, you would still look to adjust and refine based on the learnings that are coming in after launch.

 

Tracey  38:59

Yes, just because you're not A/B testing something into production doesn't mean that you shouldn't use all of the insights and multiple data points to inform that experience and then to monitor it, and analyze it and then provide recommendations for next time. So that's still an optimization piece, a data-driven experience that's monitored and reviewed, but A/B testing is not always appropriate.

 

Gavin  39:31

Number two, what's been your most challenging personal struggle with experimentation? 

 

Tracey  39:46

I think probably, everybody that has been a CRO specialist has been through this. Usually, when you feel like you're doing pretty good, and then you have a new stakeholder, and you've got to start from scratch or they don't believe the results or the process, and then having to start that influencing journey again. So, yes, that's it's probably you always take bit of a hit when you're like, hang on. We've been through all of this, but I think I'm quite calm about it. Now, you're always going to have new stakeholders, and you just got to go slowly, take your time with them.

 

Gavin  40:28

So what's your approach, or your strategies for overcoming those trust issues with respect to the data?

 

Tracey  40:39

I think, understanding their needs and their drivers. So, if they're coming into the business relatively senior, understanding what they see as good, and then maybe moving, shifting, tilting the CRO program towards what they see as important, and then just being consistent and taking them on the journey. Yes, I think experimentation should always ladder up to what your senior stakeholders vision is, with all the right metrics in place, but they're important. They're instrumental. So just understanding what the new needs of the program are and being flexible

 

Gavin  41:28

It was interesting. I had a personal experience in one of the organizations that I was working with, a new head of product came in who had no experience with experimentation and was really struggling and grappling to come to terms with the whole concept of experimentation that we don't just launch based on what we think is right now, past experience that wherever possible. We would look to test prior to launch, and the new stakeholder just couldn't wrap their head around it was continually challenging, challenging the data. Believed it to be unreliability and untrustworthy, and one of the things I did at the time for, a key strategic hypothesis that we were testing was to A/A test it five times, and the data fortunately came back identical for those five tests, and that was a really good way to shift the perspective of this stakeholder, and we never discussed the data, the unreliability and untrustworthiness of the data ever again, and was converted from that point. So yes, sometimes it just takes a little bit of reassurance to move people along the journey.

 

Tracey  42:50

Yes, and I think I haven't had to run an A/A test for quite some time. It has definitely been something I've also had to pull out every few years throughout the journey, just starting from the bottom. Is the data correct? 

 

Gavin  43:07

Yes, the biggest misconception that people have about experimentation.

 

Tracey  43:19

There’s two camps, you either believe it or you don't believe it. So misconception, sure, maybe that it's always needs to be revenue driven. I think one of the important pieces is understanding your customer satisfaction alongside that, and knowing when you've got good revenue and bad revenue. We like to call it poisonous revenue. Sometimes people purchase things because they feel like they have to. I'm not sure if that's quite a misconception, but yes, just understanding both revenue and CSAT hand-in-hand. 

 

Gavin  44:14

So thinking about that point that you just touched on, then people believe it or they don't. Why from your experience, do you think people don't believe it?

 

Tracey  44:27

Because sometimes they're big ones, and when they're big numbers, everyone's skeptical. It's almost that too good to be true, and that's when you probably need to lean on your AHS to say no. We are. We work for a company where you can actually have experience plays a big part in the success of an airline, and really, there's nothing else to it. If you're an airline, you have an experience, and you've got to provide the best one or the most fitting one. If you're a low-cost carrier, then you need to live up to that. If you're a premium carrier, then experience is even more important. So, yes, I think when you are able to get wins or improve the experience enough that the revenue impact is large, people tend to just go, Oh, do we really believe that? But it's just making sure that you take them on the journey doing those AHS, making sure that you can tell the story of why that is true, and cross-checking. Every data point that you have available so that they can believe it and can continue to believe in it.

 

Gavin  45:56

Good point. So just finishing off our final fast four closing question, what are some of the resources, maybe books, blogs, things that you've found helpful on your personal journey?

 

Tracey  46:17

I think just being really curious and follow a lot of different blogs and create your community. So CXL, their platform has been great because they not only have their own voice, but they facilitate a lot of other people's voices as well. I guess what you're trying to do, Gavin, by bringing everyone together for this APAC experimentation Summit. So, creating your community so you can learn from others. I think there are a lot of resources on how to A/B test, but it's probably looking at neuroscience. And the way humans think and behave that has probably been the research that has had the most impact. Everything else you can Google. You can figure out how to do A/B testing. You can figure out insights and analysis and what data points to come. But unless you can really deeply put yourself in customer shoes and understand motivators, you can't really impact experiences. So I think that's CXL and then just outside of industry, so behavioral science, books, courses, lots of short courses out there, just to keep learning about humans and how strange we can be.

 

Gavin  47:53

Yes, behavioral psychology. 

 

Tracey  47:56

Yes. 

 

So just wrapping up our session today. Tracy, what's the one thing that people need to take away from our chat today?

 

Tracey  48:06

Good question. I think the one thing is that you are an influencer. You need to influence at different levels and learn how to speak to your stakeholders at different levels. Those stakeholders need different levels of information, so understanding what's important to each of your stakeholders and then how you can influence them. I think that what you said, as well as carving out time to influence, is how you will grow a program.

 

Gavin  48:39

What a great way to finish our chat today, Tracy. Thank you so much for your time today. Great to chat and really appreciate it.

 

Tracey  48:47

Wonderful. Thanks, Gavin.