Melanie Kyrklund - Specsavers - Establishing A Global Experimentation Centre of Excellence

Episode 024 - Melanie Kyrklund - Specsavers - Establishing A Global Experimentation Centre Of Excellence

 

Gavin Bryant 00:03

Hello and welcome to the Experimentation Masters Podcast. Today, I would like to welcome Melanie Kyrklund to the show. Melanie is the Global Head of Experimentation at Specsavers, a multinational optical retail chain operating in 11 countries with revenues of more than 4.1 billion euros. At Specsavers, she is responsible facilitating enablement of experimentation across the organization, including building processes, tooling, data and technical governance to scale experimentation reliably. Prior to this, Melanie has worked in product and experimentation, [email protected] Staples Europe and Liberty Global. Welcome to the show, Melanie.

 

Melanie Kyrklund 00:49

Hey Gavin, thanks a lot for that wonderful introduction. Actually have one more interesting stat to throw in, because I just discovered this relatively recently, but spectators have 44.6 million customers worldwide. Crazy, isn't it?

 

Gavin Bryant 01:07

It is.

 

Melanie Kyrklund 01:08

Yeah, people aren't always aware of the scale of the company.

 

Gavin Bryant 01:13

Now, we've been trying to make this one happen for a little while, haven't we? We've been tick, tacking backwards and forwards throughout this year, and we're really finishing the year with a bang to be able to record this one before 2024 closes out. And I've been really looking forward to recording this episode for a long time now, because what we're going to discuss today is something that I'm really passionate and interested in about, and that's really around experimentation, operating models. And Melanie is going to take us through her first person experience of establishing the centralized experimentation function at Specsavers, and then over time, progressively transitioning across into current day or current reality, which is a center of excellence model. So looking forward to really discussing the practical nuances and warts and all of making this somewhat complex transition in a large global organization.

 

Melanie Kyrklund 02:21

Yeah, same. It's been a really sharp learning curve for me, despite being in experimentation for 15 years. So now I'm really happy to share what I've learned so far during this transition.

 

Gavin Bryant 02:33

Let's start with a little bit about yourself. Melanie, if you could provide our listeners and our audience with a little bit of color and a little bit of flavor about you and your background and your personal journey, please.

 

Melanie Kyrklund 02:47

Yeah, sure. So people might be slightly baffled by my accent. They'll say Finnish sounds really British, but there's something quite right about it, then you're correct. So I'm half British, half Finnish. I grew up in Italy, in Rome. So when I was 18, I moved to the UK to study. I stayed there for 10 years, but I have now been in Holland, in the Netherlands, for over 15 years. So I guess I'm on to my fourth culture, and I'm a mix of all these places. So as mentioned, I've been working in experimentation for about 15 years now, but I've been in digital for over 20 so I started out on the digital marketing side as a media buyer and doing affiliate market, both agency side and client side. I mean, I'm not a very cool person, but I've had moments where I was doing quite fun stuff, one of them being buying digital media for Disney and Pixar movies for a while in my 20s. And the transition to experimentation happened by chance. 

 

So I was interviewing for the sort of E commerce deployment role at a large digital brand and TV provider here in Holland. I didn't get that role, but a short time after they had some budgets available for consultant, they just brought in Adobe target, and they were looking for someone to sort of start using that within the organization. And I jumped at that chance. And, you know, I was hooked immediately. I think a lot of people are when they enter the experimentation space. And following on, from that start, I then moved on to Booking.com I ran experimentation at Staples as well. And now I'm at Specsavers, it's coming up to five years. So my background really is in, like global, large, complex organizations, which, of course, also the complex products. And that brings a specific set of challenges, of course. 

 

Gavin Bryant 05:00

Melanie says she's not cool, but she's very cool today, and she's leaving me well and truly in the dust with the fashion stakes today. 

 

Melanie Kyrklund 05:12

Yeah, back in.

 

Gavin Bryant 05:14

It was never out there.

 

Melanie Kyrklund 05:16

It was for a while. It's back now, yeah, it's cool again, which I'm delighted about. 

 

Gavin Bryant 05:22

I do have some leopard print in my cupboard, so maybe-- 

 

Melanie Kyrklund 05:26

Time to get it out 

 

Gavin Bryant 05:27

--It was cool back in the day. Okay, let's just discuss that journey a little bit further. So you've been on an experimentation journey for 15, 16, years now. What do you know about experimentation now that you wish you knew earlier.

 

Melanie Kyrklund 05:44

So it's about the integration within the business being key, so embedding experimentation into marketing and digital product just workflows as much as possible. I started out in experimentation in the era where it was in most companies was spun off as a sort of silo, and I feel that in some of the roles, even at Specsavers, for example, I could have pushed for that integration sooner, because it's really key in scaling experimentation and building the culture. But you often fall into the trap, and understandably so saying, I need to prove the value first and partner with the business. So I think it's quite a bold move to insist for integration just because it's sort of correct from a theoretical perspective, without building that momentum and that trust in the process. So, I mean, it's an alternative way of looking at it, sort of jumping in and pushing for the organizational piece from the outset.

 

Gavin Bryant 07:04

That will require a lot of bravery and boldness to be able to do that.

 

Melanie Kyrklund 07:10

Yeah, and a lot of persuasive skills, I think. I think, I spent a lot of time sort of building experimentation strategy and experiments in a siloed way earlier in my career, and sort of thought about the integration maybe a few years too late in some of the organizations I worked in. 

 

Gavin Bryant 07:32

I think you make a good point, though, that what's most important are those early wins and those proof points to be able to upsell the benefits and yeah, maybe mounting that case for an embedded function without those proof points might be a little challenging.

 

Melanie Kyrklund 07:52

Exactly, yes, I agree.

 

Gavin Bryant 07:55

Okay, another question I wanted to ask you on a similar vein and a similar theme, was what's a strongly held belief about experimentation that you've since changed your mind on, or you now have a different perspective.

 

Melanie Kyrklund 08:11

I think it's my belief that the actual experiments are the most interesting part of working in experimentation. So having worked in strategy for a long time, churning out experiments, sharing the learnings, I always thought that's what it was all about, and that's where I would derive satisfaction from. But since I've moved towards the task of thinking about enabling experimentation within an organization, I found that is far more multifaceted and interesting in many ways, just kind of unexpected for me, because, of course, the cultural change is comprised of so many parts, So you're tackling it from a data perspective, tech process, governance, workflows, enabling people, training and, of course, there's a whole cultural part building psychological safety. So yes, there's just so much to learn I feel in that space and being on that journey compared to running experiments. 

 

Gavin Bryant 09:24

That's an interesting point. As you know, we recently hosted the first Asia Pacific experimentation Summit here in the southern hemisphere. And one of the points that Lucas Vermeer touched on was and we were talking about the experimentation flywheel paper that Alexander Fabian produced with Lucas, and there's now maybe a sentiment and a feeling that the flywheel needs a bit of a refresh or an extension, because there's all those elements that you just touched on that they're like an overlay or an umbrella, but sit over the top of the experimentation flywheel like that are maybe often not considered or maybe underestimated.

 

Melanie Kyrklund 10:12

Yeah, absolutely. I'll have to circle back to that paper. I haven't read it in a while, but yeah, to your point, I'd be keen to see if there indeed some components that should be added in. Yes, it's a vast topic and really interesting to be on this journey. 

 

Gavin Bryant 10:33

One of the things that I've been thinking about and going off on a little bit of a random tangent here is, and extending on the experimentation flywheel is when we think about a product, we have product market fit, and we also have the concept of channel fit as well. A thing that I've been reflecting on and stewing over is concept of experimentation and organization fit, and for every organization that the fit of experimentation in that specific business or company will be different to the next, and there are a number of different factors and variables then influence the fit of experimentation being right for that organization at that right point in time. However, that experimentation organization fit needs to change and evolve over time. 

 

Melanie Kyrklund 11:29

I think I'll add to that, it's even at an organizational level, the Fit varies compared to the department or the team that you're working with, if you just look at it at the highest level. So for example, what does a good experiment look like? What does good failure look like for this department or for this type of work? The way you design an experiment is very different if you're working content marketing versus if you're working in a very sort of complex product, such as a checkout or payment for example. So I agree, we have to have some sort of framework, a more abstract way of defining what good looks like, and tailor that really to organizations, but also to different teams within organizations as well.

 

Gavin Bryant 12:29

Yeah, that's a really good point. That fit can potentially vary depending on the business unit.

 

Melanie Kyrklund 12:35

But we need to find the common things that we should be aiming for, because then those can help you scale that message and that culture across the organization. So those common things are making sure we do everything back by data. You know, regardless of where we're working-- What other examples can I give minimizing the cost or failure? So we will look at time and resources in a specific way-- So yes, there need to be some common elements, common cultural elements, but then indeed, the application can be, must be certainly different for the most part, especially if you're working within a complex organization lots of different teams and products.

 

Gavin Bryant 13:25

Let's shift the focus of the conversation to Specsavers experimentation program, and we'll spend the rest of our conversation today largely focused around the transition and the development of the operating model over time you've been working a lot more strategically, and you touched on earlier about experimentation enablement, when our audience and our listeners are thinking about experimentation strategy and program strategy. What does good look like in your experience?

 

Melanie Kyrklund 14:02

So first of all, the experimentation strategy needs to be aligned or focused on certain outcomes that you're looking to achieve for the business or the customer. So experimentation really drives the accumulation of the decisions that you make in order to sort of hit that outcome. So I'd say that's point one. The second, which is sort of commonly known, is that, strategy should be anchored in research and data. So looking at qual and quant methods to bring a really coherent picture of the problem we're going after and the customer pain points we're looking to address. There should also be understanding that experimentation is iterative, so there can be alternative solutions to a problem, even if the first experiment doesn't work. Similarly, we should double down on the things that do work, and there should be space to balance between small wins and bigger swings as well. I actually do believe that a lot of success has borne out of small, incremental, fast decisions. But at the same time, there should always be room for bigger ideas to be explored within that. And lastly, yes, the experimentation strategy should be really focused on minimizing risk as well in the decisions that we make, and that means also making sure that the times and resources spent on failure are not too costly for the business. 

 

Gavin Bryant 15:52

So thinking more specifically about that journey at Specsavers over time. So thinking about the timelines and some of those pivotal moments over those past five or so years, what does that journey look like for the organization, please.

 

Melanie Kyrklund 16:14

So the experimentation program predates me. It was kicked off seven years ago by some current peers of mine, they were working digital analytics and CRM for the group function so in a global capacity, and they brought in an agency to kick start their experimentation efforts across regions. So I would say, because it was a small part of the responsibilities, the velocity was initially slow, maybe 12, 15, experiments per year, something like that to begin with. And I was brought in four and a half years ago. So just before the Coronavirus pandemic began, and I was tasked with driving the experimentation program forward, my key stakeholders are and were the digital product teams and then our regional digital teams who are responsible for the sort of trading performance in our different regions from a digital perspective. And the first thing I did was bring strategy in house. So I feel that business partnering is really key. You need to spend time with all the stakeholders to understand what all the regional nuances are, at the same time run focused research efforts that allow me to understand what the common themes are to go after, the common pain points that can then be fed into solutions for global product development. So I'd say I was probably running about 80 to 90% of the experiments within the business at that point. And the beginnings of some local optimization teams in the UK, and then some competence for optimization also in Australia. But at that point, it really was a case of the central team running the majority of the experiments on behalf of digital product owners and the regional teams. So I guess my real focus at the beginning was the business partnering and showing the value of experimentation and increasing the velocity as well, which, as we said, are our key parts to building the visibility of experimentation within the organization. But I'll say that previously to me running some of the experimentation was a bit fragmented. So it was kind of just based on sort of tactical opportunities that were spotted across any market or any part of the journey. So I bought more of that sort of global structure, global themes and research to the picture at that time.

 

Since a year, of course, the journey has changed since about a year.

Gavin Bryant 19:35

So the business has been experimenting for seven or so years that you've been leading the program for the last five or so years, so from those early days, how recently has the organization made that transition to a COE model?

 

Melanie Kyrklund 19:56

So it started with sign off, let's say, on the approach just over a year ago, and it kicked off in earnest in March of this year. So I mean, a key step in that process was getting buy in from stakeholders around the business, they felt that their teams should be absorbing more of that experimentation process. I then had to bring in a new agency. So we appointed Click Value, an Amsterdam based agency who have a lot of experience in scaling, setting up COEs and scaling experimentation within large organizations, which is kind of slightly different skill set to, let's say, full service experimentation agencies who are more focused on the delivery of experiments. So they officially started on March. So we're about nine months in now, on this journey.

 

Gavin Bryant 20:55

So, let's just zoom out a little bit to discuss that change in a little more detail. So, what were the specific business needs or requirements that really drove the need to transition from that more centralized function to a COE model?

 

Melanie Kyrklund 21:19

But ultimately it becomes from a decreased capacity to meet the experimentation needs of our stakeholders. So ideation rapidly becomes a bottleneck if you have a central team who is juggling multiple journeys, multiple regions, because, of course, my team has to look after the appointment booking flow, as well as contact lenses and glasses, eCommerce, that's at a minimum, so you rapidly get pulled in a lot of directions. And I think ideation does need focus really. So that's that already becomes a bit of a bottleneck as well as development, of course. So we were funding the CRO through our agency budget, so then we're able to flexibly pull in the resources we need in order to deliver experiments in the majority of the teams, and of course, we weren't able to meet demand quite as well, which got me thinking about what's the most cost effective option is it to add more resources to the agency, add more experimentation specialists to my team who can interface with the business, but I kind of came to the conclusion that was more cost effective to distribute parts of the experimentation process wherever possible within the teams. 

 

So ideation really should be coming from the teams as a bare minimum, even if you don't have the time to or the resources to develop or design experiments, then at least you should have a good grasp of how to do ideation, how to do problem discovery, and be able to feed that into my team for delivery. So there's that bit. So that's scaling. And then the cost part, I think there's also the impact. Is something that we were struggling with? I feel that if you have multiple teams really layer [phonetic 23:32] focus on the KPIs that are important to them and driving this forward, have that autonomy around it, you're going to get more impact the other side, compared to having one team broadly responsible for lots of KPIs, lots of outcomes. So I guess those were the, sort of the key steps that initiated that conversation for me internally.

 

Gavin Bryant 23:58

And you touched on a little bit earlier that, you know, there was a need to showcase and to get rubber stamp on this new transition and the new model. So could you talk through a little bit about the process of engaging with stakeholders and the executives to then get the green light to make that transition? What was involved there?

 

Melanie Kyrklund 24:29

So it actually wasn't a hard sell, I think I was quite lucky in that way. So we've been working with these teams for many years, basically. So there was appetite for all teams to be able to experiment more and to do more. And that was clear with all the Heads Of we spoke to across the digital product and the regional teams. So I think the why we were doing it was never questioned, and I think that's already like a big win, isn't it? It was more like how we're going to deliver that within the resource constraints that we have in a number of teams. So even though we're sort of moving to COE, we actually fund a lot of CRO resources from, let's say, a server side developer to work on appointment booking journey to having our agency resources click value for client side builds. 

 

So I guess they were keen to know that, that resource would still be there for the teams who needed it, but at the same time, they needed clarity on which parts of the experimentation process they were going to be encouraged to take on and be trained on. So I think on the lower we still did a maturity model, of course. So we have three buckets of teams. 

 

So, let's say the highest level, we have teams who are basically autonomous with experimentation. They do have the design and development resources so they might put us into consult on solution design or how to analyze an experiment, for example. And then in the middle tier, we have teams who have the resources and the know how to start doing simpler tests, so client side tests, and then on the lower end of the spectrum, which is the majority of the teams, at the moment, they do need to rely on us on a lot of the delivery and analysis of experiments, but we're training them on ideation, because I think that is; a) it's the first bottleneck you encounter, having a backlog of well-defined opportunities to go after. And secondly, the training that I'm in problem discovery is really the first step in getting them to think like experimenters as well. So I guess it's getting them to understand the why we were doing it, which was fine, and then how we're going to do it, and how the teams are going to be impacted, but impacted by it. And I think once we had clarity on those, on how we're going to fund those gaps that were still present in experiment resourcing, then we were given the go ahead. They all agreed that all teams should be experimenting when embracing this as a natural part of the decision making process.

 

Gavin Bryant 27:29

That's a much easier position to come from, isn't it? The organization is not asking why we’re making this change. It's more a transition plan and an execution strategy around how you're going to do it, which is a much easier sell, isn't it, once people can understand what's in it for me. 

 

Melanie Kyrklund 27:47

It is and I was recently asked by someone else speaking at the conference, and she asked me, I don't have buy in to run more experiments, how can I do it? And I had a moment where I thought, actually, I've never had to encounter like senior management and leadership who were totally against what I was trying to do. So it's hard to be able to give someone advice. I'm like, this is exactly how I approached it with leadership, but the one thing I've always done is insinuate myself into processes and teams and sort of get things moving quietly orchestrating things. And that's another way of doing it. And then they can be like, Oh, actually, I do like what you have been achieving. So that's just a bit of advice, insinuate yourself into processes as much as possible and get things moving.

 

Gavin Bryant 28:48

It's always a good approach, isn't it, to try and embed yourself into as many meetings, forums, reviews, decision making processes as possible. 

 

Melanie Kyrklund 29:01

And try and solve problems for people around the business, I think that's when you become quite a valuable business partner. And then, to make some conversations easier.

 

Gavin Bryant 29:11

Let's think about some of those challenges you've touched on a couple of points so far, and that was really helping the organization understand that the transition pathway and the execution plan, what are the benefits for them? And also you mentioned, as far as roles and responsibilities are concerned, identifying strategic opportunities and ideation and some training and capability build involved?  Were there any other challenges or things that you needed to consider during this migration?

 

Melanie Kyrklund 29:53

Yes. So this might be a long answer now that I think of it. So the transition from, let's say, a centralized COE model, and to be honest, we're still hybrid model, because there's still a lot of centrally funded experiments. But of course, it's a journey. The ethos is, is that we're about more about enablement now, so at a sort of highest level, it's a move away from saying, Okay, we run experimentation strategy and what to test and getting experiments out there. And we're going to add a couple of other services now to what we do. And one is the enablement piece, so how to experiment and being able to facilitate that in teams and in other domains. And then the third piece is, how do we make this scale reliably? Obviously, this is going to be a big consideration if you have more teams testing independently. So that's really about thinking in systems and building the data, tech and process governance you need in order to enable scale so that you don't start having tech problems. You don't have teams bumping into each other, you don't have bad data, for example. 

 

So these are all things you need to start preparing for when you're not in charge of all experiments anymore. So those were the sort of the three pillars. But if I think about it more in a linear way, the first step was actually defining how experimentation is done, so really standardizing the experimentation model. But the sort of three or four steps you go through as an experimenter, and what all the constituent tasks and workflows that happen within each part of the experimentation process. And what we did is we took on the experimentation model of our partner agency, click value, which is focused on problem discovery, problem validation, and then goes through to solution discovery and solution validation. So that's step one, and that, of course, underpins the training. Then you build out the curriculum and the training around that experimentation process, and you document all parts of the process as well. 

 

Melanie Kyrklund 32:19

And then the next part is the governance bit. So someone in my team did a great job of defining and making really explicit of who can test where, because you have products and feature optimization, and then you have trading optimization, which is around content marketing and merchandising. And I think that was the natural space, of course, between the digital product and the regional teams. And then we looked at OKRs as well, setting OKRs with teams. So that was quite a shift for us, because we had to have capability, OKRs in place. So holding ourselves mutually accountable to absorbing parts of the experimentation process and giving them the tools and the training in order to do so, whereas previously, we're just focusing on business outcomes and how we could align the experimentation strategy to the teams. So there's also capability building OKRs there, and then we have to find ways to reinforce ways of working. This is the last one. I'm almost done, Gavin. 

 

I thought it'd be a long answer, because culture is also in the way things are done. So if you want people to work a certain way, you have to give them tools and workflows which allow them to do so. So we've been building out our air table capabilities, as well as providing teams with test plans briefing templates where they really outline, how they came to a problem and assess the problem through to some automations as well, and ways we create visibility around experiments and what's going live, which is also important, of course. So yes, there's a lot more to consider. We've created a lot of new work for ourselves.

Gavin Bryant 34:18

Good. It's all positive work though.

 

Melanie Kyrklund 34:21

It is absolutely. 

 

Gavin Bryant 34:24

One of the things that I wanted to ask you a little bit about was resistance to change. As humans, you know that often we don't welcome change with open arms. There can be resistance, and one of the points you touched on earlier was around cost and who's going to pay for this? Were there any other points of resistance that cropped up when making the transition?

 

Melanie Kyrklund 34:48

Yeah, good question. I don't think so, because all teams want to be empowered. I. And experimentation and having the tools to experiment are a great way to empower teams and give them the autonomy to understand their customers and to solve problems for the business and for the customer. So no, I think Kevin was really on board with that, with that change. Now it's more in the detail of bringing it forward and dealing with issues as they come up, and bringing people along-- our colleagues along on that journey. But I think from the outset, there wasn't friction in that change. It's all in the execution. 

 

Gavin Bryant 35:42

Now, one of the things that I was really curious about was institutional knowledge and the retention of learnings and insights from experiments. So pushing further out from that centralized model, how did you manage that process is all of the knowledge, is that still centralized in a knowledge base or how do you retain that really strong library and institutional knowledge once you're pushing out into the regions and different teams?

 

Melanie Kyrklund 36:17

So this is something where we haven't quite nailed yet, but there's a couple of steps we've taken. So for the past at least two years, we have a lot of experiments in air table. So all stakeholders now have access to air table, and they're able to look up experiments in a particular region or on a particular part of the journey, and then click into the experiment and that will have all the visuals within the database and the results. So that's one way of doing it. Now we also have all the experiment reports as PDFs. So what we're going to do now is we have a secure instance of copilot. So that is like Microsoft's ChatGPT or LLM, and as it's a corporate instance, it means our information isn't retained or used to train models. I am exploring the option of building some agents within copilot that would allow colleagues to query that database of PDFs, because that's what it is easily in order to get to the insights. So they could say, for example, show me all experiments which I've run on glasses product pages in the past year, which ones were positive, which ones were negative, so forth. So I think that's where my thinking is going so outside of air table, which is a very good and coherent resource we have for our experiments, it's that building this agents to query the broader experiment reports that we have in a way that's easier for users to come to what they need fast.

 

Gavin Bryant 38:23

So thinking about success, then when you set out on this journey, what did success look like for the transition, and what are the measurable improvements that you've witnessed since making this move to a COE hybrid model?

 

Melanie Kyrklund 38:41

So my initial vision and North Star was to double the number of experiments we run by end of next year by having at least six teams testing autonomously across digital products and regional teams, I think I can still hold that North Star metric insight. Now in terms of tangible improvements, our find and book teams, so they're the team who work on our appointment booking journey, which is obviously the main functionality of our website, is getting customers in stores. They are a bit more advanced in the sense that the product owners there have been taking a lead on experimentation already, since over a year, and there, we've seen velocity double, and all new features are AB tested, so they have feature flags, feature rollouts, so we've gone server side there, and we've integrated with product engineering. So that's really what we're aiming for all [product teams. 

 

So there we've really seen that change happen in terms of velocity and number of features being validated and experimented with. Now in terms of the regional teams, we're definitely seeing movement in the number of refined opportunities and ideas that they're presenting to us. So they're really taking ownership of the ideation piece. I think what's changed also is that within Australia, New Zealand and our Northern European region, as of this year, we have a point like dedicated optimization managers as well. So that shows that there's also like a shift within the organization in terms of putting people in lead positions to run optimization within those regions, and for them seeing momentum in terms of energy and engagement and wanting to make this transition. So I would say I'm still in-- We're still in that phase, nine months in, where we are getting-- putting effort into shifting the flywheel. And you know, the first shift, or first couple shifts, take a monumental amount of effort, but we're starting to see the early signs of that cultural change in having optimization managers in region through to in our more advanced areas, such as the booking engine, the booking flow, really seeing that Velocity and feature validation being at the heart of what the product owners do. So that's it. 

 

Melanie Kyrklund 41:46

Within a year and a year and a half, I'd like to see that velocity pick up in all the regional teams, and also get a couple of more of the product owners moving in that direction as well. But to be honest, most of the experimentation is still done within the Center of Excellence. So I say when we started out, it was 80 to 90% of experiments going through the center of excellence. My end point is just 20% so to shift that so then 80% of the experiments are happening directly within the team workflows. I guess that's a two year journey from where we are now.

 

Gavin Bryant 42:28

On top of seven years to date. So it's nearly a decade of sustained, consistent effort and investment to get to that point.

 

Melanie Kyrklund 42:39

Yes, exactly. That's the reality of working within these large global organizations you're constantly adapting to change, digital transformation happening around you. So yes, it's something that requires patience and tenacity.

 

Gavin Bryant 43:05

One of the things that is really stuck out to me about what we've discussed so far Melanie was I'm not going to-- I guess I'm going to use the word ease at which you were able to make this transition from a centralized to a COE function. But it was everything that preceded the kickoff of this change that made the change much easier, all of the hard work that really been done over the previous five to seven years by the experimentation teams that you know there was a high level of trust there. There was a high level of respect. There was known business benefits and tangible outcomes that have been provided by the team. And the function that it was really a question of, how do we do more of this to amplify quality of decision making even further. So I think that's the real sort of key overarching thing that sticks out to me here. And it gets back a little bit to that point you made about the person the conference asking, you know, I'm bit stuck in being able to progress experimentation further. Yeah, it's everything that is laid down over a period of many years that enables those gateways and those doors to be unlocked, to make those step changes in maturity.

 

Melanie Kyrklund 44:29

Yeah, I think, that's spot on and sometimes you sort of get discouraged because you're not where you want to be, but at the same time not acknowledging all the effort that's gone into getting you to where you currently are. So I think it was very nice for me to hear that comment Gavin, because again, it's allowed me to take a step back and acknowledge all the work that me and my team have been doing to get us to this point, even though we acknowledge that we're a couple of years away from where we want to be.

 

Gavin Bryant 45:10

When I was doing the introduction for the APAC experimentation summit, I used the metaphor of Marathon, and I likened experimentation to a marathon, in that when you're training for a marathon, it never gets easier. You just go faster. And the destination for an experimentation program is a destination you never reach. So it's a timeless journey.

 

Melanie Kyrklund 45:39

Yeah, I think I'm coming to the same conclusion as well. I think that's what always surprised me about experimentation, is the fact that it's always something to do. There's always a new angle, there's always a new challenge. It is just a continuous process. And I'm kind of surprised if you take obviously, big tech after the equation. So the top few percent of companies you're booking, Airbnbs, Microsoft, after the equation. There's a vast number like companies such as Specsavers, we make the majority of the companies out there. And if you look within this big group of companies, we belong to. It is a long journey and a challenging journey. When you speak to other peers within the industry, you always, sometimes think that some of the challenges are only yours and are symptom of the type of organization you're in, but actually the happening across the board, just building that course of experimentation is a long task, yeah.

 

Gavin Bryant 46:50

So, that's a nice segue into my next question. So other organizations who are-- There may be at that inflection point of looking to take that next step, change or leap in their program, maturity and transition from a centralized function to a hybrid or COE based on what you know now, having been through this process, what would be your key pieces of advice for them?

 

Melanie Kyrklund 47:21

Well, of course you have to gauge appetite for that, for the change within the organization, so get feedback from all the stakeholders involved. And I think the second bit is, of course, you have to think about how you're going to fund that change as well. I was in a fairly fortunate position in the sense that I have an experimentation budget that I can use flexibly within the organization in order to gain momentum. So for example, server side experimentation shifted some budget there and then elsewhere, more on the client side bit. So a lot of people will say to you, that's cool, but I don't have the resource, of course, so that's something you're going to have to consider. There might be product teams who are primed to absorb experimentation, because they have design and development resource. So then it's more about getting them to follow the right process and the experimentation model, but then there'll be a lot of other teams who definitely have patchy resources when it comes to following the workflow end to end. So it's not only about the appetite, it's also about the funding and who you can help and how much you can help them with the budgets you have. So if you're trying to embark on this without a budget, then it's probably going to be difficult. You have to secure the funding to help customers on that journey, not only conceptually, but in terms of actually delivering experiments. 

 

Last point; the thing I am aiming for in a couple of years is also like to pull back on the funding, of course. So within a year, the conversation will be like, Okay, your teams are working this way, and it's going really well, and you're seeing the results. You're seeing the psychological impact, hopefully, when your team's being empowered to experiment. But then, like, how can we make this more structural? How can we fund this moving forward? That's the next phase. 

 

Gavin Bryant 49:33

Yeah, so just before we move on to close out with our fun fast questions, is there anything else you'd like to add about this transformation journey that you've been on? Any closing thoughts?

 

Melanie Kyrklund 49:53

Not really, because I think my closing thought was going to be around like the patients required to do this type of work, to instigate this type of cultural change, and how multifaceted the challenge is. So I think I just want to reinforce that, obviously, I've talked about the pillars of work we're doing, I think I've also outlined a lot of the work streams and the processes we're going through. But as you mentioned, it is a multi-year journey, so you need to be patient and tenacious in order to see this through within organizations. But the results are great, if you can get there, the rewards are great.

 

Gavin Bryant 50:40

Okay, let's wrap up with our closing fast four fun questions. These are just four fun questions to round out our discussion today, Melanie. So first one, what are you obsessing about that we should know about?

 

Melanie Kyrklund 50:54

I'm not sure this is fun. I didn't know that was part of the brief, but I am obsessing about a book called "Right Kind of Wrong" by Amy Edmondson, who is the world's leading expert on psychological safety. She has a really interesting framework called the failure framework, which allows you to distinguish between failure types, optimize good ones, build psychological safety, anti-standards. So read one about failure frameworks. I'd say I've been enjoying that a lot.

 

Gavin Bryant 51:29

I did get on to that based on your recommendation before our chat today, and I read a good article from her on Harvard Business Review, the framework that she was talking about there, how there's bad failure, unexpected failure, and then praiseworthy failure. It was a good way to think about it, and also the different drivers and behaviors that can occur at each of those different spectrums. It's well worth a look, and I'll reference these in the show notes for everyone. 

 

Okay, number two, what's the biggest misconception people have about experimentation?

 

Melanie Kyrklund 52:14

Well, it's about numbers, that's about revenue, it's about value. I think a benefit that's really underplayed is the psychological benefits. And I don't know if I'm the only one who thinks this, but I've gained so much by having this pragmatic process via experimentation that I can constantly refer back to. So it doesn't matter how much chaos is going on within the organization around me, I know how to approach finding problems and solving them. And I think giving anyone the chance to be able to work that way is of great value. Being able to navigate uncertainty like that. Just know how to move forward. So I'd say the psychological value.

 

Gavin Bryant 53:03

Yeah, I like that one, it's really the ultimate form of truth, isn't it? It cuts through all the BS.

 

Melanie Kyrklund 53:10

Yeah, absolutely. You can handle almost any situation, I think business situations, I'm thinking like an experimental I think.

 

Gavin Bryant 53:19

Number three, what continues to surprise you most about experimentation?

 

Melanie Kyrklund 53:24

Well, I think this is just an observation, but I get the impression that industry is still very small. Somehow, 15 years on, it seems to be like a small group of people who are talking about experimentation and out and about at conferences. Yes, somehow I find that baffling.

 

Gavin Bryant 53:48

One day we will have the scale of accountancy, doctors, lawyers and dentists. No doubt, still early days.

 

Melanie Kyrklund 53:58

Yes, let's see.

 

Gavin Bryant 54:01

Okay. Last one, number four, what's the one thing listeners should remember from our chat today? You mentioned earlier? Be persistent. Be resilient. Be tenacious. Is that what we need to close out on today?

Melanie Kyrklund 54:16

Yes, pretty much, and insinuate yourself within the business, within processes and help people solve problems, and that will pay itself back.

 

Gavin Bryant 54:29

Fantastic, an excellent way to close out our conversation today. Melanie, thank you so much for your time today, it's been fantastic to learn more about the Specsavers journey, and we really appreciate your time. 

 

Melanie Kyrklund 54:42

Thank you, Gavin, it's been a pleasure, and thank you for your thought provoking questions. I really enjoyed it.

 

Gavin Bryant 54:47

You're welcome.