To Get OKRs Right, Forget About Features

Did you know that features are just but a lot of guesswork of you being able to address the problem? If you want to get OKRs right, you must first understand and focus on the problem, know how the customer measures that problem, and forget about features.

In this episode, Jenny Herald is joined by Mike Pilawski, the Chief Product Officer at Smallpdf where he leads product, design, and data teams. Prior to joining Smallpdf, Mike has held leadership roles in product, engineering, marketing, and data at Typeform, Leanplum, Vungle, and NativeX.

Listen in to learn how OKRs can work great for you if you focus on the problem and if you know how your customer measures the problem. You will also learn about the two camps concerning OKRs and performance management and why teams should celebrate OKRs. 

“The more mature and experienced people you have on the team, the less processes and control you need.”- Mike [21:33]

Show notes

• [1:48] Mike on his love for building and data, plus his journey in the tech world.
• [8:12] The different reasons that lead to technical or organizational debt.
• [15:44] The important factors to consider when designing organizational blueprints.
• [23:56] How to build values on your company by focusing on choices people have to make.
• [29:30] How to measure the customers’ problem; and then give your team the freedom to work out the best solution.
• [35:28] How not to isolate engineers/product developers from the customers.
• [40:19] The importance of KPIs as a metric plus Mike describes his team’s OKRs review process.
• [46:47] Understanding the two camps concerning OKRs and performance management.

Transcript

Mike [00:00]: The biggest problem that organizations do, especially large organizations, is they start creating layers between the teams that work on features or solving problems and the customers. I am a huge, huge proponent of not having dedicated research organizations within the larger organization.

Jenny [00:22]: Hi, and welcome to Dreams With Deadlines. A podcast where you'll hear real stories of trials and victories in business. I'm Jenny Herald, VP of product marketing at GTM Hub. Our mission is to prevent organizational hypocrisy. Inspired by the proven objectives and key results goal setting methodology, GTM Hub offers the most flexible results management system for mission driven organizations. Check us out at gtmhub.com to learn more. Mike Pilawski is the Chief Product Officer at Small PDF, where he leads the product, design and data teams. Prior to joining Small PDF, Mike has held leadership roles in product, engineering, marketing, and data at Typeform, Leanplum, Vungle and Native X. In this episode, we chat about the dimensions of organizational blueprints. How one goes about marrying up agility and OKRs for product and engineering teams, and the two camps concerning OKRs and performance management and more. Let's jump in.

I'm so stoked to have you on the show. It's weird. Thank you for joining us on Dreams with Deadlines today.

Mike [01:40]: My pleasure, my pleasure.

Jenny [01:42]: So who's Mike? Can you share a bit about yourself and your professional journey, whatever you fancy.

Mike [01:48]: First of all, I like to think of myself as a builder and a data nerd. I love data, I love product, and I love the smile that you see on customer faces when you actually solve a problem for them and they see something happening and it's like changing their life, positivity. My journey to this, well, I was born in Poland during communism. Which is really interesting, because during communism in Poland there was almost nothing to do. And I was lucky enough to get a computer at a very young age. We didn't have games, so I actually had to get newspapers with game prints and essentially type them and record them, and that's how I started to learn how to code. And then internet showed up with IRC and I started talking to people in the US and people in the UK and all over the world.

And I discovered how a fantastic world exists outside sort of my city, outside of Poland. And I think this inspired me to go into technology. Because technology was this thing that enabled me to communicate, to experience other people's lives, to learn, even though, you know, I was living, initially, behind sort of the iron curtain and then in a country that was transforming itself from that. So that was kind of the inspiration. I studied business and I studied engineering. So my first real job was at Nokia, and when you show up with two degrees like that, they generally say you should work in product. I don't even remember if it was actually called product at Nokia because they changed the title a couple of times. But essentially, yeah, you work with the engineers, you build something, come up with a concept for a product, and that's how my journey started.

And actually the first thing I was building was like a tiny, tiny feature on a Nokia device, internet radio. So I can say that I put the internet radio on a phone. That was back in 2003 or 2004. But the crazy thing about the Nokia scale back then, is we built this feature, it was a really, really simple app. And I think three or four months later, we saw something like 20 million active data users. And it just blew my mind that, you know, you build something that was so easy and just because of the scale of the company, suddenly 20 million people worldwide are actually using your product every day. Using it, experiencing it and so on.

So that's how it started. I ended up at Nokia working on different entertainment products. And then the last two years I was involved with a lot of MNA. And the MNA was really exciting because I actually went to see startups from insight. And I had these observations that really shocked me. Because at Nokia, sometimes it would take a 400 people team to build the simplest feature. And then you go to a startup that is actually out-performing you and the entire company is 30 people. And you get this revelation, wow, first of all, how is it possible? It's not that they are somehow having, you know, 240 hour days. Or that, by any definition, they are 20 times smarter than we are. So what is it that is unique about startups that they can actually do that? And I guess that interest pushed me out of the company and into the startup world.

So then I spent, essentially, the last decade in different companies. First in, I would call it, maybe app tech or mobile app tech. A company called Native X, a company called Vungle, then at Leanplum in marketing technology. Most recently at Typeform. And as of next week, I will be starting as Chief Product Officer at Small PDF. So I've been through a lot of these companies. I think everyone was unique. And also, you know how it is with startups and scale ups, a lot of changes. So also, I can say that over that time I held multiple roles. I mean, product is always very close to my heart, but I managed engineering teams, I managed data teams, marketing, advertising studio at some point in time. So I've been in sales, business development, I've been in many different roles. And I think this also is really interesting because you can actually see how product is perceived from those different sort of roles and from different angles and you can learn a little bit more about your job from the perspective of your stakeholders.

Jenny [06:18]: What a great idea or thought. Because I have switched roles myself in the course of my career and never really thought about - well, maybe I was kind of doing it and I didn't realize it. Observing product from the stakeholder vantage point. And it is different. Okay. Wow. I mean, that's a whole podcast on its own, honestly. But I think we're going to transition a little bit into some stuff that you shared with me about how you think about building and scaling teams and agility and all this stuff. I'm excited to jump into it. So you have a deck that you kind of shared with your teams called designing agile and scalable product engineering organizations. And when I flipped through it, I was like, wait, there's a blog post that I've read. And I think Steve Blank did this way back in the day, where he says, organizational debt is like technical debt, but worse.

And he actually writes that organizational debt is all the people, the culture, the compromises that you make just to get it done in the early stages of a startup. And just when things should be going great, that organizational debt can turn a growing company into a chaotic nightmare. Growing companies need to understand how to recognize and quote, unquote, refactor organizational debt. And then if you read Scott Belsky's book, like the Adobe CPO EVP of Created Cloud, he has a book called the Messy Middle, I imagine you have seen this. Do you think that this issue is really evident in just startups or have you seen this in different teams? Because you've seen the Nokia's of the world and you've seen the Leanplums and the Typeforms. What have you seen out there and what are you observing?

Mike [08:13]: Yeah, so I would first say that organizational debt, like technical debt, is not always the result of you actually doing something wrong or not doing something. Technical debt often shows up because, well, technology, for example, changed. Or market requirements changed. I'll give you an example from Leanplum. One of the reasons why a company like Leanplum could exist was because the old school or the previous generation marketing platforms couldn't deal with data in real time. And they couldn't deal with data in real time because when they were actually designing and architecting it, the idea of actually handling that volume of data in real time simply didn't exist. And the customers also didn't have a need for it because they were rarely getting anything on mobile. So responding to you based on your context, based on your circumstances, was not a big idea.

You were mostly receiving email and you read it whenever you felt like it's a good time to read it. So a change in the market and change in technology essentially created a technical debt - which talking of Adobe, I believe at one of their conferences, they actually shared, it will take them up to two years to essentially redesign the entire architecture of the system to go through real time data. And I'm sure they have amazing engineers, but essentially, it's a decision who's validity, let's say, expired over time. And I think it's similar with organizational debt. I would say every organization changes, and I think that the good sort of points to recognize is you'll first change at 15 people, then you'll change at 50, 150, and then probably at around 500 to 750 and so on.

Partially, there's this concept of the Dunbar number, which relates to how many people we actually can store in our memory and what level of sort of intimacy in our memory we can have with them. So we cannot trust more than 150 people. That's just what it is, right? So you have 150 people, usually politics start to happen in the organization. Usually you start seeing us versus them, department versus department, silos and so on. But that has something to do with the fact that you actually have too many people for you to believe that all of them are good, all of them are trustworthy. And for you to get to know all of them, to the point where you can build that relationship. And you know, at the earlier stages, the same things happen. So you have to start introducing, sometimes policies, sometimes processes, sometimes rituals, sometimes tools, that help you solve those problems.

And I think that's one element. The second element that organizations often forget is with every new hire, your culture changes. There is the culture you're designing, and there is the real culture, which is really just the subset of behaviors of every single person in the company and how they make their choices. So whenever you introduce a new person and that person has a different set of values or different behaviors, it will influence others, your culture is evolving. So over time, whatever you design will change and you have to bring it back. And I think, you know, there are also other reasons. I would say COVID is a great example of, regardless of scale, we had organizational debt. I mean, a great example is right now I'm in Barcelona, and some of the more traditional companies in Barcelona measure people's performance by how much time they actually spend in front of their computer at work.

Yes, different culture, different traditions. We're not talking about the startups of Barcelona, we are talking more about the traditional companies over here. But it still exists over here, people still - even the law over here actually says that one of the reasons for you to be fired legally is that you have to be missing work by a certain amount of minutes, coming late to work regularly. So it's actually built for more of how we were thinking about, you know, building teams in the 80s maybe than now. But imagine such a traditional company suddenly has to go completely remote. Because all the businesses, except for the essential businesses, were essentially shut down in Barcelona, for example, between March and June last year. So suddenly, three, three and a half months, you have to completely rethink how you measure performance. And you have to retrain your manager so they can actually bring some modern models to it.

So that would be one example of change. I think new regulations is another example of change. You know, laws change, employment laws change, but also privacy laws change. Those things will create certain organizational debts in the company. And then, you know, also values change with every generation. I've read so many blog posts about how terrible millennials are, and now I'm reading blog posts that the next generation is terrible. And they're not terrible -

Jenny [13:07]: Back in my day, we didn't do any of that.

Mike [13:11]: Right, exactly. They just come with a different set of values, often an upgraded set of values. But the organization has to adapt to the younger workers that come in. So you have all of those elements, not to mention entering new markets. We have a team, for example, in China at Vungle with close to 40 people.

And you have to start learning that there are certain behaviors and certain practices that are normal there that not will be normal in Silicon Valley and vice versa. For example, quite often we would see their engineers come to the office and then sleep at their desk. And the first reaction will be, people would essentially say this. They essentially, whenever they are tired, they would take a nap of half an hour at their desk and then they would continue working. And they would continue developing. But the perception of that person might be very different culturally. And if you don't adapt your company to actually recognize that in different cultures, this work can happen slightly differently, you'll have different mismatches. So I think, you know, all of those things are there, there are always all the reasons compared to the dynamics. You mentioned Nokia, great example. Nokia was amazing in terms of competing on hardware. One of the best value chains in the world. Suddenly, it has to compete on OS. Suddenly the release cycles should be monthly instead of every two years. And the organization is just not ready for it. [14:34 Inaudible] eventually fails and sells for nothing. And essentially, you know, it's a big transformation, but it's a transformation of values, of processes, of rituals. Even performance reviews had to change accordingly. And it's not easy to do it.

Jenny [14:52]: I can only imagine. Oh gosh, there's so much to unpack there. But we're going to kind of press on because there's other things that I really wanted to get your thoughts on. For example, you sent over an article from Berkeley, Dimensions of Organizational Blueprints for High Tech Startups. And they outlined various dimensions, attachment, selection and coordination control, and then five blueprint archetypes. I saw that you added that in your deck. Can you talk through what those employment blueprints were or are, why you shared that at all, and then how you think about what blueprint makes sense for whatever organization you're working with? Because I thought it was a really interesting way to frame how you develop, maybe adjust, create your organization over time.

Mike [15:45]: Yeah. So I would say maybe a starting point is, I don't think this is - so this article I think is now almost 15 years old.

Jenny [15:53]: It is older. Yeah.

Mike [15:55]: But I think it's the only one which is short enough to actually share with teams that actually remind you that you're making a lot of implicit decisions or no decisions without even knowing about them. So let's go through the three criteria, right? What is the basis of attachment and retention? Why do people come to work for you and why do people stay with you? It's a great question. Let's take Leanplum, for example, because we actually studied this at Leanplum. People came to work for Leanplum for very interesting and impactful projects. We had 30 engineers and we were handling, I think, about 10 billion events per day. For most engineers, that level of scale, knowing that the team is only 30, is exceptional.

Hey, there's only 30 people and you handle that much, that means I'm going to learn a lot. It's great. But they never stayed for that. Half a year later when you ask them why you're here, they said, well, because I love the other people. I don't know if we're going to be as successful as a company, but I love the other people so I want to be here. It's not maybe what you want to always hear, but also, what you want to hear. That essentially you created an environment in which people really want to work because they love the community and everyone else around And I think it's a question of whether that model is a sustainable model as you're scaling. It's also a question of whether we actually plan to do it this way, or it just came out.

When we read the article, we actually at least went and actually checked those things. But before they were happening sort of automatically. And I think you have those three things, right? Which is people come for money. Generally, we know some industries are very specific, we also know some companies that pay extremely well, attract people because they come for the money. You know, the upside is you can attract really good mercenaries. The downside is when something goes wrong or someone else pays, the mercenaries are gone, right?

Jenny [17:51]: Where are your evangelists, right?

Mike [17:52]: So you can scale quickly, but then you can lose a lot of people quickly, too. Yeah, the qualities of work is a tough one, because how many companies actually have the scale of Leanplum or Google, right? Google is great at attracting you for a specific project that has huge impact and you'll go there. But then I've talked to a few friends who left Google and they all shared the same story. My project finished, the next thing that was happening was not as insightful, so I decided to move on. And so all of those things have positives and negatives and you have to think about it. And then if you are creating sort of a community, you have to understand that in a community, community is often like a family. You don't fire your uncle because he's 10 minutes late or underperforms a little bit.

So it changes also the culture. You cannot have a tough sports team mentality, for example, and at the same time, say, okay, we all love each other and we are all here for each other and so on, right? So there are some tradeoffs to every single element. And I think a similar criteria for selection. A lot of people will tell you, we interview people for skills, experience, their potential and their culture. And I would say it's bullshit. You will always find an organization that will compromise on culture if they think that they've found the skills necessary. Or will never compromise on culture, even if they found the best person with the best skills. Similarly, the potential, hard to estimate, right? I mean, what is the potential, right?

Jenny [19:25]: Like how do you measure that? How do you track that, even? Sure. Yeah.

Mike [19:28]: So you have to come up with very specific questions, testing intelligence and so on. Which not every company is ready to do, and not every company feels comfortable doing. So then your potential is estimated based on how you described your last job, that doesn't seem like a pretty good proxy, right? So essentially all of those things are something that is left unsaid and people are essentially building some kind of a blueprint without knowing what they're building. And I think that's kind of the third one. And then the means of coordination is also an interesting one. You know, at 15 people, you don't need means of coordination. Generally, everyone knows what everyone else does and whether they contribute or not. When you hit 50, 150, what's going to happen? Are you going to hire a lot of managers and have them control everyone, and look at what they are doing?

Are you going to create a lot of processes? Are you going to start doing team reviews? And some people will feel upset about their judgment or talk about favorites getting the best votes because people like them. You know, you have to figure out the model and essentially get people to that model. Because if you start hiring people that like three different models, you'll actually have a culture clash within the company and people pulling in different directions. So at minimum, it's worth reading the article to figure out what are some of those dimensions and I would maybe add a couple of others that I typically think about. So one of the things is just national culture and values. It's something that generally people don't spend that much time thinking about in Silicon Valley because it's kind of a mix of different cultures and a lot of nationalities.

And on an average team, you're going to have 20 different nationalities. But as you, for example, come in to Spain or move into Zurich, you start thinking about the culture actually affecting how people work much more because it's more homogeneous. Or, for example, if you work with teams in China or teams in India, where essentially one nationality dominates the team. I think the other part is - I've read this new book about Netflix and it talked about talent density. And essentially, it's a very interesting concept because the way they describe it is, the more mature and experienced people you have on the team, the less processes and control you need because these people naturally -

Jenny [21:43]: They've got self-governance and self-direction. Totally agree.

Mike [21:48]: So now the next thing is, if you're thinking about building a certain type of organization, you will have different talent densities for that organization. Next one, industry verticals, right? You're selling to enterprise? Very different model than if you're selling to SMB or selling to customer or to consumer, right? If you're selling to consumers, quite often you can miss deadlines as an engineering team because there's no one keeping you accountable if something ships four weeks later. But try to miss a deadline if it's in the contract with your customer and see how well that will go with your sales team. So you have to adapt and you have to think. And then I would say maybe the last one that I think is also important is the level of competition or the dynamics of the market. You know, is this a winner takes all market? Are you in a market like Amazon or in a market like search engines and so on where one player will really dominate because of economics of scale or brand or network effects? Or are you in a market where there will be hundreds of players, like, let's say, the new clothing brands, and you can push for profitability at more sustainable pace. That will also affect what model you need.

Because if you're pushing for really, really fast growth, the model of company as a family might not be compatible with that, for example.

Jenny [23:06]: Oh, absolutely not. I have seen this myself.

Mike [23:10]: So I think that's kind of the interesting point. And I think people just need to consider this. There's no one size fits all. And you just need to understand your company and build from there.

Jenny [23:23]: Fair enough. So let's assume that you've gotten your blueprint. You've thought through how you're going to design your organization, and things are humming along. At some point, hopefully, the organization is starting to develop its set of values; how they operate, how they behave. When you're thinking about this - in terms of the engineering or product teams you've led - what things do you believe they should value? And how do you come up with those shared values together, if at all?

Mike [23:56]: Yeah, I would say I mostly am negative of values as a concept. And I know it sounds strange, but in most companies I've seen values mostly being the posters on the wall that people pass by and sometimes if you ask them, name our five values, they will think about the one or two that they like and they'll forget the other three. And then the company tries to change the values and they introduce a new concept of values. Maybe they'll build some nice acronym to make them memorable. But, you know, values are values. Values are what you do, how you make decisions. So in a sense, a company has a set of implicit values. If you want to have values for your company, just ask people what they think the current values are. And then, if you don't like some, then you have a problem and think about how you can change one or the two. But most of you should [24:51 Inaudible] what already happens that works for you.

Because otherwise, you're essentially trying to change someone else's values, which, if psychology will teach you, it takes about a year on an individual basis to change someone's values, truly. And if you're trying to change organizational values, it's almost impossible without replacing a significant amount of people. So that's essentially the challenge, number one. I would say the other part is - but there is value in codifying. Once you find out that your team has three or four values that they actually share, codifying them is great because it serves two purposes. First of all, when someone new comes to the company, they can read those values and decide whether they actually embrace those values or not, and should join or not. And the second thing is, after they join, they are reminded if they have to adapt one of them, what that value is, and essentially their decision to join should be the decision to adapt those values.

I would say, in the deck I shared eight values. I would say, maybe, the things that I've learned from that time is first of all, eight is too much. People will remember three or four. If you have eight values, you might as well have one hundred. No one will remember them, you'll remind them every quarter, and people will remember their three or four, the ones they like. They'll forget the ones they don't like and it'll not work. I think the second thing is making the values not obvious. I remember there was a value, quality and scalability building. You know, operating at a large scale, it's very important for us to ship quality. And I had a conversation with my colleague at Typeform, Mateas, who once told me, what's the point of this value?

And I'm like, what do you mean what's the point of this value? He said, well, so would you put that reverse value on any team? Would you say bad quality built in? Or would you say we ship without quality and fix it two years later? No, you wouldn't. It's obvious. So you're actually not providing any directional value to people, you're just stating the obvious. You might as well write, we all breathe air, and it will be equally valuable. If you're doing values, focus on choices. If there are two choices that are diametrically different, then your values should encompass the right choice. If you, for example, say, no politics - which was one at the company Leanplum - it's also a stupid value. Which company would say, I want you to engage in office politics every day. That makes no sense.

Everyone knows that politics are frowned upon. Everyone knows that your colleagues will not like you for the politics, and maybe there'll be some bigger repercussions. So stating it doesn't really help in changing the people or giving them direction. So I would say, from my perspective, if there are three or four things - for example, if you have a growth team that needs to be shipping a lot and they fail a lot, maybe for them, having a value that says, we actually ship [27:49 Inaudible] is a good value because it reminds them that two out of three experiments will fail and what they should emphasize is velocity of experiments and then fix problems after the experiment. And for that growth team, this value makes sense because it's counterintuitive. Usually, we expect to have great test coverage and deliver high quality code before shipping. So at that point in time, I'm talking about quality makes sense. But generally saying, we should ship quality code, it's just whatever, everyone knows that. It's not very helpful. So I'll say -

Jenny [28:21]: It doesn't help a person make the decision. That's ultimately it, right? That's the bottom line. If I am confronted with choosing A or B, I will choose B because, value. And it helps keep everyone kind of in that same camp, if you will. I want to spend time talking about this, and there are so many people who have asked me that I really have lost count, how does one actually go about marrying up agility - big A, small A, doesn't matter - and OKR, specifically, for product and engineering teams? I want you to share with us, maybe, some of the general aspects of how to think through this, as well as the tactical bits, if you're willing to share even examples. Because, oh gosh, there's so many people, maybe who have approached you, and they're like, we're so accustomed to shipping, isn't that good enough? Why do we have to do OKRs? Why do we have to be business outcome focused? I feel like our bosses are pressuring us to showcase why we should tie to these organizational goals, blah, blah, blah. So talking to someone who's actually done this is incredible.

Mike [29:30]: I think two things that really help get the OKRs right is, first of all, to forget about features. Features are bad.

Jenny [29:39]: Oh my gosh. Features are bad, my drop.

Mike [29:43]: No, because features are essentially guesses, or they have a level of uncertainty in them. They are a hypothesis on you being able to address a problem. But until you address the problem, you don't know that you address the problem. And you have an idea that a particular feature or solution or something, we'll address that problem. But you know - or if you don't, then you really have a big problem as a company - what is the biggest problem your customer has? You're addressing that problem, right? So let me give you an example. I think I read two or three years ago in one of those HubSpot's state of the market for marketeers, a statistic that an average marketeer spends six hours a week copying data from one system to another because they are now using 10 to 15 different tools, and essentially to keep all of them in sync to make sure the right customers are getting the right messages. They're essentially copying those CSV files from one to another. So for a team, you could say, hey, well, that's a real problem. That's a huge waste of time. How do we solve this problem?

Let's set a goal of solving this problem for marketeers and reducing this time, let's say, to an hour per week. Now, how can you solve this problem? Well, you can build an integrated cloud and try to build every feature, so your integrated CRM will essentially solve all of those problems. Here you go, Adobe. Here you go, Salesforce. That's your strategy. Or, you can build a CDP and say, well, we have a centralized data system that will have integrations to every other platform, and now you have segments. Or you can say, well, why would you need that? You just essentially need to have connectors to all of those different platforms that sync this data, and then you build Zapier or [31:25 Inaudible] or one of the connector paths. You have the same problem, you have three different solutions.

If you are a startup, probably solution number one, meaning building every possible feature doesn't work for you, but if you're a Salesforce, it can be very interesting to consider. The other two are potentially approaches you can take. And then, as you progress closer, you'll have a higher number of integrations. So now, segment is integrated with 7 out of 10 tools, so I only spend an hour copying. Now, segment added two more integrations, I spend half an hour copying. And you can progressively see how you're solving this problem with a metric over time. At the same time, you're not telling the team what they need to do. They have complete freedom to test, to do different hypothesis, to design different products, to choose order of integrations once they find the right product and so on. And essentially that leaves it agile, but at the same time, pretty definite.

So for me, OKRs were great if you focus on the problem and if you know how your customer measures the problem. So what are the customer outcomes for measuring the problem? The customer says, I need to save five hours. That's how you measure the problem. If the customer says, I need to look great in front of my boss - which quite often happens with, for example, marketing tools, right? Your users are not your buyers. Your users want to impress their boss so they can get promoted. You might actually start thinking, how can I help my users be promoted by, for example, showcasing the results of their work to their boss through different reports and dashboards and so on. And then getting feedback from customers asking, hey, are you actually feeling like your boss knows better what you're achieving? So all of those different things can be done if you have a metric to measure it, and you give a team the freedom to work with customers, the data, and figure out what is the best solution.

I would say, one last thing that really works well, especially in the OKR structure. Most companies I know either have annual OKRs or quarterly OKRs or both. Which essentially means you have about six sprints, if you're doing biweekly sprints in a quarter to hit your OKRs. If you have a lot of ambiguity over what's going to work and move the metric, you can actually focus the first sprint on some very rapid testing or very rapid research. There's three or four different alternatives. Choose the one that is the most valuable and continue working with it. Or if you can spend the first two or three sprints, ship something small to a subset of users, test whether it works and you still have time to ship something else. So essentially what it does is it promotes experimentation, it promotes the minimum viable product or minimum lovable product thinking among the product managers. And it also gets everyone on the team involved. Because what happens is, when engineers know that they are testing, they are choosing different solutions, they actually start to be involved with this as well. They are starting to look at the data, they're starting to listen to the customer's feedback because they also don't like to throw away their work. So they are more incentivized to actually work in an experiment early on.

Jenny [34:31]: How do you - or would you advise any of these product and engineering leaders to help their teams kind of overcome this shift? Because I hate to be so kind of brutal about it. I remember talking to a senior leader and they're like, you know what Jenny? I find that engineers are often very introverted, probably even anti-social. They just want to code. They don't want to have to report, they don't want to have to be held accountable for all of these businessy things. They just want to sit there and do their job. And then now, someone's come in - manager, product manager, whatever, and saying, we've got to align and we've got to do all of this stuff. How would you respond to that person or that line of thinking?

Mike [35:28]: I think there are two things that make the work meaningful, right? On one side, you could say, okay, there is a bright level of challenge and the right level of skillset, and you have the conditions for flow and you're solving difficult problems while learning and progressing, and engineering job encompasses all of that. But there is a second element that makes your work meaningful. A broader impact on community, broader impact on the people you know. When you see that your work actually benefits someone else, it starts getting a very different meaning. It's something more than the code. It's something that you can actually go back home and say to your wife or partner or someone else, hey, you know, this thing that we ship, I actually learned that people are now sleeping better. Or I've learned, actually, from a customer who used to spend Fridays doing [36:11 Inaudible] that he can actually spend time with their kids, isn't that great? We helped them.

And I think, you know, what I've learned over time is the biggest problem that organizations do - especially large organizations - is they start creating layers between the teams that work on features or solving problems and the customers. I am a huge, huge proponent of not having dedicated research organizations within the larger organization.

Jenny [36:41]: Really? Interesting.

Mike [36:43]: Yes. If you have a UX researcher that gives a power point to engineers, then you've lost the empathy. Now they are numbers on the PowerPoint. There are no human beings, you haven't seen their faces, you haven't seen how it affects their lives. It's good to have a research organization that work with the teams, but not instead of the teams talking to customers. But to facilitate, to help them, to make the research better. It can be even a dedicated organization, but embed those people with teams. Have them actually bring the engineers on site.

You know, at Leanplum we shipped engineers to Singapore for two weeks to actually sit at the office with Grab, to understand the problems they have, so they can actually see first, directly, how they're affecting their lives. And they, for example, learn one particular problem. We didn't have the ability to schedule experiments on the platform. And some experiments had to be scheduled in the middle of the night because they might be happening on a different continent. So people were waking up at 2:00 AM to go to our platform, press a button, start, and then they were going back to sleep. Now you hear that on the slide, that's one thing. You meet the person who tells you that every Tuesday they don't sleep well because they wake up at two in the morning to press the button, you go and code the button. You don't even have to be told that. You just go in and write it.

And I've seen that many times, that essentially what happens is when engineers experience and talk to a customer, the PM is almost not necessary. Sometimes things happen just magically. You come back the next day and the team comes to you and says, oh, you know the thing that they complained about? We fixed it overnight. Like, what do you mean you fixed it overnight? Oh, we stayed in the office, we ordered some pizza, now it's solved. Like, wow. The last time we talked about it, I heard the estimate is two sprints. Well, yeah, but you know, it's kind of like, we understood the problem so we could solve it quicker. And it's like a very different type of motivation because you are actually helping another human being. So from my perspective, there's nothing that can replace them with customers. Of course, codifying this, making sure the interviews are properly run, analyzing the results, it's very helpful to have experts for this, but you cannot isolate those who create from the people who actually will be using the product.

Jenny [39:07]: I couldn't agree more. Goodness. Like the best organizations I've ever worked with were the ones where you took rotations and answer support tickets, or you actually watched the footage or you talk to the customer and they've walked you through what they were doing. And then you kind of felt embarrassed for yourself and for your team for having failed them. Like, it’s something that happens. Intrinsically you're like, oh, well, I did this and I made their lives worse. I have to fix it. And they will. It's incredible. I couldn't agree more. Kind of on the lines of metrics, right? And tracking and looking at the data. Wanted to ask you questions about KPIs. So what place do KPIs hold for you and the teams that you have led? Obviously, it seems like you track them. You talked about it in the very get-go that you're a data nerd. You love this stuff. I do too. And then what's your practice of reviewing those, let's say, in concert with your OKRs? I'd love to be able to dig deep on the mechanics of what this looks like with you and your teams and folks who are delivering the solutions to the customer.

Mike [40:19]: Yeah. And so I would say, first of all, OKRs are really important if you're growing and you want to give autonomy to the teams. There are essentially three reasons for that. First of all, KPIs allow you to align the teams around the things that they have to be focusing on. They give them a few metrics and you know what they are working on, you know what problems they're solving, and that creates [40:42 Inaudible]. The second thing, it creates autonomy because it provides continuous feedback. And that's the important thing with metrics. You cannot have metrics like MPS or retention that the team cannot move or will take six months to see you. You need to have leading metrics, preferably ones that change week to week, or at least every couple of weeks or once a month so the team can actually see that what they are doing is making an impact or not, and it's taking them in the right direction or not.

And I think those two are sort of essential for teams. And I think the third element that is also important about metrics is their objective. In the end, you know whether what you achieved, you achieved it or not. People would argue, this is beautiful, this is ugly, this will work, this will not work. But in the end, yes, people spend now an hour extra with our platform and they actually engage with this feature a lot as a result of that one extra hour. Okay, it works. People like it, people want to engage with it, right? So essentially these are the three elements. Generally, the way we work on them is we start with our north star metric, which, for example, at Typeform, it would be the total number of responses for a period of time, right?

So Typeform enables companies to use forms or run surveys or collect data. And based on conversations with customers, essentially what they told us is the value they get from the platform is directly correlated to how many responses they're getting. We see also in our data, that there is a strong correlation between the two and as you stop getting responses, you're very likely to [42:14 Inaudible]. And eventually, we also aligned our pricing to this. So it became essentially the growth metrics. So if you're, let's say, collecting a million responses per month, you're not paying the same thing as someone who's collecting one hundred responses per month. So as we deliver you more value, you're also more willing to pay for it. So essentially you have one metric that unifies the company goals, the customer goals, and they feel comfortable with the pricing around it because they see they're getting more value as it scales.

Then you go one level below and you'll say, okay, what drives the number of responses? Okay. Number of customers, obviously, number of forms or surveys they are sending, number of respondents per survey and the conversion rate, so how many people start versus how many people finish. Okay. So you have the four sub metrics. And then from there, you can go one level below and two levels below, and essentially that third level is usually where the team [43:08 Inaudible]. Because at the level of number of customers, if you are a 300 people company like Typeform, there is no team of five people that can own that metric. It becomes an effort between product, between marketing, between sales and so on. Similarly, conversion rate is usually not a thing that one team will own, because it will be affected by the experience of the people who are actually using the mix of customers, the type of surveys that are being brought.

But two metrics below, you will find things that impact. Let's say - a good example would be retention. Retention is correlated to the number of integrations someone has. So if someone has forms that, for example, are integrated to MailChimp, to send email responses that they collect, and to Google sheets to send the results. And maybe they've also integrated it to Slack to essentially get notifications whenever they get a response. And let's say, a couple of other integrations like that, their environment, their workflow is very efficient. They take advantage of every response they are getting, and I would say at around 45 integrations, they almost never chart. Because they are getting so much value from the product. So now, you can have a team that actually owns the responsibility for the number of integrations. They will work on building new integrations by analyzing, let's say, Clearbit data and looking what other platforms our customers have that we are not integrated with.

Then we'll look at discovery. They will work with the marketing team to make sure that customers, as they become aware of Typeform and Onboard, know what integrations are already there, and they know how to request new integrations. And essentially, they have a very nicely scoped mission that they can own. They can see a metric, let's say number of integrations per X number of customers, or total number of integrations - even better metric - per month that is being added. And they can track it, they can get feedback and they can see if they're going in the right direction. And if they integrate with a platform that nobody uses, then we'll know that too, because the metric will not move. So essentially, it very quickly aligns them and gives them a mission and metrics that they can control. And they don't need me to give them feedback and tell them you're doing a good job. They actually know it, that they have the numbers.

And now you ask the second question, which is, how often they should be reviewing it? And I would say that very much depends on also the cadence of the metric, but at minimum, I see teams - first of all - looking at this every two weeks, because you have biweekly sprints. You have sprint [45:39 Inaudible], sprint retrospective. Sprint retrospective, what changed with our key metrics? Okay, this changed. Okay, what did we ship? What happened that the metric changed? What should we do next? So you have this natural cycle. And then at the end of the quarter, you're reporting essentially what you have achieved by saying, okay, the adoption of this particular system inside the adoption of integrations increased by 4% or whatever the metric is.

Jenny [46:01]: There are two camps, it seems, when it comes to OKRs. Those who believe that it should be a performance management tool and the other which says, no, divorce it. People will sandbag. Do not include this in your performance evaluation, it's a bad idea. But you also want to be able to incentivize people, right? To be able to care about the delivery of their results. And as we mentioned earlier in the blueprint section, people are motivated by all kinds of stuff. Some people by money, some people by the challenge of the work, some people, something totally different, like the team. I do this because I feel duty for my team. Where do you stand on the recommendation or advocacy of OKRs as part of performance evaluations or reviews and why?

Mike [46:48]: So I would say, if you make OKRs the central point of performance reviews, two things will happen in my experience, because I've been in companies that have done that. First of all, you'll have a lot of sandbagging. And the idea that an OKR, people will be aggressive in setting it and will put stretch goals and they will go for 68% nonsense. Everyone will set it so they always hit one hundred percent, because nobody wants to get fired, nobody wants to have problems and nobody wants other teams to know that this has happened. But there is a second element to this, which is product development is a risk-taking exercise. No matter how much research you do, you always have a hypothesis that this particular solution will solve this problem. As I said before, you will truly know whether it's solved it, after it has solved it.

So essentially, what you're doing is you're telling your team, don't take big risks, minimize risks. Do things incrementally, do things that are obvious. Because only that way you will have full control of your outcome. Because if your outcome is to move a metric by 10%, you might say, okay, there's two scenarios. I can move it by 2% every quarter by working incrementally, or I can try this one crazy idea that can move it by 20%, but might move it by zero, and it will take two months. You would say from pure statistics that if your outcome is 20% or zero in two months versus 10% after 10 months, the right approach is to take a big risk. But if you fail, you might get fired. Okay, then I won't do it, right? Because suddenly now, it's actually a very different incentive behind it. But having said that, I think it's good if teams celebrate OKRs.

And one model that particularly works well, and we use it both at Vungle and at Leanplum, was a model where at the end of the quarter, each team essentially presented what they built, how it impacted the customers and then we let teams vote which team they believe have delivered the most value to our customers. You cannot vote for yourself, obviously. And we didn't have an organization where people would vote for their friends. I think, also, because it was teams voting on teams, it kind of removes this, like, a couple of people ganging up to make someone else win. And essentially, there was a simple prize. We would give each team one thousand dollars per person to organize some team building activity. So it's positive, you know, teams will go to a concert and have a dinner together, or maybe they'll fly to LA for a weekend from San Francisco and do something fun.

But what was really interesting is, in a couple of circumstances or situations, I actually had to remind the team to take advantage of the prize because the prize didn't matter as much as the fact that they got recognition from other teams. And it became such a big point that nine months after I left Leanplum, one of the teams called me to tell me that they finally won this prize for the first time. And this was nine months after I was already gone, but they were so proud. Because at the beginning, they were coming sort of at the lower end, getting a [50:08 Inaudible], and finally they [60:09 Inaudible] one quarter. But it was something else they were craving. They were craving respect of their colleagues. And we have to remember that performance evaluation - essentially the role of that is to do two things. Tell you how well you're doing in terms of feedback. If that is associated to the salary, then usually you are not really interested in negative feedback and you're really going to dispute it and you're not going to absorb it, right?

And the second thing is to essentially reward people who are putting in an effort, right? People who are actually following on the values and the things you want to reward. Now, I don't want to reward people who got lucky because the experiment worked out, but I do want to reward people who put an effort and they try and they improve. So if you build performance reviews around the effort, around the improvement, around the progression, and separately set some kind of a fun, gamified activity that actually cherishes the customer value, then you can achieve both goals without necessarily having to tie them into an OKR system.

Jenny [51:16]: Thank you so much for your time. And I really am glad we met, and were able to talk about all of the cool things that you've done, all of the things that your teams have done, what you've learned, how you're sharing that with the world. Thank you so much. I appreciate it. I hope you have a really good evening.

Mike [51:33]: No, it's my pleasure. As you know, very well, I follow GTM Hub from the very, very early days. I'm seeing the growth. Fantastic company. Good luck. I think you're on a mission to do great things for other companies and enable them to really perform well. And great podcast. It was a pleasure to be on.

Jenny [21:52]: Thanks a lot.

That's it for this episode of Dreams With Deadlines. Thanks for listening. If you liked today's episode, please subscribe and share. Show notes can be found on gtmhub.com/radio. If you want to learn more about our product and services, head out to gtmhub.com. If you have questions that you'd like answered on the show, shoot us an email at radio@gtmhub.com. Tune in next time.