Intro

Recently, Shishir Mehrotra (Co-Founder & CEO at Coda) shared some of his insights at a TPF webinar named: Lessons for Product Leaders. In this webinar, he covered various topics.

However, I’ll talk in detail about two of the topics that he touched:

1. Measuring the performance of your product team.
2. Conducting meetings effectively.

About Shishir

Shishir Mehrotra (CEO of Coda)

Shishir Mehrotra is the CEO and Co-Founder of Coda. Earlier, he served as the Vice President of Product and Engineering for Youtube/Video at Google Inc. He also served as the Director of Program Management at Microsoft. He holds a Bachelor of Science in Computer Science and a Bachelor of Science in Mathematics from the Massachusetts Institute of Technology.

Measuring the performance of your product team

When Larry Page took over as the CEO of Google in 2011, he split the company into eight divisions based on business operations instead of functional. Shishir headed the Youtube division. While this decision allowed everyone to move a lot faster, there were some issues that they noticed. One of them was how to evaluate promotions with the new setup. Promotion at Google (termed as Calibration) was done by a committee instead of your direct manager.

Shishir was asked to set up the new Calibration process. After looking at the current rubric, they concluded that the current metric is designed around an increase in scope, i.e. Feature > Feature Group > Product Sub Area > Multiple Sub Area of a Product > Product > Product Line.

However, defining promotion solely based on scope resulted in several issues.

Issues with promotion solely based on scope:

1. Unfair across teams of various sizes: Some groups had only one product like Google Search, while others like Google Ads had 50 products. Because of the large number of projects in a single group, everyone wanted to work with Google Ads and not Google Search.
2. The scope is an input, not an output: It is unfair to evaluate a PM based on the scope which was handed over to them. They didn’t choose it.
3. Devalued experimental projects: Current rubric pushed people to work on larger existing projects instead of new smaller experimental projects. This resulted in some of their best PMs avoiding experimental projects, even though it could grow over time.

While everyone agreed that scope is a useful dimension, Shishir suggested adding another dimension to the rubric. This new rubric was already being used at Youtube for a while. It was called PSHE (Problem, Solution, How, and Execution). Here’s how it works:

Rubric to evaluate your product team

Adding PSHE (Problem, Solution, How & Execution) as another dimension:

• Given: SPSH / Manage Execution: At an early stage of their career, a PM would be handed a Problem (what problem we are trying to solve), a Solution (What is the proposed solution) & a How (what meetings to attend, what documents to write, etc.). They were required to manage the Execution of the given project.
• Given: SPS / Manage How: As they get more senior, they are given a Problem statement & a Solution too. They need to figure out the How:
• How should I set up the milestones?
• How should I set up the meetings?
• What should the cadence be like?
• Which writeups should we do/not do?
• Given: SP / Find Solution: As they rise further in their careers, they are given a problem statement. They have to figure out a Solution:
• What target market to go after?
• What are the different possible solutions?
• How to price it?
• Given: S / Discovering Problems: Finally, you hand senior PMs a Space, and they come back with problems that need to be solved.

In the above diagram, you can see that the Scope dimension is on the X-axis while the PSHE dimension is on the Y-axis. To test this rubric, they marked various PMs in Google on the graph to check where they landed upon the graph. They noticed that people naturally fit along the black line.

Early on, one is growing mostly along the scope axis. Similarly, later in their career also, they are growing mainly along the scope axis. i.e., more significant divisions, larger products, and so on. It was in the middle where they grow vertically rather than horizontally. At this stage of their career, what they are doing in their job is not very different, but how they are doing it. The majority of the candidates for evaluation by the calibration committee were in this middle region.

A trough of Disillusionment:

Trough of Disillusionment

Shishir named this region: “Trough of Disillusionment.” It's because this is often the area where product leaders get stuck as there is no way to see that they are growing. However, if they evaluate it from the new lens, it’ll become more evident that they are growing but on a different dimension (i.e., PSHE).

But is this applicable beyond Product Managers?

Although this idea was designed for evaluating Product Managers, the concept can be applied to every role: Designers, Developers, Marketers, Salespeople, etc. To give you an example, if you asked your sales team who would you say is the best salesperson:

• An obvious answer might be the one who has the best quota attainment. But if one looks deeper, they can see that this may not be the case as higher quota attainment is all about negotiating. If you set your quota low, you’ll get good quota attainment.
• However, the correct answer would be the one who can take on an unknown path, go after new markets, identify the problems, define solutions, and figure out how to execute them. As you might have noticed, it has the same “Problem > Solution > How > Execution” pattern.

Conducting meetings effectively

Mandatory Dilbert Comic (26th Aug 1992)

I’m sure you must have landed up in a meeting (many times?) where the lack of structure resulted in it going nowhere. Shishir counters that it doesn’t have to be. Referring to a book: Game Storming, He explained that if you define certain rules to any meeting you can switch it from being a no-outcome play to a productive and effective game, a game where people abide by certain rules and have a clear goal and outcome.

From the book:

"Imagine a boy playing with a ball. He kicks the ball against a wall, and the ball bounces back to him. He stops the ball with his foot and kicks it again. By engaging in this kind of play, the boy learns to associate certain movements of his body with the movements of the ball in space. We could call this associative play.

Now imagine that the boy is waiting for a friend. The friend appears, and the two boys begin to walk down a sidewalk together, kicking the ball back and forth as they go. Now the play has gained a social dimension; one boy's actions suggest response and vice versa. You could think of this form of play as a kind of improvised conversation, where the two boys engage each other using the ball as a medium. This kind of play has no clear beginning or end; rather, it flows seamlessly from one state into another. We could call this streaming play.

Now imagine that the boys come to a small park and that they become bored simply kicking the ball back and forth. One boy says to the other, "Let's take turns trying to hit that tree. You have to kick the ball from behind this line." The boy draws a line by dragging his heel through the dirt. "We'll take turns kicking the ball. Each time you hit the tree you get a point. The first one to five wins." The other boy agrees and they begin to play. Now the play has become a game; a fundamentally different kind of play."

The authors suggest that meetings are not different than games. They have the same dynamics. You can design your meetings like a game, resulting in effective meetings.

Here are a couple of examples:

Upvote/Downvote Game:

Instead of asking attendees what do they think of ideas, let people upvote/downvote on ideas. Here’s an example of how it might look using Coda (you can do it in any other way too, however).

Upvote/Downvote Game

Here are some benefits of why you should do this:

• Equalize the audience: Every meeting has that one person who will skew everyone’s opinion as they speak the loudest. This results in a biased view of the larger group. Instead, let the group vote on it instead. This will result in equalizing everyone’s opinion. Thus collecting an unbiased view of the group.
• Discuss what matters: If used for answering QA meetings, it can be an effective tool to order in decreasing order of what matters most to the group. You have high confidence that what matters to most people is being answered first.
• Well-formed questions: By asking everyone to write it down, there is a high chance that the question is well thought out and elaborate. It reduces the chance of wasting everyone’s time in a meeting.

"Pulse Check" as a way to remove group-think:

This tool can be used in many different ways, like to know what everyone is concerned about, where everybody stands, etc. Here’s an example of how it looks using Coda.

Pulse Check Game

Some of the benefits of using this game:

• Avoid group-think: It removes an implicit bias to go along with what the first person said.
• Equalize hierarchy: Voicing what you are feeling against your manager/boss can be intimidating. Sharing a private pulse can help get an honest pulse check.

To read more about such games you can check out the following resources:

• Game Storming
• Shishir’s guide to distributed teams
• Gokul's S.P.A.D.E. Toolkit
• An inside look at how Figma ships product
• Claire's Offsite Toolkit
• How to Run your Distributed Staff Meetings like Zapier
• Career Development Doc

Conclusion

We looked at two unique lessons that Shishir shared in his talk. Many more exciting topics were discussed in his talk. I highly recommend you go check out the full webinar by following the link below:

‍