Article

11min read

Product Manager vs Product Owner: What’s the Difference?

What’s the difference between a product manager and a product owner or are these roles actually one and the same?

If you’re wondering, you’re not alone. The terms are often used interchangeably and not without reason as sometimes the two roles’ responsibilities may overlap especially in small to medium-sized companies. 

However, Product Manager (PM) and Product Owner (PO) are two distinct roles. Although they do share a common goal, delivering products users love, the scope of their responsibilities is actually not identical. 

Generally, a product manager is responsible for the why. They shape the product roadmap based on users’ needs and desires. They are focused on business metrics and on whether the product is going in the right direction on a larger scale.

The product owner, on the other hand, is responsible for creating and managing the product backlog. They are operational and on a deadline. For them, it’s all about Scrum, back and forth with developers, and getting stuff done.

However, the lines between the two do often blur. Depending on where you work, you might be used to a variety of setups. 

What are the differences in responsibilities of a product manager and a product owner?

The short answer is: it depends.Product Manager Experience

Now, I’m sure you’re thinking ‘of course! Isn’t that true for everything?’

But in this case, it really does depend on a lot of different factors. A chunk of this article will go over them in detail but first we will discuss each role to highlight the responsibilities of each one on its own to be able to then understand their differences.

The product owner role

The term “product owner” comes from Scrum- an Agile framework that helps teams structure and manage their work and solve complex problems effectively by adopting a set of values and principles. A typical Scrum team consists of Product Owner, a Scrum Master and Developers with specific accountabilities. 

According to the “Scrum Guide”, the product owner is responsible for “maximizing the value of the product resulting from the work of the Scrum Team”. Consequently, this role is usually found in organizations that have adopted an Agile methodology. 

Among the key responsibilities of the product owner is creating user stories for the development team to implement and ensuring that these stories meet customer requirements. In other words, the product owner advocates for customer needs and represents the voice of the customer to the development team by making sure that the right product is being built. 

Some other responsibilities of the product owner include:

  • Manage product backlog by creating and communicating the backlog items and prioritize them accordingly to maximize value
  • Define and manage the product vision 
  • Understand market and customer needs and turn them into actionable user stories
  • Collaborate closely with cross-functional teams such as developers to ensure that products meet customer needs and requirements. They also make sure that goals are clear and business objectives are aligned with the product vision 
  • Act as the primary point of contact for all stakeholders and ensure they have proper buy-in on all major decisions and strategies 

The product manager role

As we’ve seen, product owners are more tactical in that their focus is on maximizing value through creating and managing the product backlog. Additionally, product owners are more detail-oriented and have more of a short- to mid-term focus. 

Meanwhile, product managers play a more strategic role, which means their focus is more on creating the long-term vision of a product as well as aligning the product roadmap with larger organizational goals. 

They then build the product strategy in order to build a concrete product based on its overall vision that spans across the entire roadmap. Put simply, a product manager manages the entire product life cycle and oversees every aspect of its development from the early stages of user research to the moment of product launch.

The responsibilities of the product manager may vary from one organization to the next but they usually include the following tasks:

  • Conduct user research to determine customer needs to determine product vision 
  • Create and align teams around the product roadmap 
  • Decide what features to build next
  • Supervise teams and projects to ensure the successful launch of the product 

A product manager role is usually more outward-facing in that they speak to customers to define the requirements of the product to be built while a product owner is more inward-facing as they work closely with development teams to ensure the product is being built according to these requirements. 

We can look at product owners as an extension of product managers as they apply the strategies outlined by product managers and transform it into an actionable backlog.

In other words, the product manager decides what products or features to build next while the product owner oversees that developers build these products. 

Do organizations need both roles?

The simple answer to this question would be it depends. 

Various factors will need to be taken into account when it comes to deciding whether you need one or the other or both.

In an ideal scenario, there would be a product manager managing the product strategy and vision and a product owner responsible for executing that strategy. On the surface, it may seem that the responsibilities of the two roles may sometimes overlap as they’re working towards common goals but in reality, their day-to-day tasks differ significantly.   

However, not all companies may have the sufficient resources to have two different people dedicated to each of these roles.

Indeed, many small companies may find that having two distinct roles is not necessary and the responsibilities of the product development process can be carried out solely by the product manager. However, processes may become more complex in larger companies and could require a product manager to manage the product life cycle and outline the overall strategy and a product owner overseeing the development process to ensure the tactical execution of that strategy.

It’s important to also keep in mind that the product owner role is tightly linked with the Scrum development approach. Therefore, it’s typically found in teams practicing Scrum. Product managers, for their part, exist in organizations that aren’t necessarily following a specific approach or methodology. They can operate within any framework as part of the product team while product owners in highly Agile businesses come as part of a Scrum team.   

Nonetheless, many organizations struggle with the decision of whether to have one or the other or both. 

The most important consideration when making this decision is focusing on outcomes and not on the titles. Organizations need to examine their objectives and any weaknesses and bottlenecks in their current processes or structures that may hinder them from achieving these objectives. Such insights are key to building your winning team. 

When to consider a product manager vs product owner

Let’s not forget, there are products of all kinds: industrial products, agricultural products, services, chemical products, fashion and software products. 

In the SaaS martech space, the thinking goes that all products are built the same way and that companies are all structured the same way. They tend to think there is always a Product Manager and a Product Owner behind each feature.

We’ve already mentioned some things to consider when you’re deciding whether to have both roles. In this section, we look in further detail at some of these factors that determine the right setup for your organization.

1. Momentum

The most important criteria is ‘the momentum of the company.’ What does this mean?

How new/old is the organization?

This question is an important one. It is not directly related to the size of the company or to the number of clients, either. 

But rather, the right question here would be: How far or close is the company in relation to a specific “key moment”?

product owner vs product manager ab tasty

What we mean by ‘key moment’ are key phases or events in the life cycle of a company. Some examples include: 

  • the creation of the company
  • its sale
  • new funding
  • tax inspection
  • a competitor’s release
  • a crisis
  • replacement of the CEO 
  • … etc 

All these different key stages will have an impact on how the product development is executed.

And therefore, they affect what a product manager does and what a product owner does.

For example, just after a company is founded, the CEO is presumably the best person to take care of the product roles. They are very often (at least in scalable SaaS companies) the one who wears the hat of PO, PM, QA & Support. It is not a choice or a strategy. It is just what the moment implies.

Also, in general, the way a company’s momentum is handled defines how “well” a company is doing. 

Has the moment happening right now been anticipated, or is it just the consequence of the reaction of the last moment? The more a company reacts, the more “Agile” you can consider it to be. The more moments are anticipated, the more “visionary” you can consider the company. It’s challenging to be both.

2. Size matters

Size here doesn’t necessarily refer to the number of employees a company has. The size of a company could also refer to the number of:

  • products
  • models
  • features
  • scopes
  • price grids 
  • or markets they address. 

Cutting the products in the right “dimensions” and deciding if we split per market, per range or per topic can be a lifelong job, because it changes very often and the faster the company evolves, the more it has to be changed.

Size can also be defined by the number of users, clients, partners, retailers, languages, unit systems,… But also and more frequently, a combination of all the above will tell how many PMs, POs and Heads of Product your organization will need. 

For example, in many non-profits, product managers are based on the donor personas. You have a PM for small donation amounts (say $10 to $500), one for larger ones ($500 to $10,000) and one for those that are even bigger. Why? Because expectations from each persona type differ radically, and being able to cater to each will help the non-profit grow in the long run.

In short, the size of the company (its client portfolio, its product catalogue…) may define what the PM does, what the PO does, and how many of each there will be.

3. The remaining criteria

But these are not the only factors that define how many PMs, POs or any other product-facing people you will need. Many other factors can also be taken into account, including: 

  • number of developers
  • number of designers
  • shipping velocity
  • technical stacks
  • tooling
  • product lifecycle 
  • individual wishes
  • growth potential
  • existing need coverage ratio 
  • legacy ratio
  • human resources policy 
  • the market you address
  • the level of politics installed in the company.

There is some kind of global agreement (at least in SaaS businesses) on the fact that the split between PM and PO is based on: vision vs. operations, projection vs. immediacy, value vs. metrics. But both are sides of the same coin.

In short, no two organizations are identical. In other words, there’s no one set answer to the question of what a PM vs. PO should do. Some organizations will need many, some will need none. Enabling people to work successfully together is hard enough. Copying and pasting a method won’t help – you have to find your own. 

What skills are needed to be a good product manager and/or product owner?

Both product managers and product owners need a good balance of hard and soft skills to carry out their tasks efficiently. 

The necessary skills for product managers include:

  • Excellent communication and collaboration skills
  • A good amount of technical expertise
  • Prioritization skills
  • Good business acumen
  • Analytical skills

Meanwhile, the necessary skills for product owners include:

  • Problem solving
  • Project management skills 
  • Great communication and strong storytelling skills
  • Deep understanding of user data and analytics

The skills necessary for product managers and product owners may overlap especially in smaller companies where a product manager could also be a product owner and vice versa. 

What KPIs are used to measure success for product owners and product managers?

It depends and may vary from one organization to the other depending on their objectives 

However, if we tie them into their individual responsibilities, we can give a general overview of the kind of KPIs that can be used to measure their performance. 

A product manager, for example, has a wide range of responsibilities including the creation and prioritization of the product roadmap along with cross-team collaboration to ensure alignment around the roadmap.  

In that sense, a product manager’s performance is primarily related to product success in line with the overall business goals. Thus, usually product managers’ KPIs should be based on business metrics such as growth, revenue, churn rate and costs. 

Meanwhile, product owners have a narrower role and based on the responsibilities we outlined above, their success is measured using KPIs based on delivery, quality, and internal team satisfaction.

Conclusion

A product owner and manager are great assets to have but having both or two separate people taking on these roles is not a must for every organization. 

At the end of the day, the question is not whether you should have both a product manager and product owner but whether having both of those roles is right for your organization. You will have to look inward to understand what your business actually needs. 

How you decide to structure your teams will depend on the processes you have in place and the kind of outcomes you’re hoping to achieve based on business objectives, customer and company needs.

While we give an overall idea of each role, these responsibilities are not set in stone and they could widely vary on a case by case basis.

What really matters is that you have the right people with the right skills working towards a shared goal, which is building a product customers will love and meets their requirements and needs. 

Subscribe to
our Newsletter

bloc Newsletter EN

We will process and store your personal data to respond to send you communications as described in our  Privacy Policy.

Article

13min read

How to Deal with Low Traffic in CRO

If your website traffic numbers aren’t as high as you may hope for, that’s no reason to give up on your conversion rate optimization (CRO) goals.

By now you must have noticed that most CRO advice is tailored for high-traffic websites. Luckily, this doesn’t mean you can’t optimize your website even if you have lower traffic.

The truth is, any website can be optimized – you just need to tailor your optimization strategy to suit your unique situation.

In this article, we will cover:

CRO analogy

In order to make this article easier to understand, let’s start with an analogy. Imagine that instead of measuring two variants and picking a winner, we are measuring the performance of two boxers and placing bets on who will win the next 10 rounds.

So, how will we place our bet on who will win?

Imagine that boxer A and boxer B are both newbies that no one knows. After the first round, you have to make your choice. In the end, you will most likely place your bet on the boxer who won the first round. It might be risky if the winning margin is small, but in the end, you have no other way to base your decision.

Imagine now that boxer A is known to be a champion, and boxer B is a challenger that you don’t know. Your knowledge about boxer A is what we would call a prior – information you have before that influences your decision.

Based on the prior, you will be more likely to bet on boxer A as the champion for the next few rounds, even if boxer B wins the first round with a very small margin.

Furthermore, you will only choose boxer B as your predicted champion if they win the first round by a large margin. The stronger your prior, the larger the margin needs to be in order to convince you to change your bet.

Are you following? If so, the following paragraphs will be easy to grasp and you will understand where this “95% threshold” comes from.

Now, let’s move on to tips for optimizing your website with low traffic.

1. Solving the problem: “I never reach the 95% significance”

This is the most common complaint about CRO for websites with lower traffic and for lower traffic pages on bigger websites.

Before we dig into this most common problem, let’s start by answering the question, where does this 95% “golden rule” come from?

The origin of the 95% threshold

Let’s start our explanation with a very simple idea: What if optimization strategies were applied from day one? If two variants with no previous history were created at the same time, there would be no “original” version challenged by a newcomer.

This would force you to choose the best one from the beginning.

In this setting, any small difference in performance could be measured for decision-making. After a short test, you will choose the variant with the higher performance. It would not be good practice to pick the variant that had lower performance and furthermore, it would be foolish to wait for a 95% threshold to pick a winner.

But in practice, optimization is done well after the launch of a business.

So, in most real-life situations, there is a version A that already exists and a new challenger (version B) that is created.

If the new challenger, version B, comes along and the performance difference between the two variants is not significant, you will have no issues declaring version B “not a winner.”

Statistical tests are symmetric. So if we reverse the roles, swapping A and B in the statistical test will tell you that the original is not significantly better than the challenger. The “inconclusiveness” of the test is symmetric.

So, why do you set 100% of traffic toward the original at the end of an inconclusive test, implicitly declaring A as a winner? Because you have three priors:

  1. Version A was the first choice. This choice was made by the initial creator of the page.
  2. Version A has already been implemented and technically trusted. Version B is typically a mockup.
  3. Version A has a lot of data to prove its value, whereas B is a challenger with limited data that is only collected during the test period.

Points 1 & 2 are the bases of a CRO strategy, so you will need to go beyond these two priors. Point 3 explains that version A has more data to back its performance. This explains why you trust version A more than version BVersion A has data.

Now you understand that this 95% confidence rule is a way of explaining a strong prior. And this prior mostly comes from historical data.

Therefore, when optimizing a page with low traffic, your decision threshold should be below 95% because your prior on A is weaker due to its traffic and seniority.

The threshold should be set according to the volume of traffic that went through the original from day one. However, the problem with this approach is that we know that the conversion rates are not stable and can change over time. Think of seasonality – i.e. black Friday rush, vacation days, Christmas time increases in activity, etc. Because of the seasonal changes, you can’t compare performances in different periods.

This is why practitioners only take into account data for version A and version B taken at the same period of time and set a high threshold (95%) to accept the challenger as a winner in order to formalize a strong prior toward version A.

What is the appropriate threshold for low traffic?

It’s hard to suggest an exact number to focus on because it depends on your risk acceptance.

According to the hypothesis protocol, you should structure a time frame for the data collection period in advance.

This means that the “stop” criteria of a test are not a statistical measure or based on a certain number. The “stop” criteria should be a timeframe coming to an end. Once the period is over, then you should look at the stats to make an appropriate decision.

AB Tasty, our customer experience optimization and feature management software, uses the Bayesian framework which produces a “chances to win” index which encourages a direct interpretation instead of a p-value, which has a very complex meaning.

In other words, the “chances to win index” is the probability for a given variation to be better than the original.

Therefore, a 95% “chance to win” means that there is a 95% probability that the given variation will be the winner. This is assuming that we don’t have any prior knowledge or specific trust for the original.

The 95% threshold itself is also a default compromise between the prior you have on the original and a given level of risk acceptance (it could have even been a 98% threshold).

Although it is hard to give an exact number, let’s make a rough scale for your threshold:

  • New A & B variations: If you have a case where variation A and variation B are both new, the threshold could be as low as 50%. If there is no past data on the variations’ performance and you must make a choice for implementation, even a 51% chance to win is better than 49%.
  • New website, low traffic: If your website is new and has very low traffic, you likely have very little prior on variation A (the original variation in this case). In that case, setting 85% as a threshold is reasonable. Since it means that if you put aside the little you know about the original you still have 85% to pick up the winner and only 15% to pick a variation that is equivalent to the original, and a lesser chance that it performs worse. So depending on the context, such a bet can make sense.
  • Mature business, low traffic: If your business has a longer history, but still lower traffic, 90% is a reasonable threshold. This is because there is still little prior on the original.
  • Mature business, high traffic: Having a lot of prior, or data, on variation A suggests a 95% threshold.

The original 95% threshold is far too high if your business has low traffic because there’s little chance that you will reach it. Consequently, your CRO strategy will have no effect and data-driven decision-making becomes impossible.

By using AB Tasty as your experimentation platform, you will be given a report that includes the “chance to win” along with other statistical information regarding your web experiments. A report from AB Tasty would also include the confidence interval on the estimated gain as an important indicator. The boundaries around the estimated gain are also computed in a Bayesian way, which means it can be interpreted as the best and the worst scenario.

The importance of Bayesian statistics

Now you understand the exact meaning of the well-known 95% “significance” level and are able to select appropriate thresholds corresponding to your particular case.

It’s important to remember that this approach only works with Bayesian statistics since frequentist approaches give statistical indices (such as p-Values and confidence intervals that have a totally different meaning and are not suited to the explained logic).

2. Are the stats valid with small numbers?

Yes, they are valid as long as you do not stop the test depending on the result.

Remember the testing protocol says once you decide on a testing period, the only reason to stop a test is when the timeframe has ended. In this case, the stat indices (“chances to win” & confidence interval) are true and usable.

You may be thinking: “Okay, but then I rarely reach the 95% significance level…”

Remember that the 95% threshold doesn’t need to be the magic number for all cases. If you have low traffic, chances are that your website is not old. If you refer back to the previous point, you can take a look at our suggested scale for different scenarios.

If you’re dealing with lower traffic as a newer business, you can certainly switch to a lower threshold (like 90%). The threshold is still higher because it’s typical to have more trust in an original rather than a variant because it’s used for a longer time.

If you’re dealing with two completely new variants, at the end of your testing period, it will be easier to pick the variant with the higher conversions (without using a stat rest) since there is no prior knowledge of the performance of A or B.

3. Go “upstream”

Sometimes the traffic problem is not due to a low-traffic website, but rather the webpage in question. Typically, pages with lower traffic are at the end of the funnel.

In this case, a great strategy is to work on optimizing the funnel closer to the user’s point of entry. There may be more to uncover with optimization in the digital customer journey before reaching the bottom of the funnel.

4. Is the CUPED technique real?

What is CUPED?

Controlled Experiment Using Pre-Experiment Data is a newer buzzword in the experimentation world. CUPED is a technique that claims to produce up to 50% faster results. Clearly, this is very appealing to small-traffic websites.

Does CUPED really work that well?

Not exactly, for two reasons: one is organizational and the other is applicability.

The organizational constraint

What’s often forgotten is that CUPED means Controlled experiment Using Pre-Experiment Data.

In practice, the ideal period of “pre-experiment data” is two weeks in order to hope for a 50% time reduction.

So, for a 2-week classic test, CUPED claims that you can end the test in only 1 week.

However, in order to properly see your results, you will need two weeks of pre-experiment data. So in fact, you must have three weeks to implement CUPED in order to have the same accuracy as a classic 2-week test.

Yes, you are reading correctly. In the end, you will need three weeks time to run the experiment.

This means that it is only useful if you already have two weeks of traffic data that is unexposed to any experiment. Even if you can schedule two weeks of no experimentations into your experimentation planning to collect data, this will be blocking traffic for other experiments.

The applicability constraint

In addition to the organizational/2-week time constraint, there are two other prerequisites in order for CUPED to be effective:

  1. CUPED is only applicable to visitors browsing the site during both the pre-experiment and experiment periods.
  2. These visitors need to have the same behavior regarding the KPI under optimization. Visitors’ data must be correlated between the two periods.

You will see in the following paragraph that these two constraints make CUPED virtually impossible for e-commerce websites and only applicable to platforms.

Let’s go back to our experiment settings example:

  • Two weeks of pre-experiment data
  • Two weeks of experiment data (that we hope will only last one week as there is a supposed 50% time reduction)
  • The optimization goal is a transaction: raising the number of conversions.

Constraint number 1 states that we need to have the same visitors in pre-experiment & experiment, but the visitor’s journey in e-commerce is usually one week.

In other words, there is very little chance that you see visitors in both periods. In this context, only a very limited effect of CUPED is to be expected (up to the portion of visitors that are seen in both periods).

Constraint number 2 states that the visitors must have the same behavior regarding the conversion (the KPI under optimization). Frankly, that constraint is simply never met in e-commerce.

The e-commerce conversion occurs either during the pre-experiment or during the experiment but not in both (unless your customer frequently purchases several times during the experiment time).

This means that there is no chance that the visitors’ conversions are correlated between the periods.

In summary: CUPED is simply not applicable for e-commerce websites to optimize transactions.

It is clearly stated in the original scientific paper, but for the sake of popularity, this buzzword technique is being misrepresented in the testing industry.

In fact, and it is clearly stated in scientific literature, CUPED works only on multiple conversions for platforms that have recurring visitors performing the same actions.

Great platforms for CUPED would be search engines (like Bing, where it has been invented) or streaming platforms where users come daily and do the same repeated actions (playing a video, clicking on a link in a search result page, etc).

Even if you try to find an application of CUPED for e-commerce, you’ll find out that it’s not possible.

  • One may say that you could try to optimize the number of products seen, but the problem of constraint 1 still applies: a very little number of visitors will be present on both datasets. And there is a more fundamental objection – this KPI should not be optimized on its own, otherwise you are potentially encouraging hesitation between products.
  • You cannot even try to optimize the number of products ordered by visitors with CUPED because constraint number 2 still holds. The act of purchase can be considered as instantaneous. Therefore, it can only happen in one period or the other – not both. If there is no visitor behavior correlation to expect then there is also no CUPED effect to expect.

Conclusion about CUPED

CUPED does not work for e-commerce websites where a transaction is the main optimization goal. Unless you are Bing, Google, or Netflix — CUPED won’t be your secret ingredient to help you to optimize your business.

This technique is surely a buzzword spiking interest fast, however, it’s important to see the full picture before wanting to add CUPED into your roadmap. E-commerce brands will want to take into account that this testing technique is not suited for their business.

Optimization for low-traffic websites

Brands with lower traffic are still prime candidates for website optimization, even though they might need to adapt to a less-than-traditional different approach.

Whether optimizing your web pages means choosing a page that’s higher up in the funnel or adopting a slightly lower threshold, continuous optimization is crucial.

Want to start optimizing your website? AB Tasty is the best-in-class experience optimization platform that empowers you to create a richer digital experience – fast. From experimentation to personalization, this solution can help you activate and engage your audience to boost your conversions.