Building a product roadmap that works

Natalie Nedre | 14 min read

Real-talk time from RealtimeBoard: Building a product roadmap is a complicated process. As with many other business processes, we would love for building a roadmap to be simple and straightforward, but it is often littered with hidden pitfalls, which is unfortunate because a good roadmap can mean the difference between success and failure for a start-up.

Poorly conceptualized roadmaps can lead to catastrophic failure in a product, and if you fail with a product, do not expect that your customers will turn a blind eye to it. That is why any mistake in product development can either significantly set you back or even finish your story before it really has a chance to begin.

In 2014-2015, even though our company was doing well, our Product team determined that our roadmap didn’t work as well as it could.

So we wondered: Are there best practices we could implement that ensured sustainable growth in our product roadmap?

The answer is yes! Now, we want to be clear that there can never be a 100% perfect solution to any problem, but the results we achieved after a year show that we found the right direction for our case. That is why we decided to share our experience and the approach that helped us to:

  • Cover the majority of roadmapping gaps that we have previously had;
  • Establish clear communication and revision processes;
  • Increase team engagement;
  • Enhance collaboration with a constant conversation between in-house and distributed departments;
  • Make the product faster, friendlier, clearer and more reliable.

In this article, we are going to describe the signals that show a product roadmap doesn’t work, what to do once you notice the signs, how the RealtimeBoard roadmap looked before and after our revision experience, and how you can benefit from our story.

How do I know that the product roadmap doesn’t work?

When attempting to determine if a product roadmap doesn’t work, we found that there are six major issues that crop up. If any of these are true of your roadmap, it should be an automatic red flag!

6 signes that your product roadmap that doesn't work

The goals are not measurable

It is simple. If there is nothing to measure then how do you know whether you achieved a goal or not?

There are no results in the first 6 weeks for the startup and 6 months for the company, and you don’t make any changes in a roadmap

Roadmap components are usually determined by the timeframes. And the average checkpoint is 6 weeks for a startup and 6 months for a company according to Paul Adams, VP of Product at Intercom. If this period passes by while the intermediate goals are not achieved and actions are not reviewed, it is a signal that there is a problem.  

The product roadmap changes every week

This is an opposite problem to the above. Too many changes accelerate indecision, which in return prevents the organization from achieving its strategic goals. Try to find the balance!

Roadmap formula components are unbalanced

It is somewhere near impossible to ship a good product with a set of new mind-blowing features while avoiding feature requests from existing or potential customers or disregarding bug tickets and quality performance. Although the roadmap formula changes over time, it is incorrect to focus on one component and neglect the others continuously.

The team doesn’t understand or share priorities from the product roadmap

A product roadmap allows you to achieve goal alignment for the entire company if created and communicated properly. It means that the document should let everyone know not only “What” to do, but the roadmap revision process should also give the answer “Why” it needs to be done. Running a collaborative roadmapping workshop with relevant employees can be an option to engage everyone, leverage ideas and stimulate open discussion. When everyone knows why he does something, the overall team becomes more motivated and achieves a synergistic effect.

Product roadmap is groundless

It is great when management has a vision, but if that is the only criteria used to prioritize the features in the roadmap, it may become over-optimistic and not realistic. Ask yourself what were the grounds for including features into the product roadmap. Inspired by the Josh Porter’s approach to designing new products, we have determined that a perfect product management decision lies somewhere at the intersection of Metrics, Vision, Empathy and Customer feedback.

A perfect product management decision

A perfect product management decision lies somewhere at the intersection of Metrics, Vision, Empathy and Customer feedback.

What do I do if the roadmap doesn’t work?

Considering that a product roadmap is integral to, and practically the heart of, product development, isn’t it time to give up or start from scratch if everything screams that this plan is not successful enough?

What do I do if the roadmap doesn’t work?

Definitely not! Scrapping an entire roadmap because it is currently unsuccessful is only an option for those tough nuts who treat a product roadmap as a hard and fast plan, and who refuse to adapt. Hopefully, since you are reading this guide to fixing and improving your roadmap, that description doesn’t apply to you! If you are the kind of person who is interested in learning from your failures and improving your roadmap, then read on!

The first thing to keep in mind is that there is always a reason why your roadmap failed, and if you can determine what the root of the problem is, then you can begin to fix it. Explore all the factors that have impacted your plan and don’t be satisfied with the first idea that comes to mind — continue to dig past the initial, easy stuff; think about the whole range of influencing factors: external or internal environment, managing approach, staff unengagement, poor knowledge or lack of experience, low flexibility, lack of resources, etc.

Once you have extensively examined the factors involved, it is time to really dive deep into the issues in order to find a root cause — we suggest using the 5 WHYs approach. This visual tool will help your team to focus on seeking answers to problems, rather than trying to place blame, which is not productive and can be harmful to team unity.

Product Roadmap Review Process

The good news is that, by starting problem-solving analysis, you have begun to get back on the right track! The bad news is that you won’t really have an opportunity to confirm that you dug to the true root of the problem until the next roadmap revision, so you will need to be patient, and allow the process to occur naturally.

During the next revision, focus on the two main points from the initial round of revisions: the problem and the root cause of the problem. Ideally, they should be decreased or eliminated. If there are still problems, it means that you haven’t truly discovered the root of the issue, and you will need to go through another round of problem-solving from the beginning. However! Do not let this discourage you! Like we said before, the fact that you are reading this means that you are someone who is interested in learning from your mistakes. Don’t worry if you don’t immediately solve every problem in your roadmap; it is all part of the learning process.

How do we know this? Because we have a “map” of this “road,” so to speak. We experienced this journey with our own roadmap, and we put a lot of effort into improving the Product Management process. Roadmap revision helped us a lot, so we’d like to share our case from the beginning, to give you greater insight into how RealtimeBoard discovered the problem with our roadmap and managed to solve it.

RealtimeBoard case study: the problem

RealtimeBoard's logoUnsurprisingly, when RealtimeBoard was making our first steps into the market, it was much easier to prioritize goals and manage product development because we were mainly focusing on our primary target audience and our vision, so our roadmap was much simpler.

However, after we acquired 100K users, it became more challenging to add features in the roadmap, because our customer base expanded, and with it, the needs and desires of that customer base grew exponentially.

RealtimeBoard is essentially an online whiteboard, which means that it is potentially useful for thousands of unique use cases. We were overwhelmed by the options! Before we sharpened the focus, it was really hard to prioritize what to do next, as we had too many ideas and too many ways to implement them. If you are a startup too, you should know research and testing are the best ways to find a product/market fit. We knew this, but the question then became how to prioritize thousands of ideas when we were comparatively new to the market and didn’t know what would work and what would not!

The failed attempt to revise

Since implementing the new features looked less rewarding, we decided to focus on something that we felt was easier, and which would immediately improve the quality of our product, so we worked on improving the not so reliable experience of existing customers. Unfortunately, that’s why in 2014 our roadmap looked more like a backlog full of detailed information, speed complaints, and bug reports. We were so focused on trying to fix what was wrong, we forgot to consider why these things were an issue, and why they needed to be fixed. A product roadmap is not, and should not be, a backlog!

Read more here: Why a product roadmap is not a backlog.

The better solution

Do we need to say that this was completely unsustainable and absolutely not something we could continue to do if we wanted to succeed? Thankfully, we realized this, and we conducted a 5WHY exercise. We found that the root cause was that the process required a manageable methodology that could cover new features development as well as improvement upon the old ones. Searching for a solution, we stumbled upon Inside Intercom — a blog with lots of outstanding articles on product development, and all the pieces of the puzzle came together. Luckily, you can now find all those articles merged into an ultimate guide to Product Management by Des Traynor’s team.

In 2015, during an annual roadmap review, we decided to apply Intercom’s approach because the ideas described in the ebook seemed helpful for finding the balance between quality improvement and new features delivery. After almost a year of new roadmap implementation, we are ready to share our story and provide the lessons we learned that have positively influenced our product management process!

The process of building a product roadmap in RealtimeBoard (Intercom’s approach)

As we mentioned earlier, there is a formula for improving your roadmap that includes several components: new ideas, features to help scale, quality improvements, customer problems etc. If we simplify this formula even more, there are essentially two ways to improve a product–either add new features or improve existing ones. Since each process is important, we cover them both in our product development. Below is a separate look at both processes and how they were reflected in RealtimeBoard’s Product Roadmap:

Part 1. Existing features

Product evaluation via features audit

To identify which product features are the most and least engaging we used Kissmetrics data to track adoption and usage frequency of every product feature.

Note: There were separate analyses for Free, Premium and Team accounts features.

Building a Product Roadmap: Product evaluation via features audit

According to the results, there were three top features, while others required improvements in adoption and frequency. This then led to the question of why were these others adopted less, and what kind of experience did our customers have using those features?

In other words, the quantitative data required a qualitative analysis. We started a survey via Freshdesk and Intercom messages to analyze those features regarding value and quality. Live discussions allowed us to get feedback from users who had firsthand experience and to learn why a customer did or did not use certain features.

Building a Product Roadmap: features audit

As you can see, we maintained the color coding for each feature from the previous diagram (green, light green, pink and red) while reorganizing the experience by negative, positive and neutral perception. Fortunately, almost all features from the red section didn’t receive specifically negative feedback (45-92% of neutral feedback), so it wasn’t necessary to kill any features, we simply had to make them quantifiably better in order to increase the adoption rate and frequency.

Action plan for features’ improvements

Determining the necessity for improvements is one of the prerequisites for a step-by-step action plan and roadmap. But before assigning the tasks, we had to establish the direction we needed to take for each feature and set clear expectations, so we started by separating our features list into three improvements groups and crafted our first improvements graph from the results of the features audit.

Value (Deliberate) improvements Frequency improvements Adoption improvements
Grey arrow Red arrow Blue arrow
High-performing features that you know how to make even better Features that are mostly relevant for a particular job, so you want all job representatives to use them on a regular basis Features relevant for the majority of your users but applied only by small group
Connection lines, sticky notes, collaboration in real-time: all our customers use and like them, and there was opportunity to add significant value. Frames, wireframes, templates: the majority of our customers used them infrequently, but using it more would be of benefit to them. Search, link sharing, comments, export: those helpful features hadn’t been quite adopted, while several changes could make the feature more attractive and useful.
Make improvement to features based on the MAJORITY of user feedback. “High risk – high reward” improvement. Apply a four-elements pattern by Nir Eyal, author of Hooked: Trigger, Action, Reward, Investment. Apply 5WHY approach to remove experience obstacles for current customers.

Building a product roadmap: Action plan for features’ improvements

After we determined the direction we were going to take for improvements for each of our features, it was the right time to transfer actions to the backlog while adding new features to the roadmap. But before we began adding new features, we had to finalize our vision for the product’s future.

Part 2. New features

Jobs-to-be-done

When the features have not been invented yet, you don’t even know their names and functionality. There are two most popular approaches in supplying new features: User stories and Job stories. (As a side note, these approaches are usually positioned as opposites, but we believe that the differences between them are mostly semantics. In terms of value, both approaches are useful, because ideally they are both based on solid user research and user interviews.)

As we have previously mentioned, we started our roadmap revision based on the methodology developed by the Intercom team. Since they strongly advocate for the jobs-to-be-done approach, we decided to give this method a try too.

We apply Jobs-to-be-done approach as well as use Personas for journey mapping.  

First, we reviewed the target audience, trying to find a sharper focus — active and payable teams. Since the product is suitable for several use cases, we grouped the audiences by areas and conducted an online survey in order to define the jobs our target audience is trying to accomplish via RealtimeBoard. The quantitative data from 48% of our users allowed us to specify the Job stories, which were additionally proven by almost 50 face-to-face and Skype interviews.

The results that we received were then separated by groups and we mapped them as Job stories using the following template:

When …(something happens), I want to …(do something), So I can …(get expected outcome).

Building a product roadmap: brainstorming new features

The middle column contained potential new features that we then examined from the point of value for the customers. We removed the features whose expected outcome didn’t correlate with at least one of three worthy signs as listed below.

Signs showing that the feature will be welcome by the audience New features
Taking out steps Creating stickers from the spreadsheets in one click, bulk add feature
Reducing costs / time or money spent Search on the boards, new team account structure
Removing limitations in the workflow JIRA, Slack, Dropbox integrations

When the first cutting-out was finished, we were ready to finalize the list by administering the acid test.

Acid test

The acid test is a list of questions any new feature must score straight yes’s on. If it does not, it is rejected. Previously, we used three simple questions for assessing new features:

  • Will it make a better product?
  • Will it serve all the users?
  • If we don’t implement it, will our customers leave?

Currently, we use 10 points (developed by the Intercom team) that are even more difficult for each new feature to pass. However, if it does, you can be sure you are much nearer to a success.

The acid test is a list of questions any new feature must score straight yes’s on.

Once we had winnowed down our list of potential new features to the strongest contenders for actual user desirability, the final step was to figure out how best to add them to the roadmap–for this, we used an ROI test.

ROI test

An ROI test is the last filter for the features before being added to the roadmap. It is a graph where remaining features are mapped according to the level of effort required and the value received by the user.

ROI test is the last filter for the features before being added to the roadmap.

The results that it showed helped us to arrange the features in the roadmap so that they added more balance in the roadmap formula:

  • Anything in the light grey area has the lowest priority in the roadmap.
  • Dark grey area features require increased support during release because any bugs or  questions could easily provoke irritation from the customers.
  • Light blue area features have the first priority in the roadmap — their development takes time while customers are expecting the results, so the team should be ready not to miss the deadlines.
  • Features from the light green area should be scattered across the blue area features to let users feel you are constantly shipping new pieces of product, not making them waiting for a long time.

Part 3. Building product roadmap

Do you remember the problems we described in the first part of this article? Before this experiment, our product development strategy was responsive rather than predictive, and it was difficult to prioritize target audience demands.

One of the main advantages of the new approach is that sufficient time is devoted not only to building a product roadmap, but to preparatory steps too. Anticipating and addressing issues before they come up ensures that you will rarely face surprise difficulties, leading to a smoother and more pleasant experience for both you and your customer.

Building a product roadmap

At RealtimeBoard, we offer a workshop for developing a roadmap that invites representatives from the main departments to the discussion. Uniting relevant employees at the workshop will help to build transparency, which is key for personnel motivation during further implementation. Although it is almost impossible to finish the roadmap during brainstorming activities, it will be easier to adjust this raw plan according to the resources you have, shared responsibilities, KPIs and environment.

1 Features audit: usage Prepared by Product Manager and presented to the team as results of the surveys
2 Features audit: performance Prepared by Product Manager and presented to the team as results of the surveys
3 Creating improvements vision Prepared by Product Manager and presented to the team. Can be discussed and updated a bit.
4 Getting ideas for deliberate improvements Team brainstorming under facilitation of Product Manager
5 Getting ideas for frequency improvements Team brainstorming under facilitation of Product Manager
6 Getting ideas for adoption improvements Team brainstorming under facilitation of Product Manager
7 Collecting job stories Prepared by Product Manager and presented to the team
8 Costs and benefits analysis Prepared by Product Manager and presented to the team
9 Acid test Team brainstorming under facilitation of Product Manager
10 ROI analysis Prepared by Product Analyst and Product Manager and presented to the team
11 Building a product roadmap Product team under facilitation of Product Manager

As you can see, our roadmapping workshop involves the Product Manager at each stage of the process, and utilizes them as an extension of the Product team, because how the product is going to be developed requires immersion into the supportive areas like development, design, monetization, etc.

download-13

Pre-made EDITABLE board with all templates described in this article

button-2

You can edit and customize templates for your product — just save the board!

Conclusion and lessons learned

Since the beginning, RealtimeBoard has performed pretty well, and in 2015 reported 675k signups. But the growing popularity of the product made it difficult to set priorities and define what features to add to the product roadmap.

Knowing that our roadmap formula was unbalanced, we decided to start an experiment and apply external experience, which helped us to cover the majority of gaps that we have previously had.

RealtimeBoard, Customers' feedback

UX consultant Yannis Marcou, Directors of the documentary about Silicon Valley startups and Simon Tomes, Co-founder of Qeek (a realtime test management system) via Twitter

Creating an analysis-driven roadmap, through increased communication and new revision processes, allowed us to align the goals for the entire company, increase team engagement, visualize future success, enhance collaboration, and keep product meetings brief and constant while maintaining a constructive conversation between departments.

A roadmap board mapped in RealtimeBoard is shared within the company, allowing the team to share questions or ideas. A widely communicated roadmap helps to easily engage all employees, including remote ones: the roadmap is communicated during a dedicated meeting using voice chat and real-time presentations in RealtimeBoard. Each team member has access to the roadmap board at any given time from any place, and we believe that having the plan in everyone’s PC/Mac, tablet, or smartphone is helping us achieve results.

All the changes definitely made the product faster, friendlier, clearer and more reliable. Since we began our journey to improve our roadmap, we have learned a lot and have made significant changes over the past year. Our current roadmap formula looks like this, which is a direct reflection of what we wanted to achieve:

Product Roadmap Formula

However, since a roadmap should be like a living, flexible organism, and we aim for continuous improvement; hopefully next year we will be able to add more value to innovative ideas and scaling opportunities.

We hope that you can learn from our experience and that the lessons we learned might be helpful for you, but keep in mind that there is no single right approach for building a product roadmap. Since we are always surrounded by a highly-changing environment, before turning to any new roadmapping approach, you’d better analyze what worked well and what was wrong with your previous one. Leveraging the problems you faced in the past will allow you to create an effective and customized solution.


Since our new approach positively influenced not only the product but also our business processes, team, budget and overall growth, we were inspired by the outcome to share our case and make the experience applicable for other startups. We have added a pre-made demo board that will save you hours if you decide to review or build your product roadmap based on Des Traynor’s e-book.

Note: demo board describes a demo scenario, which serves as an example. You can easily copy it to your space and edit as a regular board.

What to read next:
Over 1,000,000 project leaders, marketers, designers, developers and creatives trust us worldwide.
49D45371-3741-47B6-A405-E7F59FC490F8 Created with sketchtool. CBF1CF60-720A-4C6C-9EAF-AAB7DCF790B3 Created with sketchtool. 8284589B-8B7E-4A34-B044-B30C6E0413E4 Created with sketchtool. pdf Created with Sketch. 9758F27C-DEBF-4E3B-A60D-64FC34F0846A Created with sketchtool. FE695667-7811-47C2-A48F-232F5268CA04 Created with sketchtool. BC8C23D0-913C-4B78-A357-47DD3050A246 Created with sketchtool. 3F252687-0750-4634-B414-6E1FCF7080FB Created with sketchtool. 91908614-38FB-46A0-B6F0-DF62FA1A7127 Created with sketchtool.