Assumptions Are a Dangerous Wager
Failure to validate hypotheses explains why 75 percent of all startups go bust. As pointed out by Steve Blank, renowned author, business advisor, and academic, too often founders assume that consumers will “get,” like, or need their product.
But developing a new product, potentially in a new market, startups are essentially operating on a series of unknowns that they can only hypothesize about.
According to the Lean Startup Movement popularized by Blank and his former student Eric Ries, startups should not execute a business model, but rather continually search for and optimize one. Creating a product that people will find valuable requires more than just a founder’s intuition. It requires a business to listen to what potential customers actually want, and build it through incremental, validated development.
Meet Polymail
This dynamic was not lost on Polymail CEO and Co-founder Brandon Foo, who has adopted Lean Startup methodology for the software development cycle of his collaborative email tool. Polymail, a YC Alum, is a tool designed for businesses to use email more efficiently for customer interactions, tracking communications, and internal messaging.
With 700k in seed funding, Brandon and his Co-founders, Brandon Shin, and Shahan Khan used prototyping, customer feedback, and iteration to build a loyal user base early on. Today Polymail has grown to more than 25,000 active users. Borrowing Lean Methodologies of (build ⟶ measure ⟶ learn), Brandon and his team have made customer development central to product engineering and management:
It’s important to always be in a constant learning process. At all stages throughout the product lifecycle it’s important to continuously validate what you plan to build next, making sure that your perspective is aligned with the actual market needs, and what users are actually looking to do, and the problems they are actually having.
-Polymail CEO and Co-founder Brandon Foo
In an exclusive interview with Codementor, Brandon revealed how Polymail has incorporated user feedback at every step of the product development lifecycle. By doing so they have become a staple tool in the workflows of their 25,000 strong user community.
Share Prototypes to Validate Your Ideas
Don’t just build a product. Build a community of users to help you develop it.
You can spend time and energy to rapidly build a feature full product. But what happens when you launch, and your product goes flop? You are back to square one, or maybe negative one since you have probably burned through your runway. To avoid this, you should be validating your development by sharing prototypes.
Before developing a more complete version of your product, you need to confirm that you are actually solving the right problem. Your solution doesn’t matter unless it is a shared user pain point. To validate your idea in the early stages, Brandon offers these words of wisdom from Facebook, “move fast and break shit.” Foo explains that they launched the first alpha prototype of Polymail on Product Hunt as soon as it was possible to be used. They began collecting feedback from real users, and iterating from there. A short time after launching an alpha to Product Hunt, Polymail received the 3rd most upvotes on the platform up to that time.
We incorporated our users into our entire product development process very early. We could get user feedback on basically everything we were doing. We were able to iterate very quickly from an early stage based on that feedback. Users [could] play an interactive role in the product development lifecycle and communicate directly with the founders and engineers.
In the early days of Polymail, user-engineer feedback occurred via a Slack group created for that purpose. With their Product Hunt success, Polymail had begun building a loyal user base of people who were happy to give honest feedback. Aside from Slack, Polymail also used email surveys, and regularly conducted user interview calls. Brandon describes the user feedback as crucial to their product development: “It’s tremendously valuable, and they are doing you a great service, essentially providing you with free information.”
Free is always good, especially when it helps you make money.
The Fallacy of Features
In our talk with Brandon, he emphasized the fallacy of over-leveraging resources to build a “very polished” version before it is released for testing. “You think your product should be really great, but it’s probably not going to be since you haven’t been collecting feedback.” Remember, the M in MVP stands for Minimum — don’t expend valuable time and money building a ton of features that people may not want or need.
In their new book Hacking Growth, Sean Ellis and Morgan Brown remind us that product market fit, or the “Ah-ha moment” as they refer to it, is often found by eliminating features, not adding more. Instagram is a stand out example. Instagram was originally launched as a multi-feature tool called ‘Burbn’. Burbn allowed users to check-in, plan future check-ins, earn points, and share pictures. But the tool felt cluttered and was difficult to use, and wasn’t really taking with users. After digging into user data, founders Kevin Systrom and Mike Krieger scrapped all features except photo sharing, repackaging Burbn as Instagram. After throwing features overboard, Instagram took off. Everyone knows the app is now immensely popular, boasting 400 million active daily users, and 95 million photos and videos shared every day.
Get Outside and Get Validation
The No. 1 rule of Steve Blank’s The Startup Owner’s Manual is “There are no facts in your building, so get outside.” Once you have a working prototype or MVP out there, momentum to continue forward will be dependent on the team’s ability to respond to user feedback. When you “go outside” what you are really looking for is customer validation of your design, features, usability, and overall value proposition. If you don’t find it, you need to iterate or pivot.
For iterating beyond MVP, Brandon’s recommendation is in line with Lean Methodologies: “develop a rigorous process around experimenting, testing, and validating your ideas.” For Polymail, this included direct customer feedback via their Slack group, sharing a product roadmap in Trello in which users could propose features, and taking as many customer calls as people they could get on the line.
As noted in Hacking Growth, when eliciting feedback, it’s important to be “dispassionate about your product.” Take care to not ask leading questions, or even be too specific in your inquiry. An example of a bad question would be “do you like this feature?” You are likely to receive a binary ‘yes’ or ‘no’ answer, and be none the wiser as to the all important why. Instead, ask questions that will let the users talk, and let you listen. You want to learn about the users’ problems so you can develop the perfect solution.
As part of customer development at Polymail, before beginning any interviews they first gather some background info to help construct customer profiles. Recording this info will help you to hone in on who your specific customer personas may be, and identify the decision makers that will bring you business in the future. Building a persona database early will be of lasting value, and will help you segment and survey users later. Here are some basic questions to help construct customer profiles:
- What company do you work for?
- How big is the company?
- What is your role within the company (title)?
- What does your average workday look like?
Brandon and the Polymail team then focus on learning about users’ current workflows as it relates to email. For example, in customer interviews they may ask:
- What is your biggest pain related to email?
- How much time are you spending on this?
- How does this impact your workflow?
- How does this impact your business?
- What current solutions are you using?
- Would this (Polymail) solution help to alleviate this wasted time?
After the product has been tested also include:
- How can we improve onboarding?
- What additional features would you like to see?
Notice, before pitching his own product, the interviewer focused on obtaining information unbiased by the potential Polymail solution. Brandon says even listening to customers vent and complain can be extremely helpful in validating the need for what you are building.
Data Driven Decisions
Just like your ideas don’t matter, neither do your opinions. When deciding which features to develop next, what matters is what the opinions and numbers of users say. To get these insights, startups should be having systematic interactions with their early adopters.
Some teams do this with what is known as a Customer Advisory Board (CAB). A Customer Advisory Board is essentially a representative group of your potential customers who have been recruited for a highly engaging beta. You treat these “advisors” like VIPs in return for in-depth and long term feedback, in the hopes that they will help you to develop an awesome product while becoming early evangelists. One of the most notable examples of a successful CAB is the one put together by Peter Kazanjy for his recruiting startup, TalentBin.
At Polymail, they use a suite of tools and methods to get the same type of valuable feedback that a CAB aims to provide.
We marry data with qualitative feedback we are receiving, and come up with ideas from that angle.
-Brandon Foo, Polymail CEO
As their user numbers have grown, so has their sophistication of collecting and organizing user feedback. Polymail maintains the Trello roadmap from their beta, but has streamlined the process so its usability has scaled with their growth. Now, Polymail creates feature cards based on customer interactions, and any user can join the roadmap and vote for the features they would most like to see. Some of the cards have more than 100 votes, and Polymail’s most popular features have been developed based on user engagement with the roadmap.
Participation on the open roadmap is supplemented with additional feedback from email, Intercom, Twitter, Facebook, and Slack for pro users. Despite the growth of an active community, the Polymail team still takes the time to manually categorize user feedback from these various sources. Polymail builds a detailed user database with Asana by categorizing feedback according to the frequency of request and user type, including free users, paying users, and pro users.
They were kind enough to provide a screenshot of their Asana organization for our learning purposes.
This categorization system allows Polymail to analyze feature requests based on revenue potential, and make a quantified list of priorities to discuss and put into production. “It takes a little leg work, but it’s worth it,” says Brandon.
In addition, Polymail uses Mixpanel to collect quantitative behavior insights such as user acquisition data, retention rate, conversion, feature use, and more.
With their rich quantitative and qualitative data the team can save time in development meetings, and has easy access to statistical evidence to support their hunches. If there is a feature they want to develop, they can simply dig into the user data to see if there is demand for it. If there isn’t, they can pivot and avoid mistakes. To synchronize their engineering efforts with the customers’ need, they will often conduct interview calls to get more detailed qualitative responses than are recorded in their user database.
The relationship that you have with a company building the product that you use shapes how you feel about it in a lot of ways. If you add transparency into your process of building, intake feedback, and share your roadmap, that shapes people’s perception of you and their relationship with your company in a meaningful way.
Ongoing Iteration — MVP All the Way
Even now that Polymail Pro has launched and has a growing number of paid subscriptions, they still use the same MVP approach of minimal complexity for continued product development.
We take an MVP approach to all features we release. Never release a feature that would be too complicated, or require making too many assumptions. Take the thing you want to build and refine it to its simplest version that people can use, [but] still convey the core value of the product.
At Polymail they make it a practice of always shipping new features as early as possible, with the intention of making later iterations. Brandon and the other engineers do weekly sprints, which allows them to ship basic versions of features within one or two weeks, and then start to run tests.
The earlier you ship out a version 1 feature, the earlier you can start gathering feedback. If you continue to balance collecting feedback while proceeding with your development cycle, you may learn something that could change the direction of a feature or the way users will end up using it.
When we asked Brandon what happens when user feedback tells him something different than what the team was expecting, he said this was actually the best kind of feedback you could get. “This is generally a really good thing.That helps us to avoid making a potential mistake. We try to make that happen as much as possible.”
Steve Blank refers to recovering from this type of unexpected feedback as a “pivot.” It is not uncommon to discover that your hypothesis is false through customer development. When that happens, you take the feedback, wrap it up into a new hypothesis, build ⟶ measure ⟶ and learn again. If you have been following a lean development model of incremental iterations, pivots will be manageable, and are considered a healthy part of product development.
For more on market validation through the product development process, read this post on The Journey to Product Maret Fit
Conclusion & Recap
Customer development shouldn’t end with one cycle of build⟶measure⟶ learn. It’s important to do ongoing customer development as long as you are doing product development. Doing so will most likely save you time and money, and boost your sales potential. It will also provide important validation to the engineers doing the building.
Even while adding new team members to help with customer success, Brandon still takes the time to talk to customers every day. “It’s important to get to the front lines and not lose touch,” he tells us.
In concluding our interview with Brandon, we asked him if he had any advice for early stage startups and founders. “The most common mistake people make is spending a lot of time building, and trying to get to a place that is production ready before they let people start using it.” Brandon is in good company with people like Steve Blank, Eric Ries, and the Y Combinator partners when he suggests the smarter alternative is to “release soon…..and start collecting feedback and developing relationships with users.”
This is the way to build something that not only you think is great, but users will love too.
We hosted Brandon Foo, CEO & Co-founder of Polymail, in a live Q&A on Codementor Office Hours. Brandon shared Polymail's journey and his thoughts on how to build a startup with a customer focus!
