Fake Door Testing: The Benefits, the Risks, and How to Build Functional Tests

Wondering how to create fake door tests that work? Come on in, we’re covering how to make effective tests that benefit both your users and your business.

Ray Slater Berry
11 min read
Fake Door Testing: The Benefits, the Risks, and How to Build Functional Tests

Business: Knock, knock!
Customer: Who’s there?
Business: No one, this is a test. But...

Fake door testing is a dangerous game. Done right, you can win future customers, validate an idea for a product launch, and even find beta testers for a new feature. Done wrong, you could damage your brand reputation and risk a PR nightmare. 

So, where’s the middle ground? How can you run fake door tests that help product-led business growth while building trust and buy-in from your customers? 

Come on in; we’ll show you how.

What is fake door testing #

Fake door testing is a lean research technique product teams use to validate new products, features, or services. Fake door tests, sometimes also called painted-door tests, garner interest by presenting people with a call to action or a low-fidelity prototype of an idea and seeing how many people will click on it or try to use it. 

Fake door testing comes with pros and cons. It’s a product development method that, done right, can help you find product-market fit and lower adoption risks of new ideas. However, if it’s done wrong, it could break customer-brand trust. 

Let’s start with the positives.

How it works (and why it works so well) #

Fake door tests present the user with an invitation to use a product that isn’t actually there. This can be a CTA button, an in-app notification, a pop-up or a slide-out, or even a product video or an ad. When the user clicks or signs up for that fake product, they receive a message or land on a page that comes out with the truth the product doesn’t exist (yet). 

However, it’s something your business is either building or considering. The data collected from a fake door test—things like clickthrough rate (CTR) or page interactions—help a product team decide what could work and what won’t. 

A fake door test, done right, triggers curiosity or fear of missing out (FOMO), and it excites users to try something new—potentially being one of the first. 

Finally, a fake door test works well because not only does it help product teams evolve, it could also entice users to stick around by showing promise of what’s to come. Or, in case you run fake door targeted ads, it can help you find new customers, build personas, and define demographics.

Benefits of fake door testing and how to maximize their value #

Despite the general fear around fake doors, there are some great benefits for businesses. Let’s explore what some of these fake doors can do for your team.

Gather feedback on how users respond to a new offering #

First and foremost, gathering feedback with low investment is a fantastic way to minimize product build risk and save on engineering resources. 

Depending on the tools you use and the fake door test you choose to run, you have the opportunity to collect both qualitative and quantitative feedback that can radically improve the build and pitch of a new idea—and support a product launch.

Run product surveys with Chameleon

In-app Microsurveys allow you to capture important user feedback when it's most relevant

Evaluate even the smallest in-product new features #

You can also use fake door tests to understand how current users feel about a new feature. 

Whether your fake door test comes within an interactive product tour for new users, or you present it as a modal pop-up for long-term users, you can gauge interest in the minutest of features and get people excited for what’s to come. 

How does this work? Take a look at a hypothetical test that Curtis Stanier, Product Lead at Delivery Hero, created to showcase the example he lays out in his Medium article on painted-door testing.

(Source)

Develop features your users want and disregard the ones they don’t #

Fake door tests are a fantastic tool for validating new features and winning initial buy-in from existing users. Of course, for this user research tool to work, you need great user engagement in your platform, and people need to be accustomed to providing user feedback. 

Fake doors are also great for disregarding new features you’re considering. For example, if you’re using an MVP approach to your product, you may receive customer feedback that states the importance of a feature for a particular customer. 

However, when you run a fake door test to measure the popularity of this feature idea, it could reveal that the lift is not worth the investment. 

Lastly, fake door tests are a great way to prioritize your product backlog. If you’re unsure of what to work on first, ask your customers via fake door tests and let them answer.

Segment users for beta testing and feature up-selling #

Fake door tests can be the perfect tool to help you segment your current user base for beta testing. 

For example, perhaps you’re building a feature that a certain user really wants. Rather than offering the feature to all users, offer it to those that care about seeing it succeed.

Offering beta testing opportunities is a great way to build user engagement and gather qualitative user feedback. This also ups your feature adoption rates when you do bring it to market. 

Once your beta testing is over and you’re ready to release your new feature MVP, you already have an invested group of (ex-beta) users that could be willing to pay for this—just make sure you offer it at an exclusive rate. Say thanks.

Discover new customer cohorts #

Fake door tests can also be beneficial if run out of product and via targeted ads. They can help you find product-market fit early on and identify entirely new groups of customers for a product launch. 

For example, if you have an idea about your target audience, you can run social media ads or Google PPC ads that entice someone to a landing page and low-fi design of the product, feature, or service to come. In doing so, you can start building lookalike audiences for future advertising. 

Plus, if you’re collecting emails in your fake door tests, then you can build a list of people that want to know more and will potentially convert to a sale when your idea is ready. 

Well, we’ve covered the benefits. However, that’s not to say fake doors come without risks.

Risks of fake door testing and how to avoid them #

For all of the benefits available to a business running a fake door test, there are risks to running them as well. But these risks can be avoided, or at the least, minimized.

Fake doors can cause potential distress #

It takes time to build relationships with customers. A fake door test is essentially an upfront fib about what the customer is about to get. This causes distress and potentially diminishes a customer’s trust in a business. 

However, there are ways for you to remedy the fib immediately. On the click-through landing page, be transparent and honest with your copy. Let the visitor know that you’re considering working on this feature and thank them for their time. 

As soon as you validate a customer’s feelings, you can remedy the relationship and even get them excited about what’s to come. But, you need to act fast with your copy.

Fake door tests can trigger disappointment #

Of course, there’s a certain amount of disappointment someone will experience if the solution they thought they were getting is not available to them. 

You can avoid this by empathetically handling the user’s disappointment. Explain the reason behind your actions, and ask users if they would like to leave feedback on their expectations with a product survey or participate in a beta version of the product for free. 

Note: Integrate user feedback tools wisely, and be cautious of asking too much from someone. 

Lastly, build a sense of community around your new product and let your users know how heavily they can positively influence what’s to come. Assure them you’re essentially building something to answer their needs. Remain customer-centric.

Fake doors might diminish credibility #

If you’re a relatively new SaaS or are targeting a new audience for your fake door test, then credibility is a big thing. Ever heard the expression: starting off on the wrong foot? The same applies to your fake door. 

How do you remain credible with fake door testing? There are a few ways. 

One is with the messaging on your landing page and the things we’ve mentioned above: beta testing, feedback opportunities, sharing insights on what’s to come.

Two, if you choose to target a new audience, keep your testing cohort and spend small. You may damage a few relationships with potential customers. However, if you gain valuable insights, then you’re more likely to present a better-fitting product to the masses. 

Risks covered. Let’s move into how to make fake door tests.

How to make an effective fake door test #

Right, now that you know the benefits and risks of fake door testing, let’s get into how to make fake doors. Just saying that seems ominous, but here we are. There are plenty of product marketing tools to help you get the job done; a few fan favorites are: 

  • Mailchimp: for drag-and-drop landing pages, email collection & journeys

  • InVision: for simple and effective product mockups or wireframes

  • Semrush: for refining any PPC campaign for your fake door test

  • Chameleon: for creating in-app fake door testing pop-ups 

Now you’ve got your tools, let’s get into how to make fake doors that benefit your business and customers.

1. Pick your audience #

First and foremost, who are you creating this test for?

Will you be targeting potential customers via advertising? It’s a great option if you want to test a new business model or product. Or, will you be targeting users that are already navigating your product? This is a good option for new features and if you have an engaged user base. 

Once you decide who the test is for, you can build with greater awareness.

2. Create the fake door and what’s inside #

Next, you’re ready to build your fake door and what lurks behind it. Remember to keep things simple, and transparent once someone is inside. 

The fake door itself needs to be equal parts enticing as it is informative. Breadcrumb your users with just enough information to win a relevant click-through. 

That’s exactly what Buffer did for their new Start Page feature. By placing a CTA in the notification bar on their website Home page, they encourage visitors to sign up for early access. When someone clicks on the CTA, they land on a page where they can leave their email address to join a waitlist.

(Source

As for the landing page, keep things low-fi. One of the benefits of fake door testing is to save on design resources and engineering time, so keep it that way. 

It’s also critical to be transparent and use empathetic copy here, so spend time with your copywriter to ensure the wording is just right and you remain serving a delightful experience for users.

Things to include in your test content: 

  • The purpose of the new product 

  • The key benefits of the new product

  • A potential roadmap

  • How the user can help 

  • An option to sign-up for more 

  • Visual aids: screenshots, wireframes, or a product video

Let’s take a look at how Dropbox did it back in 2019 when they announced the new desktop app. On the announcement page, they included the opt-in option for the existing users, to get early access and take the new app for a spin. Plus, they added a dedicated button for collecting feedback.

“Ready to try out the new Dropbox experience for yourself? All Dropbox users can opt in to the new desktop app through our early access program, and we’re rolling out the other features over the next few weeks. We can’t wait to hear how you’re using the new Dropbox, what you like, and what you want more of—look for the “Send feedback” button in the lower-right corner of the new desktop app.”

All that was accompanied by an engaging new Dropbox announcement video.

3. Work on your follow-up journey #

Remain accountable for whatever you promise your page visitors. 

  • If you promise more information as you build, then deliver that. 

  • If you promise the option to be a beta tester, then invite them. 

  • If you say that the product should be ready by a certain date, then get that MVP in their hands or let them know why it’s not ready. 

Communication is critical for maintaining the relationship you’re building with the fake door test. 

Note: You may also have some stellar product-qualified leads for future marketing, so make sure you’ve got an email campaign in place for when the product is available. 

Gather data and measure success #

It’s important to set some KPIs around what would be the success of your fake door test. Are you looking for a certain click-through rate to give your team the green light and start the build? Perhaps you need a certain number of positive feedback responses to let your PM team know an idea is valid. 

A few goals and data sets to measure are: 

  • Conversion rate

  • Click-through rate

  • Email signups 

  • Bounce rates 

  • Positive user feedback 

  • Negative user feedback 

The quantitative data and qualitative responses you receive because of your fake doors will be crucial to informing your new product or feature build. 

Note: Whether this is an in-product test or something you’re running externally, make a decision on whether it’s worth it for your validation method. If the negative feedback outweighs the positive, you’ll need to either reassess your fake door test or consider scrapping it from your lean development process altogether.  

You've created a fake door test. What's next? #

Fantastic, so your fake door test is up and running, and you’re ready to start taking action. It’s time to zoom out and look at the bigger picture. It can be overwhelming getting this test right. Remember why you ran this test in the first place.

Your fake door test is a crucial part of your product development process. It helps your researchers to gather data and developers to test, iterate, pivot, and evaluate new ideas. 

Document your learnings from the fake door test and share it with your team. Who needs to know what? And why? How can you translate quantitative and qualitative data into actionable insights that will help you build better products and increase user adoption?

Closing the door on fake door tests #

That’s a wrap on fake door testing. Despite the stigma around this type of test, if done right, it can be extremely beneficial to your business and your customers. 

Remember, once a user is ‘through the door,’ they’ve already provided you with valuable insight on your hypothesis. Make sure you provide value back and continue the conversation around the topic. 

Thanks for your visit—we hope to see you again soon!

Drive Feature Adoption 🚀

Attach a release log to a widget or any native button in your app with Chameleon, and announce features in context

Effective engagement, made easy.