6 {Detailed} Tips for A Successful Program Test Pilot

There are a lot of different ways to test products and services. Application software developers use beta testing; pharmaceutical companies use clinical trials; and marketers conduct focus groups. 

If you’ve created a kick-ass program around a product, service or system, you definitely need to do pilot testing before announcing it to the world. Here are some tips to make sure your test pilot is as successful as it can be.

Let’s start by clarifying what the heck a “test pilot” means. A test pilot is the pre-launch trial run of a program or system, with a group of people (a.k.a. end users), to get their feedback on what works and doesn’t.


Tip #1: Create a list of qualifying criteria to help you select participants.

Before you find people to do your trial run with, you need to know who and what you are looking for. Here is a sample list from one of my own test pilots:

  • 3 people
  • Have a known passion or purpose in what they do—think career/life's work, not just a job
  • Are either already working in the field of their goals and are looking to grow further in it or are ready to make the first move into their passion field
  • Are friends of friends or strangers
  • Diverse in job types, salaries, locations and race/ethnicity

The important aspects of this are: how many people you want to work with; where these people are now in their life; where they want to go in their life; what you can help them achieve; and the desired relationship level and life details you are looking for.

 A good place to start is by asking yourself, “Who is your program designed for?” and create a list from that answer.

Pro tip: Unless your program is designed for a specific race or ethnicity, diversity in your test pilot people is really important. Diversity in race/ethnicity breeds diversity in thought, perspectives and experiences (tweet this). This is a great thing for your feedback and will make your program that much better.

Tip #2: Recruit the friends of your friends to your test pilot.

Your first thought is probably your friends. I recommend against testing anything on your friends who may not be as honest and open with you about the negatives, for fear of ruining your friendship. I like using friends of friends, so there is a bit of a comfort level because we have a common denominator, but enough distance between us to be able to get into the nitty-gritty of things.

What about strangers? Well, you could go down to your local grocery store and ask some strangers but that might not work out too well unless you’re testing a food sample. If you’re going to use strangers, you’re going to have to get creative in how you find them (i.e. online ads, social media posts, specific stores or events) and have a very specific list of criteria to help you choose.

Pro tip: If you are going to recruit strangers, you might want to use a Facebook ad fine-tuned to reach your ideal participant. You can customize the copy and targeting to reach your audience easily and without too much cost.

Tip #3: Decide whether to charge based on your program depth and personal preferences.

That is the question everyone asks but without a single answer. It really depends on how extensive your program is and your personal preferences. Some people recommend charging full cost, giving a 50% discount or giving it away for free.

I did my recent test pilots for free, letting them know upfront what my expectations were of them to make sure I was getting my goals met. I asked for candid feedback about everything from their thoughts and feelings that popped up along the way to constructive criticism in their entry and exit surveys (more on this later). I also asked for their commitment to the project and our multiple scheduled appointments, since my test pilot would span quite a few weeks. Lastly, I made sure to outline in the beginning the need for their testimonial and where it would be used (more on this later too!).

Pro tip: Whichever way you go on this topic, always do an amazing job over-delivering on your promises and you’ll have yourself some great testimonials and brand ambassadors for life.
Tip #4: Map out the plan and timing for your test pilot with your participants’ schedules in mind.

Before you begin your test, take some time and build out a plan based on your program. Here’s a super basic sample plan from one of my recent test pilots:

Day 1: Create qualifying criteria list
Day 1-2: Recruit using social media posts to get participants
Day 2: Filter initial participants with criteria list through email to top 3 --> schedule calls with them
Day 5: 30-min intro, expectations and info call --> email entry survey link
Week 2: 30-min survey follow-up and plan outline call --> email workbook #1
Week 3: 1-hour workbook #1 follow-up call --> email workbook #2
Week 4: 1-hour workbook #2 follow-up call --> email workbook #3
Week 5: 1-hour workbook #3 follow-up call
Week 5: 1-hour wrap-up, exit survey and testimonial call --> email exit survey link and testimonial info
Week 5-6: Apply changes and insights to program pre-launch

As you can see, my program was more extensive and required several weeks. I could’ve easily done this test pilot in less than six weeks but I wanted to be extremely sensitive to the participants’ schedules and knew that I had to break things apart into manageable chunks to get the highest quality work and feedback.

Pro tip: Save tons of time by scheduling your next appointment during your current call. Follow that up with an email confirmation and calendar invite within a day after the call to lock them in before they forget.

Tip #5: Create unbiased entry and exit surveys that give you priceless feedback and insights.

I do my entry and exit test pilot surveys in Google docs using their form builder. I’ve used Survey Monkey and Constant Contact in the past and have enjoyed using both. If that feels too complicated, you could even email them a Word document to fill out!

The key here is creating a survey that gives you the information you seek, without guiding the person filling it out. Here’s 8 Tips for Writing Effective Survey Questions from Constant Contact, which is a great place to start if you’ve never written a survey before.

In the entry survey, your main objective is to collect information and insights about your participants and their preconceived thoughts about your industry, business or types of products/services you offer. Understanding how your ideal target customer thinks before using your product is valuable information for your marketing and copywriting.

The exit survey’s main objective is to collect feedback from the participant on their experiences with you, your business and your program. It’s also important to see how their initial preconceived thoughts have shifted after working with you. I also like to include questions about my company name or branding, estimations of program value versus amount they’d personally pay, and any items that would’ve made it better overall.

Pro tip: Have a friend review and take your survey for proofing, inconsistencies, question clarity and biases. They can tell you if it feels too long or if a question is confusing.

Tip #6: Guide your participants to give you amazing testimonials.

From the very beginning, and throughout the testing period, I mention testimonials to my test pilot folks. I wanted to keep it top-of-mind that this whole exercise will have public expectations at the end. During the wrap-up call, I go over all the testimonial details and follow up with an equally detailed email.

Some people are natural writers and others aren’t. It’s important to know where your participants fall in that range and accommodate them with as much freedom or support as they need. If they aren’t comfortable writing, you can provide them with questions to act as inspiration or they could answer the questions and you could assemble the testimonial for their approval. Here are some testimonial questions for inspiration:

What was your problem before working with...?
What factors made you choose X to assist you?
What were your preconceived notions about X before we started and how has that perception changed since working with X?
What do you like most about working with me/X?
What are three of the biggest benefits of working with me/X?
How has your life changed since we began?
How would you describe the way I provided my service to you?

Pro tip: Always include the “Is there anything else you’d like to add?” question. Most of the time you won’t get great answers to this but when you do, they are usually very insightful.

Now it’s your turn. Which tips are going to help you the most? Do you have any other tips from your adventures in pilot testing? Share your insights in the comments below.

Thanks for reading and be well!