You’ll have to start letting real people use your application at some point, so you’ll probably have a private beta–that’s easy, right? Just let people into the application? Not exactly… Running your private beta is all about getting feedback and improving your product. There are several factors that can drastically affect how well it goes, and most of them require significant advanced planning and coordination.

You might think it’s easy to come by, but the reality is that few people take the time to provide feedback, and even fewer take the time to provide thoughtful and measured observations. You can’t be picky, but you can put the right tools in place to help you get as much feedback as possible. If no one is using your beta or providing their comments, you’re going to have an even harder time getting people to use it after you launch.

If you’re not receiving enough feedback, either no one is interested in your application, or your app is perfect. The latter is unlikely, so you probably need to increase the number of people trying your application. The better option, however, is to get in touch with people who’ve already tried it.

People are generally thrilled to have a chance to chime in. Most people don’t see it as an inconvenience when you contact them–when they know someone cares and their feedback is going straight to the top, they’re usually excited to share. If you ever find you’re not getting enough feedback, set aside some time to ask your customers.

You might also think that having friends use your application will give you enough feedback, but the answers you get from them should be taken with a grain of salt. Your goal should be to get feedback from people you don’t know. And it turns out that most people only offer their thoughts when the product they’re using is almost perfect for their needs–they’re the kind of people who might say, “Oh, this is just about what I need! Now if only it had…” You need to invite people from outside your circle of friends; and you’re even better off if you can find a subset for whom your product is almost a dream come true.

I don’t have any specific numbers, but I’d say that only about one in twenty of the people you invite to your beta will provide useful feedback without prompting. There are people for whom your product is perfect. There are people who find it a decent fit. And there are people who struggle to make it meet their requirements. All three groups are important, but you should also understand how their needs align with your vision. In some cases, you might find that you need to adjust your vision; in other cases, understanding these different groups may help you to focus your vision and filter the feedback accordingly.

As the beta progresses, scale up and invite more people. Although it depends on the size of your beta, you’ll probably need an invitation system to manage access. At the most basic level, you’ll need an easy way to enter an email address and generate a link with an invite code. If your beta is sizable, you’ll need to be able to process batches of emails and generate invitations automatically. It’s not terribly difficult, but it’s one more thing that requires time and attention. And remember, an invitation isn’t enough. Remind them what your product does and how it can help them. Otherwise, a good portion of your invitations will just be ignored.

Here’s what you need to bear in mind–and this is where I completely missed the boat–don’t wait for feedback to come to you, especially during the beta. You must contact your customers rather than wait for them to contact you. Feedback from paying customers is second only to revenue among the things your business needs to survive and thrive.

Don’t limit yourself to feedback and discussions over email. Talk to your customers over the phone. We were lucky in that we always received more than enough feedback through email, so I never set aside much time for customer phone calls. But don’t let our streak of luck hold you back from giving someone a call if you’re low on feedback. It wasn’t until a couple of years after our launch that I got in touch with a handful of customers about case studies–and I had no idea just how helpful those phone calls would be.

After about fifteen phone calls with customers, I had a wealth of information, and although I had every intention of putting together some case studies, I couldn’t stop thinking about all the ideas and suggestions that I received. I had heard much of the feedback before, but the extemporaneous nature of our conversations really helped me understand how they used Sifter and how it could work even better for them.

So, yes, email your customers, but talk with them too–and if you can meet them in person, that’s even better. Don’t make the same mistake I made: don’t assume that feedback from email is good enough; it’s useful, but it’s nowhere near as powerful as a real conversation.

There’s also money. The prevailing philosophy is that betas ought to be free, but I think that’s the wrong approach. Betas are for working out kinks, and that includes any flaws in your pricing and your payment system. If you don’t accept payments during your beta, you’ll miss any problems with your billing system–and those aren’t the kind of problems you want to have.

Requiring customers to pay helps further narrow your beta audience to people who are seriously interested. Getting feedback during a beta is difficult, but gating your beta can help you understand whether people will pay for your product once it launches.

We charged a flat five dollars a month during our beta. Not much, but requiring people to provide a credit card at the end of their trial period helped us gauge whether people were willing to pay for it. Don’t worry too much about whether you’re charging the right price during your beta. Your beta may be great for gathering feedback, but unless you require a credit card, it won’t actually answer the single most important question for your business: will people pay for this? If people won’t pull out their credit cards, nothing else matters.

Betas are an exciting time, and they’re also critical. There are countless ways to do it wrong. You’ll be inviting some of your most interested potential customers into the most incomplete version of your product. Set it up correctly and create healthy lines of communication with those early users, and you’ll set yourself up for success. It’s not quite as serious as your full launch, but the private beta is just as critical to your future. Do it right.

Running Closed Betas: Which Users and How Long? Des Traynor from Intercom provides some great insights on efficiently running a beta and knowing when to launch your product.