Online Panels: What They Are & How to Use Them Effectively

Online panels are invaluable tools for gathering feedback and gaining crucial market insights. However, to properly leverage this tool, you must understand how to use it effectively.

In this post, you’ll learn what online panels are, how to choose the best type, and how to set up, run, and manage a successful online panel.

Table of Contents

• What is an online panel?
• Types of Online Panels
• Advantages of Online Panels
• Disadvantages of Online Panels
• How to Run an Online Panel
• Online Panel Best Practices

What is an online panel?

An online panel is a group of individuals who provide insights or feedback through a digital environment. These participants, referred to as panelists, are typically recruited through online channels and contribute their opinions via digital surveys or virtual discussions.

Market research expert, Jim Flagg, explains the basics of online panels in the video below.

Types of Online Panels

There are two categories for online panels: open and closed. There are a few different types of closed panels that we’ll dive into below.

Open panels

As the name implies, an open panel is an online panel that’s “open” to anyone willing to participate. This means that the eligibility criteria for participants are either very minimal or, in some cases, nonexistent.

One of the reasons these panels take a more flexible approach is that it allows the researcher to access a larger pool of participants. This ‘advantage’ is not applicable across the board, but it is important in use cases where the size of the online panel is more important than specific participant qualifications. An example of such a use case might be a study to gauge general market sentiments about a current trend like artificial intelligence.

But other than simply providing access to a larger participant pool, open panels also allow the researcher to collect more inclusive and diverse opinions, which, in turn, results in more representative findings. For example, an open panel might be the preferred method for understanding consumer attitudes toward a new product launch across various demographics.

Finally, open panels are also a powerful tool for early-stage exploratory research, such as market exploration and understanding the audience or market.

Closed panels

Unlike open panels, closed panels take a more selective approach to participant recruitment. Here, panelists are filtered based on predetermined selection criteria.

Typically, these panels are formed to uncover niche insights, requiring all participants to be carefully vetted for specific qualifications, experiences, or characteristics relevant to the research focus. For example, a study to uncover pain points in a particular demographic of a product’s users will require that all participants are, first, users of that product and also fall within that demographic.

Now, within this category of online panels, several types of panels can be formed based on the study's objectives. Here are some examples:

Customer advisory panels

A customer advisory panel is an online panel made up of users of a product or service. These are some of the most popular panels amongst businesses because they provide important insights that can be leveraged across product development, marketing, and the overall customer experience.

Although these panels consist of customers, recruitment eligibility will vary based on the panel's objective. For example, a customer advisory panel that focuses on a general product feature might require participation from all users. In contrast, a panel focusing on a specialized product feature might target a specific user segment.

Expert panels

An expert online panel is a closed panel of individuals with extensive expertise in a specific field or area of interest. For example, a panel made up of financial advisors who provide feedback on a new stock investment app.

Participants in these types of panels are carefully selected and screened because they’re asked to provide specialized insights about the topic of discussion. These panels may also be structured so that participants are able or even required to provide in-depth analysis and targeted recommendations.

Employee panels

An employee panel is an internal panel created to provide feedback on processes, initiatives, policies, etc, within the organization. These panels are invaluable because they promote communication and improve organizational transparency and engagement.

That said, like customer advisory panels, employee eligibility will also depend on the panel's objectives. For example, a general employee sentiment study requires company-wide participation, while a discussion on company financial management requires C-suite participation.

Advantages of Online Panels

While panels can be conducted in person, online panels provide several advantages for researchers and participants.

1. Online panels provide a wider reach for your study.

Online panels are not geographically limited, which allows researchers access to a broader pool of potential participants. This means the study allows for a more diverse group of panelists and, as a result, more representative findings.

This also means that researchers can be more selective in choosing participants because they don’t necessarily have to sacrifice quality for quantity. Even if a substantial number of potential panelists decline, there are still plenty of potential participants to meet the quota.

2. Online panels are more time-efficient.

Online panels offer a significant advantage over in-person panels regarding time efficiency. This is because they allow researchers to streamline and automate the entire process from participant recruitment to data analysis.

In contrast, the logistics of running in-person panels require massive time commitments at each stage of the process, such as securing a location, setting up the physical event, ensuring participants turn up, etc.

Another vital factor is how time efficiency affects participation rates and engagement. The more commitment that you require from your participants, the less likely they are to opt in or engage.

3. Online panels offer flexibility for the researcher and participants.

Online panels allow for a lot of flexibility at multiple stages of the research process. Perhaps the most obvious advantage is that these panels aren’t subject to geographical limitations, i.e., hosts and panelists can participate without needing to converge at a single physical location.

Another advantage is that the tools used to run these panels also provide numerous features to tailor and personalize the experiences for participants. For example, if multiple dates are available, some tools allow panelists to choose their most convenient date/time.

Additionally, some platforms offer accessibility features such as closed captioning for videos, screen reader compatibility, and translation options to accommodate a wide range of participant needs.

This type of flexibility is essential to researchers and panelists because it encourages participation and reduces friction.

4. Online panels are more cost-effective.

The costs associated with planning and running in-person panels can add up significantly. Online panels, on the other hand, allow researchers to eliminate administrative fees, travel expenses, participant materials, hosting costs (hiring a venue), etc.

While these costs may be feasible for small-scale in-person panels, expanding to larger participant pools, particularly across multiple cities or countries, is financially impractical.

Disadvantages of Online Panels

Like any other method of collecting feedback, online panels have drawbacks and challenges.

1. Online panels have limited accessibility for certain demographics.

While online panels offer access to a broad range of potential participants, they also inadvertently exclude specific population segments due to their reliance on the participant's access to the internet and digital devices.

Also, the scope of studies that can be effectively conducted using online panels is restricted. For example, studies that require panelists to provide feedback based on physical interaction with the subject, such as product testing or sensory evaluation, can’t be conducted online.

2. Online panels have more variables and a higher potential for bias.

Online panels introduce variables and biases that may influence your study’s outcome. For example, because panelists participate from diverse locations, they are subject to various environmental conditions that could impact their responses and overall engagement. Participants who are distracted or uncomfortable during the panel are likely to provide less accurate responses.

Additionally, online panels are inherently biased as they are more likely to attract individuals who are technologically savvy or familiar with the online panel environment. Even amongst these segments, factors such as promotion and recruitment channels (e.g., social media vs email) will further influence the composition of the participant pool.

If unaccounted for, all of these variables and biases could lead to skewed findings that are inaccurate.

3. Susceptibility to technical difficulties

Most online panels rely on technology for the research process, from planning to data analysis. And while this is great for efficiency, this also means that there are opportunities for technical hiccups to occur. Furthermore, if organizers primarily rely on third-party services and software, they have limited control over rectifying any unexpected technical issues.

The risk of encountering technical difficulties isn‘t confined to the organizers alone. Considering that panelists use their tech to participate, there’s a high likelihood that at least some portion of the panel (especially in larger groups) may experience unforeseen technical issues.

4. Limited participant validation

Researchers who organize online panels are more limited in their ability to validate and screen participants. For example, many online panels operate under a good faith assumption that participants will respond truthfully to the qualification questions determining their participation eligibility.

Compared to in-person panels, where researchers have more opportunities to observe and verify participants, this reliance on self-reported information introduces a level of uncertainty regarding the reliability of the participant pool.

How to Run an Online Panel

Now that you understand the limitations of online panels, here’s a step-by-step guide to help you successfully run your first one.

1. Establish clear objectives for your online panel.

Running an online panel begins with clearly understanding your objectives and goals. It‘s crucial to ask yourself why you’re running this panel and what specific insights you aim to uncover.

These insights should then be defined as research questions or hypotheses encompassing anticipated outcomes. Some real-world examples of these questions could be:

• What are the emerging AI trends gaining traction in the market?
• What are the most common pain points customers experience when interacting with the support team?

Once these questions are established, it's essential to take a step back and determine the scope of the panel (and overall commitment) needed to address them. This is critical because it ensures your research objectives are well-defined and achievable.

2. Define your ideal participant persona (IPP).

Once your research objectives are clear, identifying the characteristics of your ideal panel participant should be a reasonably straightforward task.

A simple tip is to consider each research question and determine what person might be able to best answer these questions based on the following:

• Demographics: Age, gender, location, occupation, etc.
• Psychographics: Interests, behaviors, lifestyle, etc.
• User Segmentation: New users, experienced users, occasional users, etc.
• Expertise Level: Novice, intermediate, expert, etc.

You can then determine your panel's inclusion and exclusion criteria based on your findings. Remember, the goal is to create a profile that offers practical guidance for your recruitment and screening processes.

It’s also essential to consider the participation incentives that you’ll offer at this stage. This ensures that your incentives align with your target audience and the goals of your panel.

3. Choose your panel type.

First, you must decide whether to run an open or closed panel. Here are some questions to help you along:

• Does your research focus on broad feedback or niche insights?
• Are you interested in conducting exploratory research, or do you have well-defined research questions?
• Are you seeking diverse perspectives or targeting a specific demographic?
• How strict are your eligibility criteria for participants?

Once you’ve determined which aligns better with your research objectives, it’s time to decide the specific type of panel you’ll be hosting. For example, an expert panel would be the obvious choice if your target participants are mostly experts providing insights on a complex topic.

That said, another key consideration is resource availability, as this will determine the following:

• The feasibility of an open versus closed panel
• The type of expertise or specific demographics you can afford to target and the scope of incentives you can provide to entice participation
• The extent to which you can invest in technology and tools for recruitment, engagement, and data analysis

4. Create essential panel documents.

At this stage, you must determine what “instrument” you’ll use in your panel. Generally, this will most likely be either a questionnaire or an interview script.

Regardless of your choice, here are some tips to keep in mind:

1. Ensure each question is clear, concise, and aligned with your research objectives.
2. Maintain a focused approach to gathering relevant data.
3. Frame questions with sensitivity and consider ethical best practices.
4. Anticipate the unanticipated and create space for unscripted insights.

Beyond your questionnaire or script, you’ll also need to create supplementary documents your participants might need to fill out, such as Non-Disclosure Agreements (NDAs), consent forms, or requests for specific identity documents.

5. Recruit panel participants.

You can recruit panel participants in two ways: manual recruitment or renting participants from a database provider. Each approach comes with its own set of advantages and drawbacks.

Manual recruitment involves directly sourcing participants through social media, email, SMS, CRM contacts, etc. It’s more cost-effective and allows you to target a more relevant audience for your product or service. However, it can also be time-consuming and particularly challenging for those without a readily accessible audience.

On the other hand, using a database provider is considerably faster than manual sourcing and allows you to gather a larger group of participants faster. However, the main drawback is that database participants often fall into the category of “professional panelists,” meaning their primary motivators are the incentives rather than genuine interest in your specific product or service. Additionally, they’re unlikely to have prior familiarity with your particular brand or products, which may be vital if you're seeking participants with existing knowledge of your business.

Once you decide which method best aligns with your objectives, it's time to start recruiting. We recommend using survey software to automate most of the heavy lifting. However, here’s a simple process to follow if you’re recruiting participants manually:

1. Create a user-friendly sign-up form or method for potential panelists to express interest.
2. Develop a screening system to evaluate potential participants.
3. Set up a follow-up system to send reminders and notifications to panelists as the panel date approaches.
4. Promote your sign-up form across various recruitment channels. This may include social media, email campaigns, SMS notifications, etc.
5. Once someone expresses interest and successfully passes the screening process, assign them to an available slot and add them to your follow-up system.

6. Run your panel.

Running an online panel is all about preparation. First, you must ensure all your technical ducks are in a row. Before the panel begins, check to see if the platform is running smoothly and that your materials are ready.

Once the panel starts, here are some tips to keep in mind:

• Greet your participants warmly to set a positive tone and help panelists feel at ease.
• Ensure every participant knows what to expect and how the panel will proceed.
• Provide clear instructions for troubleshooting any technical issues. Offer assistance if needed.
• Encourage participants to share their thoughts and ideas, and don’t be afraid to deviate from the script a little.
• Ensure you cover all planned topics within the allotted time.

• Take detailed notes if possible. If anything important stands out or might not naturally show up in a transcript or recording, write it down.

After the panel, it’s time to organize your recordings and notes.

Remember, the insights you extract must align with your objectives. So, go back and consult your research questions or hypotheses to guide your analysis.

Online Panel Best Practices

To wrap things up, here are some general best practices for planning, running, or managing online panels.

Set clear and accurate objectives.

Your research objectives will frame the questions you ask, the participants you select, and how you interpret your results. So, before you look at a single tool, create a form, or reach out to a potential participant, meticulously define and refine these objectives.

Pro Tip: Use specific, measurable, achievable, relevant, and time-bound (SMART) goals. For instance, instead of a broad objective like “Understand customer preferences,” refine it to “Identify three key factors influencing customer purchasing decisions by the end of the panel discussion.” This specificity not only guides your questioning but also sets clear expectations for the outcomes, making it easier for you to measure success and apply the insights in a practical way.

Stay on top of your tech stack.

Remember to test all the tools you‘ll be using well in advance. It’s also essential to have contingency plans to handle unexpected issues if something goes wrong for you or your participants.

Pro Tip: Tool rehearsals can help you simulate the entire panel process, from participant onboarding to data collection. This helps identify potential technical glitches and ensures a smoother execution during the actual panel. It can be helpful to establish clear protocols for handling unforeseen technical difficulties, so you can troubleshoot quickly and maintain the panel's momentum without disrupting the participant experience.

Implement checks to ensure data integrity.

Your top priority should be accurate and reliable findings when running an online panel. Here are some quick tips to help you do so:

• Thoroughly evaluate your screening criteria and processes to minimize bias.
• Provide clear guidance to panelists at every stage of the process.
• Assure participants of their data's confidentiality to encourage honest feedback.
• Consider automating feedback collation and analysis to eliminate potential human errors.

Pro Tip: To further safeguard data integrity, you can create a comprehensive data validation process throughout your online panel. This involves cross-verifying responses through internal consistency checks, validating against known benchmarks, and ensuring data accuracy.

Building an Effective Online Panel

Running an online panel requires a systematic and methodical approach. From setting clear objectives to testing your tech stack, each step plays a crucial role in the success of your panel.

Remember, the goal of an online panel is not to collect data randomly but to uncover targeted insights you can directly implement in your business. If you don’t get the right information at first, keep tinkering until you find the right questions and audience that fits your needs.