ESOMAR has 26 questions to help research buyers of online samples. Opinion Search has answers.

1. What experience does your company have with providing online samples for market research?

   For over a decade, Opinion Search has been a leader in the online market research industry. We house a national online respondent panel including both consumer and business respondents. Through extensive participant screening and categorization criteria, we are able to rapidly survey large portions of the general population as well as small, low-incidence target groups.

2. Please describe and explain the types of source(s) for the online sample that you provide (are these databases, actively managed panels, direct marketing lists, web intercept sampling, river sampling or other)?

   Our online research panel is an actively managed respondent panel, continually monitored by our dedicated Panel Management team. Panel respondents have been recruited from a multitude of sources, including but not limited to; Co-registration offers on partners' websites, targeted emails sent by online partners to their audiences, graphical and text banner placement on partners' websites (including social media, news, search, and community portals), and telephone recruitment of general and targeted populations. Our panel recruitment and maintenance operations allow the firm to provide representative samples of the general population as well as identify and reach under- represented and hard-to-reach populations of interest. Each recruitment source is carefully vetted through a rigorous interviewing and testing process. Our database of respondent information is actively screened and updated along numerous demographic and psychographic variables to allow for precision in the online sample we provide.

3. What do you consider to be the primary advantage of your sample over other sample sources in the marketplace?

   The primary advantages of our sample include: the panel being actively managed by a dedicated team of professionals, our numerous quality assurance practices to ensure respondent and data integrity and our robust screening and demographic profiling. Our rigorous interviewing and testing process allows us to efficiently vet each recruitment source to ensure a diverse respondent base. In addition, we have a deep understanding of sampling, survey design and weighting which allows us to project from panel samples to the population of interest. Our proprietary propensity weighting methods are also of note as a significant advantage when comparing our services to those of other providers.

4. If the sample source is a panel or database, is the panel or database used solely for market research? If not, please explain.

   Our panel is used solely for market and opinion research.

5. How do you source groups that may be hard-to-reach on the internet?

   Our online research panel has recruited respondents by using hundreds of different recruitment sources. We utilize special recruitment campaigns or techniques for special populations such as older adults, young adults and minorities.

6. What are people told when they are recruited?

   When respondents are recruited into our panel, it is made very clear that they are joining a market research panel and that they will be asked periodically to participate in online research. They are shown the terms and conditions of panel membership as well as our privacy policy. Panellists must agree to our terms of use which state that panellists are limited to a single membership and can be removed if they are found in violation of this rule. A link to the Terms of Use is referenced on the registration page and in each survey invitation.

7. If the sample comes from a panel, what is your annual turnover/attrition/retention rate and how is it calculated?

   The rate of panel attrition varies greatly by demographic but on average our voluntary attrition rate is less than 3%. This figure is calculated by dividing the total number of active panel respondents by the number of panellists whom have terminated membership over a one year period.

8. Please describe the opt-in process.

   All panellists recruited have completed a 'confirmed' or 'double' opt-in (COI/DOI) process. This process requires that each registrant confirm his or her desire to join our panel by clicking on a link within an email that is sent to the registrant's email address upon registering. The content of the email specifies that by clicking on the link the registrant is expressly stating his or her desire to take part in the panel.

9. Do you have a confirmation of identity procedure? Do you have procedures to detect fraudulent respondents at the time of registration with the panel? If so, please describe.

   We collect a respondent's physical address when they establish an incentive account, at the time of incentive redemption and as part of special screening surveys that are conducted on an ongoing basis. In an effort to detect fraudulent respondents at the time of registration, we perform de-duplication based on email address prior to the opt-in process. In addition, the registration page uses digital fingerprinting to monitor and exclude hits from the same computer.

10. What profile data is kept on panel members? For how many members is this data collected and how often is this data updated?

    Opinion Search requires a minimum of information for panel membership: email address, gender, year of birth, and postal code. Additional demographic details, such as: income, race, education, etc. are gathered through normal survey taking activity. Key demographics are verified and updated with every survey taken. We also have a unique annual screening program through which we collect hundreds of variables on each participant (on a voluntary basis).

11. What is the size and/or the capacity of the panel, based on active panel members on a given date? Can you provide an overview of active panellists by type of source?

    We have hundreds of thousands of Canadian panel members. The breadth and depth of our panel competes with other Canadian online panels in the industry. Our industry leading capacity stems from ongoing and extensive recruitment and screening efforts.

12. Please describe your sampling process including your exclusion procedures if applicable. Can sample be deployed as batches/replicates, by time zones, geography, etc? If so, how is this controlled?

    We draw stratified random samples from our online panel based on known proportions of individuals in important demographic groups. To account for differential response rates, we can modify these sampling proportions, placing greater weight on those with lower response rates. To ensure that survey participants are distributed evenly across projects, we also pull samples based on online panel members' historical survey participation. Stratification by prior survey participation is superimposed behind the scenes on every stratified sample pull. We can exclude respondents from research in a variety of ways including but not limited to: subject matter, time period and past participation depending upon the specific needs of the research project.
    
    Samples can be deployed as batches/replicates, by time zone, geography, etc. All outbound email sample files are batched in order to manage the speed at which we deliver email messages to various ISP's. Typically we design batches by size, but we can batch based on whatever sample selection criteria our clients desire.

13. Explain how people are invited to take part in a survey. What does a typical invitation look like?

    The standard invitation format is text however; HTML is sometimes used for key segments such as lapsed respondents or young males. Invitations comply with industry standards and Canadian privacy regulations.

14. Please describe the nature of your incentive system(s). How does this vary by length of interview, respondent characteristics, or other factors you may consider?

    Our online research panel respondents are offered points which are redeemable for cash and other rewards. The number of points awarded does vary based upon survey length. Respondents are also entered into a bi-monthly cash sweepstakes available to all survey respondents. Respondents may also view aggregated answers to a select subset of the survey upon completion. We encourage feedback from our respondents and ask each one to evaluate their survey experience at the end of each study. In addition, we provide custom sweepstakes and cash incentives as appropriate for hard-to-reach professional groups.

15. How often are individual members contacted for online surveys within a given time period? Do you keep data on panellist participation history and are limits placed on the frequency that members are contacted and asked to participate in a survey?

    In general, respondents can be contacted with a new survey invitation no more than once every 7-10 days. We can exclude respondents from research in a variety of ways including but not limited to: subject matter, time period and past participation depending upon the specific needs of the research project.

16. Is there a privacy policy in place? If so, what does it state? Is the panel compliant with all regional, national, and local laws with respect to privacy, data protection and children e.g. EU Safe Harbor and COPPA in the United States? What other research industry standards do you comply with e.g. ICC/ESOMAR International Code on Market and Social Research, CASRO guidelines, etc?

    Yes, Opinion Search does have a privacy policy. The purpose of this privacy policy is to clearly communicate; the nature of the data we collect, how we manage personally identifiable information (PII), what panellist can do to update their information or be removed from our panel. We comply with all Canadian governmental and industry regulations with respect to handling data.
    Opinion Search conforms to the Marketing Research and Intelligence Association's (MRIA) Code of Conduct for Members and to the Council of American Survey Research Organizations (CASRO) Code of Standards and Ethics for Survey Research.

17. What data protection/security measures do you have in place?

    Opinion Search provides security measures against unauthorized access to our client systems including programs, files and information. The security measures provided include:

    • User Security: Users logging into the system gain level-specific access to information based upon assigned rights.
    • Network Security: Users are required to log into the network before accessing any information.
    • Survey Security: All surveys can use SSL (Secure Sockets Layer).
    • Database Security: Our databases provide security features that permits users to access only the information that is relevant to their position, including encrypted passwords, internal and external user authentication, IP address restrictions, fine-grained database privileges, and group level access control.
    • Client Project Materials: All study materials reside in restricted-access areas of our networks. Only Opinion Search staff has access to these folders.
    • Building Security: All of Opinion Search's offices require after hours swipe-card access. All IT and server rooms are housed within these buildings and require additional security clearance and swipe-card access 24 X 7. All data is subject to stringent data backup policies and practices. Automated monitoring tools alert IT Operations staffs, who are on call 24 X 7. Our Disaster Recovery Plan includes replacement of work areas, phones, data and data equipment.

18. Do you apply a quality management system? Please describe it.

    Opinion Search is committed to quality in all phases of the research process, from Proposal Development through Job Close. To this end, we have developed a Research Process Management and Improvement System which encompasses the entire project life cycle. Process Improvement teams are assembled on a regular basis to map, analyze and develop recommendations for process improvement. Each team is focused on a targeted area of the overall process. In addition, a standing committee reviews process and technology ideas that arise from production management. This committee reviews ideas against the current process and considers the impact on the overall procedure, prioritizes the ideas and then makes recommendations on which ideas should be implemented.

19. Do you conduct online surveys with children and young people? If so, please describe the process for obtaining permission?

    Opinion Search conducts data collection among children and teens for our clients in both commercial and public policy arenas. Our interviewing policies comply with the MRIA's rules for Interviewing Children and Young People contained within the Code of Conduct and Good Practice. Our typical procedure involves seeking permission from a parent or guardian in order to interview children aged 13 and younger. At the time when permission is requested, the parent/guardian is provided with a brief explanation of the research including but not limited to: the name of the agency requesting the research be conducted and the subject matter and type of questions that will be covered within the interview.

    Our policies on interviewing children and teens apply to all surveys, including those that use sample provided by Opinion Search, by our clients, or by other vendors.

20. Do you supplement your samples with samples from other providers? How do you select these partners? Is it your policy to notify a client in advance when using a third party provider? Do you de- duplicate the same when using multiple sample sources?

    In certain situations, we will supplement our samples with sample from another provider. Typically, these situations involve hard-to-reach sub- groups. Our policy is to always tell clients if we are using supplemental vendor sample. We de-dupe this sample using cookies and if desired, we offer clients the option of using digital fingerprinting.

    Opinion Search employs a comprehensive vetting process with each one of our sample vendors. We ask each vendor to complete a thorough questionnaire in an effort to best understand the composition of their panel.

21. Do you have a policy regarding multi-panel membership? What efforts do you undertake to ensure that survey results are unbiased given that some individuals belong to multiple panels?

    We have conducted extensive testing and evidence to date shows no proof that multi-panel membership alone is a cause of bias or error. We have found, however, that those with multiple panel memberships are somewhat attitudinally different in that they enjoy taking surveys and are more motivated to do so. Since our propensity score adjustment approach accounts for attitudinal correlations associated with participation in online surveys, we are confident that any possible biases have been minimized. To this end, we also can track and indentify panellists with multiple panel memberships (self-reported).

22. What are likely survey start rates, drop-out and participation rates in connection with provided sample. How are these computed?

    Amongst general panellists who are past 12-month active respondents, response rates can vary from 20% to 60% depending on the topic, incentive, population, subgroup, survey length, etc. Although response rates vary, a low response rate can still produce a representative sample. We employ a two-stage approach to ensure representativeness. In the first stage, we balance the outgoing sample demographically before survey invitations are delivered to panellists. In the second stage, we weight the resulting data we collect by a variety of factors, including but not limited to, age, sex, education, and income, region of the country, Internet usage and shopping behaviour. These two stages combined with our statistical process known as propensity score weighting helps to ensure that the resulting survey data are representative of the populations of interest to clients. In addition, our adherence to a balanced sampling methodology and best-in-class panel practices allow us to maintain low drop-out and suspend rates. Response rate is calculated as the number of responses (completes, suspend, terminate/non-qualified completion) divided by total number of invitations sent.

23. Do you maintain individual level data such as recent participation history, date of entry, source, etc. on your panellists? Are you able to supply your client with a per job analysis of such individual level data?

    Yes, we maintain this level of data on our respondents. We use this data to ensure appropriate exclusions, frequency of contact, etc. For our online research panel we also routinely balance samples based on prior survey participation.

24. Do you use data quality analysis and validation techniques to identify inattentive and fraudulent respondents? If yes, what techniques are used and at what point in the process are they applied?

    Opinion Search uses a multi-layered approach to ensure that fraudulent respondents – those who intentionally misrepresent themselves by providing inaccurate information, including those who misrepresent themselves as more than one individual by joining a panel under multiple email addresses – are detected and removed and do not affect client results.
    
    The variety of methods we use focus on the entire range of touch points that we have with our panellists from recruitment to incentive redemption to interactions with Panel Member Services and during and across all surveys taken. There is no single foolproof method for stopping fraudulent respondents (e.g., cookies can be deleted, clever respondents can lie cleverly, private internet connections have typically dynamic, non-unique IP addresses), but our multi- layered approach gives us the best possible defence against them. Our methods include: technology based solutions (e.g. CAPTCHA, cookies, digital finger printing), pattern recognition/data mining solutions, panellist accountability, survey design solutions, and panel management solutions. We continue to test and investigate survey design and technology solutions on a continual basis.

25. Do you measure respondent satisfaction?

    We measure respondent satisfaction with each survey experience based on randomly assigned multiple indicators presented at the end of each survey. We also conduct an annual panel satisfaction survey of all panellists. The results of our most recent survey show that our panellists are very highly satisfied with their experience with us. This positive feedback is further reinforced by the panel comments we receive through our Survey Help Desk. Available via phone, online, fax or mail, our Survey Help Desk manages a low complaint rate and quickly resolves questions from our panel members, ensuring long-term satisfaction.

26. What information do you provide to debrief your client after the project is finished?

    We typically provide our clients with daily updates that show the client; number of completes, incidence rate, response rate, sample outgo, etc. A final report detailing this information is provided to the client at field end.