Four Disadvantages of Online Mock Trials

Because of the coronavirus, many of our interactive research activities – like mock trials and focus groups – have switched to an online format. In many ways, the pandemic merely hastened a transition from in-person or analogue ways of conducting research, to an online and digitized medium.  For example, telephonic surveys were the way that we conducted our survey research until online survey software, like SnapSurveysQualtrics, and SurveyMonkey became widely available.  Online surveys were significantly less expensive than those conducted via telephone, and over time, the quality of the data from online surveys improved as people became accustomed speaking, or typing, their responses to open-ended questions into online survey software.  

Though less expensive, online focus groups and mock trials were less appealing than online surveys to clients because online conferencing software was seen as a poor facsimile to an in-person trial, but the move to online jury trials that resulted from the pandemic changed this way of thinking about online mock trial research.  We have found that more of our clients are open to online focus groups and mock trials because actual trials are now moving online, and because everyone has grown familiar with online conferencing software, like Zoom, Microsoft TeamsGoogle Meet, or GoToMeeting

Even as we appear to be existing the social distancing requirements of the pandemic, debates about Constitutionality of moving civil and criminal to an online format continue.  Many courts are experiencing first hand the advantages and disadvantages to the online, or Zoom-trial format.  But as courts likely step back into an in-person format for actual trials, we anticipate that online mock trial and focus group research is likely to comprise a large percentage of our pretrial research activities.  

In these posts we explore the advantages and disadvantages of online focus group and mock trial research.   There are many advantages to getting mock juror feedback online, but it is also important to keep in mind the following potential limitations of online mock trial and focus group research:

Fewer Participants in Discussion Groups

For in-person exercises, we attempt to divide participants into groups of 12 people because, for state court cases, that is the number of people who will be deliberating at trial.  For federal court cases, we tend to divide participants into 6 to 12 deliberating groups depending on what the judge has decided the composition of the jury will be, as well as our client’s financial constraints, and/or research design issues.  

Studies on virtual group dynamics have thus far been silent about the optimal number of participants for online discussions. For online jury exercises, we have found that the maximum number of participants in a discussion session is nine people. This is, in part, dictated by the limitations of being on a screen because, if you have more than nine people, then you are likely going to need more than one screen. But the nature of the technology makes it more difficult to have larger groups of respondents participate in a meaningful way.  For example, in online discussions, when multiple people are talking at once, you are often not able to hear any of them.  Though we encourage people not to speak at the same time, this inevitably happens during any group discussion, virtual or online.  When discussion groups are larger, there is a diminishing return on each person you are less likely to get useful information from all participants.  

Shorter Duration

It is more difficult for participants to focus on presentations and to complete online questionnaires while they are not participating in-person. Before the pandemic hit, we did not recommend conducting an online mock trial exercise of longer than four hours.  However, we’ve concluded, from our recent projects, that longer online mock trials still provide quality feedback on key case issues and presentation style. Still, we don’t recommend going for more than eight hours in a single day.

More Difficult to Assess Non-Verbal Communication Cues

It is often more difficult to assess participants’ engagement and comprehension when they are not in the same room.  Encouraging online mock jurors to use the chat function in the online platform provides an opportunity for mock jurors to express their views, but some non-verbal cues may be missed during online mock trials.  Together with our online technology panel providers, we work to ensure that participants are paying attention during the online research exercise.  This monitoring is very similar to what is done during in-person research exercises.  

Security Concerns

The screening and online surveys completed by participants prior to our research exercises reveal those mock jurors who have connections parties or potential witnesses.  Participants are not allowed to complete our online surveys unless they provide an assurance of confidentiality.  Participants show us their driver’s license at the beginning of the research to confirm their identity, and their eligibility as jurors.  However, as is true with in-person research, some participants may be tempted to reveal the substance of the case to others.  While we take steps to minimize this at every stage of our research, attorneys should be aware of this risk.