A mock pharmacy and therapeutics (P&T) or health technology assessment (HTA) committee meeting is one of several methods manufacturers can use to predict payers’ formulary and medical policy decisions, like forecasting payers’ reactions to an upcoming product launch. On the surface, gaining insights through a simulated group decision-making process seems like a good idea. However, thorny issues call into question the utility of this approach.
Understanding why mock committee meetings may fail to deliver valid insights is useful for two reasons. First, marketers and researchers may question whether a mock committee meeting should be conducted and, if conducted, how it should be designed. Second, these managers should be aware of how committee meetings differ in the real world.
Time warp
The format of a mock committee meeting often resembles how a real committee meeting unfolds. At a high level, there is a presentation, a discussion, and then a vote that yields a recommendation or decision. The decision-making process assumed in mock committee meetings is typically not a problem, but the amount of time spent on each step of the process is often much greater during a mock meeting.
Participants of a mock meeting are often exposed to data that either are not reviewed or given only a cursory review during a real meeting. For example, a clinical pharmacist in the real world may exclude or only touch on types of data that are routinely rejected or discounted by the committee (e.g., manufacturer-sponsored budget impact models or small differences that are statistically but not clinically significant). And a mock committee chair may unnaturally question and probe to satisfy the sponsor’s information needs. All told, a mock committee meeting focused on a single agenda item could easily run a couple of hours or more.
Yikes! That’s not how this process typically unfolds in the real world. The vast majority of real-world recommendations and decisions are not made in hours, but minutes — often 10 or fewer. And while some product reviews and discussions are longer due to the vexing nature of the problem, committees have to move aggressively through very busy agendas requiring many recommendations or decisions to be made. Dragging out one or more steps may be done to make the sponsor perceive that it is getting its money’s worth, but doing so pulls the simulation away from reality and calls into question whether the simulated outcomes are valid.
Role play
A mock committee meeting worth its salt has participants assume roles that match their actual professional roles, such as a clinical pharmacist cast as a clinical pharmacist or a medical director playing a medical director. Having participants pretend to be employees of or advisors to a common organization, however, is problematic. And asking them to replay a scene assuming additional organizational identities further strains the reasonableness of the exercise.
Nearly all participants of mock committee meetings work for different payers. Payers vary on many dimensions, including readily observable firmographic measures, as well other attributes that are difficult to see or quantify and whose impact is difficult to measure (e.g., corporate philosophy and culture). Asking participants to represent an outside organization is, then, challenging — even with a great script, good direction and, as needed, cues. For example, a pharmacy director who works for a national payer with a large commercial book of business may be asked to play a pharmacy director representing a locally dominant Blues plan in the South with a large Medicare population. It is hard to imagine the necessary information and preparation that would adequately prepare an actor to deliver a convincing performance.
Asking participants to conduct a meeting with others whom they do not regularly work with is another serious error. In the real world, committees develop norms that determine who typically speaks to which issues and whose opinion matters more on a specific topic. Prior outcomes resulting from these dynamics do influence future actions. A shared history, then, has a powerful and reliable impact on committee meetings — and this is difficult, if not impossible, to replicate. Further, participants can work for competing organizations, which may lead them to withhold insights or send false signals during mock committee meetings.
And that’s not all, folks
Curiously, a primary objective of mock committee meetings is to predict market access decisions. In the real world, some committees make only access recommendations — not decisions — or review only a product’s clinical aspects, leaving the economics and final decisions to a separate administrative committee. Some sponsors want to maximize the investment and use the mock committee to test multiple clinical and/or economic scenarios. But testing multiple scenarios extends the time, can introduce order bias, and can fatigue participants. Most sponsors want to observe — or worse — participate in the meeting. This can have a negative impact on recruiting efforts and bias those who participate.
If the intent of a mock meeting is to generate valid insights, then manufacturers could, instead, engage committee members through one-on-one interviews that take less time and money to execute than mock committee meetings. In rare cases where exploring an interaction is arguably advantageous — such as when differences in training, experience, and roles and responsibilities may yield unpredictable differences in opinion, or when it is unclear how conflicts will likely get resolved — then conducting dyads (e.g., interviews with both a qualified pharmacy director and a qualified medical director) should do the trick.
Simulations are supposed to reflect real-world situations, but mock committee meetings fail to mimic real meetings on multiple levels. And to the extent that the vast majority of mock committee meetings accurately predict recommendations and decisions, so do in-depth interviews with real committee members. It is hard to make a case that a mock committee meeting adds value and justifies the time and expense. The time has come to remove the mock committee from our consideration set.
No Comments