Expert elicitation in decision analytic modelling

Expert elicitation refers to a formal, structured process of extracting opinions about uncertain quantities or probabilities from individuals considered to be experts in the relevant field. It has been used as a tool for quantifying uncertainty in a number of fields: military intelligence, nuclear industry, engineering, environmental studies and more, but remain relatively unexplored in healthcare. Potential barriers to using elicitation could be reluctance to use subjective opinion as evidence in policy making, the lack of confidence in the ability of experts to correctly estimate parameters, concerns regarding the role of bias in subjective opinion (i.e. can experts’ personal interest affect their opinion) and the lack of guidelines on how to design and conduct the elicitation process.

Dina Jankovic thesis explored how subjective priors, formally elicited from Individuals considered to be experts in their field, can be used to characterise uncertainty in cost-effectiveness decision models when empirical evidence is limited or not available. Dina recruited a range of experts for a study (different types of clinicians and Researchers, and elicited their beliefs about previously unobserved effects of an intervention designed to prevent falls in the elderly. She then explored how experts’ skills and experience affected their beliefs about the intervention. The findings have helped develop our understanding of how we define and value experts whose priors we elicit.