Plain English summary

Many decisions made by health services about which treatments to provide are informed by assessments of value for money. To do this requires a way of measuring the impact on the health of patients. Over the past decades, a very common way of doing this has been by using the EQ-5D. This is a set of 5 questions (covering mobility, ability to self-care, ability to undertake usual activities, pain and discomfort, and anxiety and depression ) that asks people whether they have no problems, some problems or severe problems. Five dimensions of health and three levels for each dimension, hence EQ-5D-3L. Each set of responses can be given a score that has been calculated from surveys of the general population.

Recently there have been moves to replace this “3-level” version of the EQ5D with a “5-level “ version. A new study has been conducted asking the English general population to score the different health states the 5-level EQ-5D can describe. It is important that these new scores are rigorously checked because we know that they will lead to different decisions about which treatments are seen as value for money. EEPRU was asked to conduct a quality assurance exercise of the 5-level values for England. This project reports the work that researchers for EEPRU undertook.

We considered how members of the general public were asked to participate in the study, who did and what they were asked to do. We obtained the data and the statistical code that were used by the researchers to generate the full set of values. We examined all of this, line by line and considered whether it was consistent with how the researchers had described their work, and whether there were any other relevant issues that might influence how reliable the results were. 

We found some quite important problems with the study. There were problems both with the data and its analysis. There was strong evidence that people responding to the questions either did not understand what they were being asked to do, or did not really think about it. We also found some quite serious problems with the way in which the data had been analysed. In the light of these issues, we recommended that it would be wise to undertake a new study making changes to the design and analysis.