3.1 Type 1 Obsolescence—The Value Set Is Misaligned with Normative Views

We believe that a value set is obsolete in a particular context if the decision-making body advocates for or moves toward a different normative basis for deriving value sets. For example, given the increasing focus on HTA incorporating patients’ perspectives, there may be a shift toward seeking patients’ values for HRQoL [20]. Similarly, in the valuation of child health, there is increasing interest from stakeholders (e.g., in the USA, UK, and elsewhere) in HRQoL values reflecting children’s own views about their health [21,22,23]. In both cases, value sets based exclusively on the stated preferences of the adult general public may become less relevant as the sole basis for generating evidence; this would typically trigger further valuation work to develop results using the preferred normative approach. On a related point, it may be that this kind of obsolescence can apply to certain kinds of analysis within the same jurisdiction. For example, value sets obtained from general population preferences may align well with what decision-making bodies need but might be less relevant and appropriate as a means of summarizing patient data in the context of PROMs programs (e.g., where the goal is to measure the performance of procedures or providers in improving patient health).

Similarly, we argue that a value set becomes obsolete if the relevant decision-making body moves away from the use of a particular HRQoL instrument to support decision-making. If a value set is for an instrument that is no longer recommended by a particular governmental body, then the value set is itself, in a sense, obsolete for that purpose but would not require a further valuation survey (but might cause the need to begin valuation of health states described using other replacement HRQoL instruments).

In an extreme case, the decision-making body might move away from the QALY metric as a central part of their processes—the recent proposal to ban use of QALYs by the US Federal Government, for example [24]. Such a move could require a complete reconsideration of how HRQoL is integrated into the decision-making process. This would depend on the selected alternative; for example, the generalized risk-adjusted cost-effectiveness (GRACE) approach [25] continues to require assessments of HRQoL. However, if the alternative paradigm did not use such measures, then the value set would be obsolete, but this obsolescence would not trigger the need for a new valuation project.

3.2 Type 2 Obsolescence—Methods Used Have Become Outdated and/or Unreliable

A wide range of changes in valuation methods have occurred in recent years, including the type of stated preference tasks [26], mode of task administration, quality control processes [27], data analysis, and modeling methods [28]. These changes arise for a variety of reasons. Some arise as pragmatic responses to circumstances, such as the shift to online interviews as a result of the pandemic [29,30,31]. Others arise from changes in underlying theoretical emphasis, such as recent discussions over the role of time preference in trade-offs between the quality and duration of life, leading to interest in nonlinear discrete choice experiment (DCE) methods [32]. Such changes can be broken down into those supported by strong scientific evidence around methods superiority and those that represent a change in approach preferred by methodologists (e.g., because they prefer one kind of underlying theory to another for instance, such as random utility theory versus utility under uncertainty) [33], although the dichotomy will often be much less clear, with changes reflecting elements of both.

If original data analysis can be updated to, for example, run a different model or exclude data no longer considered reliable, then analysis can simply be re-run. Hence, the value set might be obsolete but amenable to updating, thus not necessitating new valuation data to be collected.

However, if the original data analysis that generated the value set cannot be updated, then it may be worth exploring whether the magnitude of the effect can be estimated and therefore inform a decision of whether or not to rely on the older value set or to conduct new valuation work. One option here would be to run a small methodological study using previous and new methods to quantify the difference. If it is demonstrated to exist and to be of an adequately large size to matter (however that might be defined), then that would then potentially be a trigger to conduct a larger valuation study using updated methods.

3.3 Type 3 Obsolescence—Populations Have Changed since the Original Valuation Work

Over time, populations may change both in terms of their demographic composition (type 3a) and their preferences (type 3b). With respect to 3a, even if the average preferences of any one sub-group of society (e.g., defined by age, culture, or any other factor(s)) remain unchanged through time, a change in the composition of the population (e.g., arising through an ageing population or through patterns of immigration) could change the overall average “societal” preferences. Recent data from Jonker [14] suggested that the effect of changing population composition is modest, but this conclusion may not generalize to specific shifts in population composition or over much longer periods of time. With respect to 3b, changes in health-state preferences might plausibly arise through time as a result of changing societal expectations about HRQoL, i.e., greater awareness of types of health problems (e.g., mental health), and as a result of relevant issues being debated at a societal level (e.g., experiences relating to the coronavirus disease-2019 (COVID-19) pandemic, euthanasia [34], end-of-life care, or abortion).

The kinds of changes in 3a and 3b might be addressed in quite different ways, with the latter relatively more likely to trigger new valuation data collection. Regarding population compositional change (type 3a), existing data can, in principle, be reweighted to explore the magnitude of the effect and to potentially develop an updated value set. However, this is dependent on there being adequate data for growing population sub-groups in the original dataset, which may not be the case for situations such as growth in immigration from countries from which people did not previously emigrate in large numbers.

Regarding preference change independent of population composition (type 3b), it may be that preferences can be monitored using a standard, low-cost survey, which can, if results indicate a change, trigger a larger valuation study. If underlying preferences have changed, then that represents evidence that the original value set has moved toward obsolescence. However, it is also important to ensure that any change has restabilized around new norms, potentially through a series of low-cost surveys. Further, a challenge here is that small quantitative studies are likely to only be able to detect very large changes in preferences; one possibility is to use results from this kind of small quantitative study as a prior in an expected value of perfect information (EVPI)-type framework to judge the value of a larger valuation study.

3.4 Type 4 Obsolescence—The Instrument Has Changed and Now the Value Set Is Not an Exact Match for the Descriptive System

The development of value sets to accompany existing instruments occurs after considerable instrument development and refinement. While instrument developers will tend to finalize an instrument before valuation commences, it may be that evidence accrues around the appropriateness of the instrument subsequent to valuation work being disseminated. For example, it may be that the severity of levels is different between countries owing to the challenge of translation. If any such issues prompt the developer to update the instrument, then it may be that any valuation work done on the outdated version of the instrument is similarly obsolete. The question to be addressed is whether the change in wording is likely to produce different values if the same valuation study were conducted using updated wording.