Introduction

Chronic rhinosinusitis with nasal polyps (CRSwNP) is often a severe and difficult to treat upper airway inflammatory syndrome.1 CRSwNP is both clinically and immunologically heterogeneous, which leads to complex and diverse patient clinical sub-types with different treatment outcomes.2 Unsurprisingly, a “one size fits all” approach of nasal douching and intranasal steroids, the current standard medical treatments, is often ineffective, resulting in the use of rescue systemic steroids and repeated surgery.1 There is thus a pressing need for better intervention.

Biologics targeting key T2-inflammation pathway signalling molecules such as immunoglobulin (Ig) E (omalizumab), interleukin (IL)-5 (mepolizumab), the IL-5R receptor (benralizumab), and the IL-4/IL-13 shared IL-4Rα receptor (dupilumab) demonstrate overall better clinical outcomes in CRSwNP when compared with the current standard treatment of nasal rinses and intranasal steroids, as reported in recent key randomised controlled trials (RCTs), discussed later.3 The outcomes from such RCTs paved the way for rapid FDA and EMA licensing, with the subsequent use of the drugs dupilumab, mepolizumab and omalizumab for CRSwNP. Clinicians now have an additional therapeutic option for CRSwNP, and other biologics for the treatment of CRSwNP are in development. There is, however, a compelling need to improve the design and reporting of such RCTs if clinicians are to better understand the indications and outcomes, when considering the expensive and potentially limited resource of biologic treatment.

Here we make the case for future biologic RCTs to not only follow a standardised patient recruitment and assessment approach but also to gather and present data in a way that makes it easy to extract for meta-analysis. RCTs that do not report full methodological details and analysis results in an interpretable and extractable manner prevent incorporation into meta-analysis or push reviewers to make unnecessary assumptions. This in turn compromises the strength of reporting in any meta-analysis undertaken. In this article we make suggestions as to what future data RCTs of biologics in CRS should clearly state, thereby allowing groups to undertake meta-analysis without data exclusion as a consequence of incomplete data or omission of key data sets.

RCTs of Biologics in CRSwNP

We recently undertook a meta-analysis of double-blind RCTs of biologics in CRSwNP of at least 12 week duration.3 Primary outcome measures were nasal polyp score (NPS), radiological extent of disease (Lund-Mackay score, LMS), peak nasal inspiratory flow (PNIF), olfaction (University of Pennsylvania Smell Identification Test, UPSIT), and disease-specific quality of life patient reported outcome measures (PROMs) using the 22-item Sinonasal Outcome Test (SNOT-22). Whilst the included RCTs were robust overall, significant design weaknesses together with methodology and reporting issues prevented us from fully interpreting the published data. Most notably, all studies failed to address disease heterogeneity. Pheno-endotyping CRSwNP is still novel, but only two RCTs have pre-specified enrolment goals of at least 50% with asthma, non-steroidal anti-inflammatory drugs (NSAID)-exacerbated respiratory disease (N-ERD), a more severe disease subtype, or both;4 only one such RCT had a second enrolment goal of 50% having had previous sinonasal surgery.4 Asthma prevalence varied from 48.5% to 100%, and N-ERD from 16.7% to 67% (Table 1). Some RCTs required at least one previous sinonasal operation for inclusion,5–7 while others did not8–10 and one specified no more than two surgical procedures.11

Table 1 A Summary of the Baseline Clinical Characteristics in the Selected RCTs for Meta-Analysis That May Determine Potential Response to Biologics

Inclusion criteria varied with regard to the duration of prior medical treatment from 4 to 12 weeks. Hypereosinophilia (often a marker of disease severity) was not a criterion. Thus, the RCTs failed to focus on more homogenous and severe disease groups. Subgroup analyses were not performed. Therefore, only limited data is available on relatively small patient numbers in those groups more likely to require biologics. Such relatively low numbers of patients per RCT, where time-limited recruitment periods often prevent designing studies with more ambitious recruitment targets, makes addressing efficacy in these patient subgroups difficult.

A study can be considered sufficiently powered if the overall sample size gives researchers a reasonable chance of achieving statistical significance in their primary analysis; that is, if a real drug effect truly exists. However, power calculations frequently do not incorporate disease subtypes and thus do not address target population heterogeneity. A larger sample size should better identify significant differences in the heterogenous treatment groups, which can be justified through formal sample size calculation for different disease groups prior to study commencement.

Inconsistent inclusion criteria as discussed above resulted in different baseline study populations (Table 1). Outcome measures also varied (Table 2). Such differences, further amplified by the variable treatment durations and assessment timepoints (16–52 weeks), do not allow direct comparisons between biologics. There is also a risk of reporting bias, particularly in RCTs funded by pharmaceutical companies.12 With such key limitations of data, it is only through meta-analysis of RCTs with quantitative data synthesis of treatment outcomes that insight into an intervention can be gleaned.

Table 2 A Summary of Data Extracted from Eight RCTs in a Recent Meta-Analysis of Biologic Use in CRSwNP3

Meta-Analysis

Meta-analysis allows statistical synthesis of the results of multiple RCTs and usually provides a more precise estimate of the average treatment effect than any one single study. Given that studies are likely to be designed and conducted in different ways, this analysis includes a formal assessment of the consistency of the treatment effect across studies. Thus, the two most important goals are to ascertain an overall estimate of treatment efficacy and consider whether estimates from individual studies differ markedly under certain experimental conditions.

There is an exigency to match the right patient to the right biologic. Despite several meta-analyses,3,13–15 it is still not possible to identify which patients will respond or, even more importantly, not respond to biologics, meaning that the current treatment approach remains somewhat trial and error. This is an indication for further urgent and more consistently designed RCTs16,17 We suggest what data future RCTs of biologics in CRSwNP should include in order to allow meta-analysis, and argue why journals must insist on a mandatory data presentation format that will allow easy data extraction.

Enabling Meta-Analysis

RCTs must facilitate systematic reviews and meta-analyses. They should ensure key features of study design and statistical results are easy to locate and extract. Specific to meta-analysis, it is important that RCTs routinely incorporate standard statistical results for data synthesis, as detailed below.

Meta-analysis is often conducted through a two-step process, with the primary aim of deriving some aggregated summary of the results from multiple studies. First, a statistical result typically quantifying the efficacy of a treatment is either extracted or derived indirectly from each RCT. Second, an associated standard error is extracted that quantifies the precision of such a study result, and from this an associated weighting is calculated. This weighting quantifies how influential the study result is in the meta-analysis, with larger studies generally producing more precise results and therefore receiving a higher weighting.

In an RCT, the main study result for a given outcome is referred to as the effect estimate and usually quantifies a difference between treatment and control groups at a given follow-up timepoint. This effect estimate could take the form of a difference in means (if the outcome is measured on a numeric scale), an odds ratio/relative risk/risk difference (if the outcome is binary) or a hazard ratio (if the outcome is time-to-event). Here, we attempt to address how researchers can facilitate meta-analysis when the estimate takes on any of the above forms, although we recognise that other types of effect estimate exist depending on the type of data collected.

In summary, regardless of its form, an effect estimate and its associated standard error are essential for a study’s inclusion in a meta-analysis and must therefore be accessible from the reported results. This does not necessarily imply these two quantities should be reported directly in the journal article, but rather, they should be easily derivable without access to the original dataset. We explain how this could be achieved in more detail in the remainder of this paper.

Data Extraction for Meta-Analysis

There are two main approaches that meta-analysts take to extract and derive the correct statistics for meta-analysis without access to the original dataset, and this is heavily dependent on how the authors of the RCT article have chosen to present their findings:

Meta-analysts can extract the effect estimate and standard error directly from inferential statistics presented in the journal article in a format that is ready for meta-analysis (ie, a pre-calculated effect estimate and standard error). Alternatively, meta-analysts can extract summary statistics relating to the outcome (such as the mean, standard deviation and sample size in the treatment and control groups) and use these to derive the effect estimate and standard error they require through additional calculations.

Of these two approaches, the most recent Cochrane handbook18 recommends meta-analysts extract summary data where possible (ie, approach 2). However, there are some exceptions to this recommendation, such as when statistical modelling is necessary to capture the true structure of the data or causal mechanism of the effect, as we explain below. Extracting summary data where appropriate gives meta-analysts more control to derive effect estimates and standard errors in a consistent and comparable format that facilities data synthesis. Furthermore, this allows meta-analysts to standardise the effect estimates if they wish to combine them with outcomes from other studies on different scales.19 The intentions of the meta-analyst can only partly be anticipated in advance by primary researchers, since they also depend on how other authors have chosen to present their findings in other articles. Therefore, it is recommended to allow for this flexibility by presenting such summary statistics.

We outline what is required for meta-analysts to derive the required statistics for a numeric or binary outcome in more detail below:

Binary outcomes: This type of outcome might be summarised by effect measures such as odds ratios, relative risks or risk differences when the trial seeks to compare treatment groups. However, primary researchers are advised to present the number of individuals experiencing the outcome and the number of observations this is based on (ie, sample size) with a clear indication of any missing data. Numeric outcomes: This type of outcome might be summarised by effect measures such as mean difference or standardised mean difference when treatment groups are compared. Primary researchers are advised to present the mean, standard deviation and the number of observations this is based on for each intervention/control group and timepoint. Longitudinal/Prospective Studies

RCTs are longitudinal/prospective, often with multiple follow-up timepoints. Here, it is important to adhere to the above recommendations for each timepoint. The meta-analyst might choose a particular timepoint that best suits their research question, or instead may perform analysis on multiple timepoints. Furthermore, it is important to document data attrition for each outcome to clarify the effective sample size, rather than simply the number of participants initially recruited. This also facilitates assessment of bias as a result of missing data.

Difference from Baseline Summaries

Some researchers present summaries at each timepoint, while others opt for “difference from baseline” summaries where each participant’s change from baseline is calculated and these changes form the basis of the raw data for analysis. Meta-analysts would generally prefer access to a summary at each timepoint, particularly in RCTs with reasonably balanced groups at baseline, although this is not currently a strong preference.18 Both options can be naturally synthesised with other studies in the same meta-analysis assuming they are presented on the same scale, and treatment groups are reasonably balanced at baseline.

Presenting Results from More Advanced Statistical Models

With RCTs, advanced statistical model techniques such as linear, logistic or Cox proportional hazards regression are used. These techniques more naturally output inferential statistics that include model parameter estimates, their standard errors and p-values. These models allow adjustment for other confounding baseline variables (eg, age, sex, ethnicity or even the outcome baseline value). Within the context of RCT data analysis, treatment groups are expected to be reasonably balanced if randomisation has been successful, and therefore these modelling adjustments for baseline variables are less essential than in observational studies (although still generally recommended). Researchers can use such advanced modelling techniques at their discretion where appropriate to present their results, but should note that summary statistics are usually preferred for meta-analysis involving RCTs. Exceptions to this recommendation are discussed below.

Such modelling techniques are particularly required in RCTs with a more complicated structure (this applies not just to meta-analysis). For example, where randomisation is performed in clusters and groups of patients are assigned to a treatment group that depends on the location of their care. These structures can be appropriately accounted for through hierarchical modelling, which cannot be said for more basic statistical techniques. Meta-analysts would usually prefer to extract these modelling results over the aforementioned summary statistics in this case, since they appropriately account for correlation within clusters.18

For outcomes such as time-to-event, the appropriate model is the Cox proportional hazards model.20 This will result in a hazard ratio that cannot traditionally be calculated outside of the aforementioned modelling framework. Therefore, a meta-analyst will rely solely on the results of such a modelling technique and seek to extract the log hazard ratio and its standard error (either adjusted or unadjusted for other baseline covariates) to perform data synthesis. Therefore, it is imperative that these inferential statistics are reported for any time-to-event outcome. Other techniques exist for analysing primary time-to-event data, and we recommend further reading to fully appreciate this topic.21

Example

Our recent meta-analysis of RCTs of biologics in CRSwNP3 confirmed the clinical efficacy of biologics, but with significant variation between each study in terms of inclusion criteria and study design it was difficult to compare the different biologics directly. There was variation in the duration of treatment and outcomes were measured at six different timepoints across the included studies. Whilst all studies used NPS as a primary outcome measure, other outcome measures such as LMS, UPSIT and PNIF were not recorded in all. Various PROMs were used across studies; most included SNOT-22 scores, while some recorded various different symptoms on visual analogue scales (VAS).

Table 2 summarises the data available for each paper and our methods for data extraction and synthesis. With eight identified relevant papers,4–11 we performed meta-analyses on ten outcomes, of which eight were numeric (eg, NPS) and two were time-to-event (eg time-to-surgery). Meta-analyses were further split into subgroups depending on the active drug in the given study, with two evaluating each of dupilumab, mepolizumab, benralizumab and omalizumab. Further two studies were excluded from these analyses as the data were mainly presented in a graphical format, with no suitable numbers available for meta-analysis.22,23Table 2 categorises each outcome from the included RCTs in terms of how well the presented results facilitated meta-analysis. We were able to extract summary endpoint data (means, standard deviations and sample sizes) in two studies4,11 for some outcomes to derive effect estimates and standard errors. For the remaining outcomes, we extracted these statistics directly from the papers for use in meta-analysis; this often-required extra calculations such as converting a confidence interval or p-value to a standard error. In four studies5,7–9 it was necessary to extract outcome data such as effect estimates and confidence intervals from graphs, reading from the axes using plot digitizing software (plotdigitizer.sourceforge.net), likely with some error. It would be extremely helpful for all the RCTs to have recorded the same outcome measures at the same timepoints. Given the potentially long-term use of biologics, assessment at 24 and 52 weeks seem reasonable.

Conclusion

Meta-analysts seek to extract or calculate an effect estimate and standard error for RCT outcomes, either extracted directly or derived from a clear tabulated summary (mean, standard deviation and effective sample size for numeric outcomes, assuming these statistics are appropriate for the distribution of the data), split by treatment group and presented for baseline and follow-up timepoints. Frequency data, including the number achieving the outcome and effective sample size, are required for binary outcomes. We recognise the power of regression modelling techniques for data analysis, and these results can be helpful for meta-analysts under certain described circumstances. Space constraints should prompt placing the above recommended outputs as supplementary material. The corresponding author should answer requests for further analysis results or additional files. If data protection requirements allow, raw data should be available in the public domain, to facilitate individual patient data meta-analysis if needed. This is a form of meta-analysis that involves synthesis of the raw data, often more powerful in generating more complete insight of efficacy in specific patient groups. Allergy and ENT specialists must work together to deliver better studies going forwards. As allergists and ENT surgeons both see patients with CRSwNP, future RCTS must be driven by both group of specialists who can bring different viewpoints to study presentation and interpretation. Also, as ethnic identity can determine specific CRSwNP endotypes and subsequent clinical response to biologicals,1 undertaking studies in more defined ethnic populations would be helpful. Future RCTs must test the drugs in more homogeneous disease subgroups using adequate patient numbers. RCTs must present data in a manner that facilitates easy meta-analysis.

Author Contributions

All authors made a significant contribution to the work reported, whether that is in the conception, study design, execution, acquisition of data, analysis and interpretation, or in all these areas; took part in drafting, revising or critically reviewing the article; gave final approval of the version to be published; have agreed on the journal to which the article has been submitted; and agree to be accountable for all aspects of the work.

Funding

There is no funding to report.

Disclosure

The authors report no conflicts of interest in relation to this work.

References

1. Fokkens WJ, Lund VJ, Hopkins C, et al. European position paper on rhinosinusitis and nasal polyps 2020. Rhinology. 2020;58:1–464.

2. Bachert C, Marple B, Hosemann W, et al. Endotypes of chronic rhinosinusitis with nasal polyps: pathology and possible therapeutic implications. J Allergy Clin Immunol Pract. 2020;8:1514–1519. doi:10.1016/j.jaip.2020.03.007

3. Kariyawasam HH, Chandrasekharan DP, Jacques T, et al. Biologic treatment for severe chronic rhinosinusitis with nasal polyps: a systematic review and meta-analysis. Rhinology. 2023;61:98–107. doi:10.4193/Rhin22.412

4. Bachert C, Han JK, Desrosiers M, et al. Efficacy and safety of dupilumab in patients with severe chronic rhinosinusitis with nasal polyps (LIBERTY NP SINUS-24 and LIBERTY NP SINUS-52): results from two multicentre, randomised, double-blind, placebo-controlled, parallel-group Phase 3 trials. Lancet. 2019;394:1638–1650. doi:10.1016/S0140-6736(19)31881-1

5. Bachert C, Sousa AR, Lund VJ, et al. Reduced need for surgery in severe nasal polyposis with mepolizumab: randomized trial. J Allergy Clin Immunol. 2017;140:1024–1031.e14. doi:10.1016/j.jaci.2017.05.044

6. Han JK, Bachert C, Fokkens W, et al. Mepolizumab for chronic rhinosinusitis with nasal polyps (SYNAPSE): a randomised, double-blind, placebo-controlled, phase 3 trial. Lancet Respir Med. 2021;9(10):1141–1153. doi:10.1016/S2213-2600(21)00097-7

7. Tversky J, Lane AP, Azar A. Benralizumab effect on severe chronic rhinosinusitis with nasal polyps (CRSwNP): a randomized double-blind placebo-controlled trial. Clin Exp Allergy. 2021;51:836–844. doi:10.1111/cea.13852

8. Bachert C, Han JK, Desrosiers MY, et al. Efficacy and safety of benralizumab in chronic rhinosinusitis with nasal polyps: a randomized, placebo-controlled trial. J Allergy Clin Immunol. 2022;149(4):1309–1317. doi:10.1016/j.jaci.2021.08.030

9. Gevaert P, Calus L, Van ZT, et al. Omalizumab is effective in allergic and nonallergic patients with nasal polyps and asthma. J Allergy Clin Immunol. 2013;131(1):110–116. doi:10.1016/j.jaci.2012.07.047

10. Gevaert P, Omachi TA, Corren J, et al. Efficacy and safety of omalizumab in nasal polyposis: two randomized Phase III trials. J Allergy Clin Immunol. 2020;146(3):595–605. doi:10.1016/j.jaci.2020.05.032

11. Bachert C, Mannent L, Naclerio RM, et al. Effect of subcutaneous dupilumab on nasal polyp burden in patients with chronic sinusitis and nasal polyposis: a randomized clinical trial. JAMA. 2016;315:469–479. doi:10.1001/jama.2015.19330

12. Lexchin J. Sponsorship bias in clinical research. Int J Risk Saf Med. 2012;24:233–242. doi:10.3233/JRS-2012-0574

13. Chong LY, Piromchai P, Sharp S, et al. Biologics for chronic rhinosinusitis. Cochrane Database Syst Rev. 2020;2(2):CD013513. doi:10.1002/14651858.CD013513.pub2

14. Chong LY, Piromchai P, Sharp S, et al. Biologics for chronic rhinosinusitis. Cochrane Database Syst Rev. 2021;3(3):CD013513. doi:10.1002/14651858.CD013513.pub3

15. Oykhman P, Paramo FA, Bousquet J, et al. Comparative efficacy and safety of monoclonal antibodies and aspirin desensitization for chronic rhinosinusitis with nasal polyposis: a systematic review and network meta-analysis. J Allergy Clin Immunol. 2022;149(4):1286–1295. doi:10.1016/j.jaci.2021.09.009

16. Borish L, Cohen NA, Chupp G, et al. Evaluating enrollment and outcome criteria in trials of biologics for chronic rhinosinusitis with nasal polyps. Ann Allergy Asthma Immunol. 2022;129:160–168. doi:10.1016/j.anai.2022.04.004

17. Naclerio R, Baroody F, Bachert C, et al. Clinical research needs for the management of chronic rhinosinusitis with nasal polyps in the new era of biologics: a National Institute of Allergy and Infectious Diseases Workshop. J Allergy Clin Immunol Pract. 2020;8(5):1532–1549. doi:10.1016/j.jaip.2020.02.023

18. Higgins JPT, Thomas J, Chandler J, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions Version 6.3. Cochrane; 2022.

19. Muller E, Algavi YM, Borenstein E. A meta-analysis study of the robustness and universality of gut microbiome-metabolome associations. Microbiome. 2021;9:203. doi:10.1186/s40168-021-01149-z

20. Cox DR. Regression models and life‐tables. J R Stat Soc. 1972;34:187–202. doi:10.1111/j.2517-6161.1972.tb00899.x

21. Schober P, Vetter T. Repeated measures designs and analysis of longitudinal data: if at first you do not succeed—try, try again. Anesthesia Analg. 2018;127:1. doi:10.1213/ANE.0000000000003511

22. Pinto JM, Mehta N, DiTineo M, et al. A randomized, double-blind, placebo-controlled trial of anti-IgE for chronic rhinosinusitis. Rhinology. 2010;48:318–324. doi:10.4193/Rhino09.144

23. Gevaert P, Van BN, Cattaert T, et al. Mepolizumab, a humanized anti-IL-5 mAb, as a treatment option for severe nasal polyposis. J Allergy Clin Immunol. 2011;128:989–995. doi:10.1016/j.jaci.2011.07.056