The current study and the associated SibSim app provide results with practical importance for researchers conducting and reviewing sibling control studies. Such studies can be very useful for accounting for unmeasured familial confounding. However–like other designs–the sibling control design has its sources of bias, and measurement error in the exposure is one of them [5, 15]. The current results showed that associations between exposures and outcomes can be substantially attenuated in sibling control models, only due to measurement error in the exposure. The results also showed that as the estimated association between the exposure and the outcome was deflated, the association between the family mean of the exposure and the outcome was inflated, thus increasing the risk of falsely concluding that familial confounding existed.

The current study adds to previous knowledge by providing results quantifying the risk of falsely concluding that true causal effects are confounded in sibling control studies. The association between the family mean of the exposure and the outcome may be interpreted as evidence for familial confounding. The current results showed that p-values for this estimate were heavily affected by exposure reliability and by sibling-correlations in the exposure. Further, the results showed a substantial risk of falsely concluding that familial confounding existed even when using very conservative p-value thresholds for the association between the family mean of the exposure and the outcome. For example, in several situations there was more than 30% risk of falsely concluding that truly causal effects were confounded when interpreting p-values below 0.001 as strong evidence for this association. When using p-values below 0.05 or 0.01 as evidence, there was as much as about 90% and 60% risk, respectively, for drawing false conclusions of familial confounding in several situations. These findings are in accordance with previous warnings of the need to consider aspects of a study when interpreting p-values rather than relying on arbitrary cut-offs for categorizing p-values as significant or non-significant [20]. The current study adds to this by quantifying the risk of drawing false conclusions in sibling control studies when not adequately considering measurement error and sibling correlations in the exposure.

A main point from the current study is the importance of highly reliable exposure measures in sibling control studies. The results showed that even with reliability levels often considered good or acceptable (e.g., proportion of true to total variance of 0.8 or 0.7), estimates of exposure-outcome associations were substantially deflated in some situations, and estimates of the association between the family exposure mean and the outcome were correspondingly inflated. This emphasizes that not only p-values, but also reliability, should be assessed in the context of the study. A sibling control study may require more reliable exposure measures than other studies, and studies examining exposures that are relatively highly correlated between siblings (e.g., educational level) may require particularly reliable exposures. This may be particularly relevant for studies including monozygotic twins who share 100% of their genes, thus potentially implying more highly correlated exposure variables than among other siblings. Some studies have reported observed corelations in the range 0.7 to 0.9 between monozygotic twins’ variables [30]. Even if the researcher has reason to believe the exposures are measured with reliability as high as 0.90, the current results showed that there will be substantial risk of bias with exposure correlations of this magnitude.

In the situations where reliability was equally high as the observed exposure correlations, the true sibling or twin correlations were 1.0. Hence, there was no real exposure discordance, and the sibling or co-twin control analyses were pointless [15]. Such situations may also appear in real-life studies, but this will not be evident unless the researcher acknowledges the importance of assessing the reliability of the exposure and its implications for the true sibling discordance.

Researchers might consider increasing reliability of their measures by aggregating variables, such as using multi-item scales for constructing latent variables, measuring the exposure on several occasions, or using self-report in addition to reports from friends or partners, or by explicitly modelling measurement error in the exposure [37,38,39,40]. The current results on detrimental effects of measurement error and sibling correlations are in line with previous results on continuous outcomes in linear regression models and logistic regression models with binary outcomes [5]. Regarding linear models with identity links, the effect of measurement error and sibling correlations can be calculated and corrected [5, 15]. The current study adds to this by showing results for ordinal outcome variables commonly used in questionnaire studies.

The results from the analyses of aggregated ordinals as outcomes showed attenuated association estimates even at perfect reliability of the exposure variables. As discussed in the introduction, this illustrated situations where the items only partially reflect an underlying outcome construct. Estimates were equally attenuated in the uncontrolled and the sibling control model in this situation, and the estimate of the association between the family mean of the exposure and the outcome was unbiased. Thus, using a less than perfect measure of the outcome did not lead to results that typically would be falsely interpreted as evidence for familial confounding. Nevertheless, the findings emphasize the importance of reliable measures of the outcome to obtain unbiased association estimates in uncontrolled as well as sibling control models.

The current findings illustrate that interpreting results from sibling control studies may not be straightforward, i.e., what results should be interpreted as evidence for confounding and what results should be interpreted as evidence against confounding may not be self-evident. An association between the exposure and the outcome that is attenuated in a sibling control model in addition to the presence of an association between the family mean of the exposure and the outcome, may (falsely) be interpreted as evidence that the association between the exposure and the outcome is (partially) confounded by familial factors. Another approach may be to consider any remaining association between the exposure and the outcome in a sibling control model as evidence supporting a hypothesis of causality. The SibSim app associated with this paper provides a graph showing statistical power to detect true causal effects in sibling control studies after accounting for expected attenuation of the estimate due to measurement error in the exposure. Hence, researchers can consider the power to detect true causal effects of their exposure on their outcome and at the same time consider the risk of falsely concluding that family effects are present for any given combination of exposure reliability, sibling-correlation in the exposure, sample size, true effect size, type of outcome variable and level of symmetry/asymmetry in the outcome in a real-life study. It may be important to consider statistical power to detect remaining true effects when assessing the risk of falsely concluding that familial confounding exists. Large samples will increase the statistical power to detect attenuated true causal effects. However, large samples will also increase the risk of falsely identifying familial confounding that does not exist. Small samples will reduce statistical power to detect attenuated true causal effects, but at the same time reduce the risk of falsely concluding that family effects are present. Hence, researchers may come to different conclusions regarding familial confounding depending on the combination of sample size and whether they choose to depend their conclusions on observed associations between the family exposure mean and the outcome or on remaining observed associations between the exposure and the outcome in the sibling control model. The current results emphasize the importance of triangulation when assessing causality and confounding [41]. Different research designs approach unmeasured confounding in different ways [se examples in 42, 43, 44], and results from sibling control studies should be compared to results from other designs with other sources of bias.

Additional sources of bias in sibling control studies: There are several sources of bias in sibling control studies in addition to measurement error. Mediators with shared effects on siblings may introduce bias as they will also be controlled when adjusting for shared confounders [45]. Siblings discordant for an exposure may be more different regarding confounders than unrelated individuals are, implying that nonshared confounders can introduce bias. This has been demonstrated by Frisell and colleagues [5], and in a recent simulation study by Esen and colleagues [46]. The latter study provides graphical illustrations of bias under different levels of exposure and confounder correlations, thus guiding interpretation of results from sibling control studies. The current study and the study from Esen and colleagues [46] together show that highly correlated exposures may increase bias from several sources (i.e., measurement error and nonshared confounders) in sibling control studies. The studies by Frisell and Esen [5, 46] emphasize the need to make explicit a priori assumptions of associations between exposures, outcomes, and potential mediators and confounders when using sibling control studies, for example by constructing DAGs (directed acyclic graphs) [47]. Selection effects (i.e., sibling control studies consist of discordant siblings rather than the entire population) and carry over effects (e.g., one siblings’ exposure affects the other sibling’s outcome or vice versa) may also introduce bias [13, 48], thus adding to the need for making a priori assumptions explicit.

The current results quantify bias in studies using sibling pairs. The analysis methods can also be applied for higher numbers of siblings per family, by calculating the family mean from all the participating siblings’ exposure variables and controlling each sibling’s exposure for this mean [49]. An extended version of the SibSim app (SibSimExtended) allows defining a percentage of families participating with more than two siblings, to examine bias in these situations.

Limitations: The current study has several limitations that may reduce generalizability of the findings. First, even if we have examined several different scenarios, there may be other relevant situations not included here. To remedy this, we have developed the SibSim app where researchers can examine many additional situations. We believe this will ensure that researchers can obtain simulated results relevant for a wide variety of real-life situations. Nevertheless, even the app cannot cover all possible variations in real-life studies, and there will still be scenarios for which we do not provide relevant information. Second, real-life researchers may not know the reliability of their exposure measures, thus limiting the usefulness of the current findings. However, even if the researcher does not know the exact reliability of their measures, the current results can be used to examine the risk of falsely concluding that true effects are confounded given different potential levels of reliability in the exposures, thus allowing the researcher to discuss this explicitly when interpreting the results. Third, it should be noted that monozygotic co-twin control studies can be biased if the assumption of 100% shared genetics is violated due to de novo mutations affecting health outcomes [50, 51]. Genetic effects will then not be fully controlled. Fourth, the current study has only examined one of the designs taking advantage of the genetic information in samples including siblings and/or twins. Such samples are also used to establish the origin of de novo mutations [50], improve other designs relevant for causal inference, such as Mendelian randomization [52], and in the study of epigenetic effects on health [53], to name only a few. Monozygotic twins have also been used to establish the importance of prenatal environment for health outcomes, as even monozygotic twins do not fully share environmental exposures in utero, as indexed by discordance in birth weight [54].

Conclusion: Sibling control studies can be valuable for accounting for unmeasured familial confounding between risk factors and health outcomes within the field of epidemiology. However, sibling control studies may also introduce bias when there is measurement error in the exposure. The current study showed that results from such studies can be substantially biased even at relatively high levels of reliability in the exposure. Also, the results showed that as the exposure-outcome association was deflated due to measurement error, the association between the family mean of the exposure and the outcome got correspondingly inflated. This association may be interpreted as reflecting familial confounding. The p-value of this association may inform the researcher’s interpretation of its relevance, and the current study showed substantial risk of falsely concluding that true causal effects were confounded in several situations even with relatively conservative interpretations of p-values.

We have developed the SibSim app where epidemiological researchers can examine many different situations not included in the paper. The current paper and the associated app provide results with practical relevance for researchers conducting or reviewing sibling control studies and may contribute to reducing the risk of drawing false conclusions of familial confounding from such studies. This may increase the validity and utility of sibling control studies within the field of epidemiology in the future.