Replicability and generalizability in population psychiatric neuroimaging

Paus T. Population neuroscience: why and how. Hum Brain Mapp. 2010;31:891.

Article  PubMed  PubMed Central  Google Scholar 

Rashid B, Calhoun V. Towards a brain‐based predictome of mental illness. Hum Brain Mapp. 2020;41:3468.

Article  PubMed  PubMed Central  Google Scholar 

Sui J, Jiang R, Bustillo J, Calhoun V. Neuroimaging-based individualized prediction of cognition and behavior for mental disorders and health: methods and promises. Biol Psychiatry. 2020;88:818–28.

Article  PubMed  PubMed Central  Google Scholar 

Tervo-Clemmens B, Marek S, Barch DM. Tailoring psychiatric neuroimaging to translational goals. JAMA Psychiatry. 2023;80:765–6.

Article  PubMed  PubMed Central  Google Scholar 

Szucs D, Ioannidis JPA. Empirical assessment of published effect sizes and power in the recent cognitive neuroscience and psychology literature. PLoS Biol. 2017;15:e2000797.

Article  PubMed  PubMed Central  Google Scholar 

Poldrack RA, Baker CI, Durnez J, Gorgolewski KJ, Matthews PM, Munafò MR, et al. Scanning the horizon: towards transparent and reproducible neuroimaging research. Nat Rev Neurosci. 2017;18:115–26.

Article  CAS  PubMed  PubMed Central  Google Scholar 

Kenett RS, Shmueli G. Clarifying the terminology that describes scientific reproducibility. Nat Methods. 2015;12:699.

Article  CAS  PubMed  Google Scholar 

Button KS, Ioannidis JPA, Mokrysz C, Nosek BA, Flint J, Robinson ESJ, et al. Confidence and precision increase with high statistical power. Nat Rev Neurosci. 2013;14:585–585.

Article  CAS  PubMed  Google Scholar 

Schönbrodt FD, Perugini M. At what sample size do correlations stabilize? J Res Personal. 2013;47:609–12.

Volkow ND, Koob GF, Croyle RT, Bianchi DW, Gordon JA, Koroshetz WJ, et al. The conception of the ABCD study: from substance use to a broad NIH collaboration. Dev Cogn Neurosci. 2018;32:4–7.

Article  PubMed  Google Scholar 

Szucs D, Ioannidis JP. Sample size evolution in neuroimaging research: an evaluation of highly-cited studies (1990-2012) and of latest practices (2017-2018) in high-impact journals. Neuroimage. 2020;221:117164.

Dick AS, Lopez DA, Watts AL, Heeringa S, Reuter C, Bartsch H, et al. Meaningful associations in the adolescent brain cognitive development study. Neuroimage. 2021;239:118262.

Article  PubMed  Google Scholar 

Button KS, Ioannidis JPA, Mokrysz C, Nosek BA, Flint J, Robinson ESJ, et al. Power failure: why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci. 2013;14:365–76.

Article  CAS  PubMed  Google Scholar 

Marek S, Tervo-Clemmens B, Calabro FJ, Montez DF, Kay BP, Hatoum AS, et al. Reproducible brain-wide association studies require thousands of individuals. Nature. 2022;603:654–60.

Article  CAS  PubMed  PubMed Central  Google Scholar 

Tervo-Clemmens B, Marek S, Chauvin RJ, Van AN, Kay BP, Laumann TO, et al. Reply to: multivariate BWAS can be replicable with moderate sample sizes. Nature. 2023;615:E8–E12.

Article  CAS  PubMed  PubMed Central  Google Scholar 

Gratton C, Nelson SM, Gordon EM. Brain-behavior correlations: two paths toward reliability. Neuron. 2022;110:1446–9.

Article  CAS  PubMed  Google Scholar 

Liu S, Abdellaoui A, Verweij KJH, van Wingen GA. Replicable brain-phenotype associations require large-scale neuroimaging data. Nat Hum Behav. 2023;7:1344–56.

Article  PubMed  Google Scholar 

Ioannidis JPA. Why most published research findings are false. PLoS Med. 2005;2:e124.

Sterne JAC, Smith GD. Sifting the evidence—what’s wrong with significance tests? Br Med J. 2001;322:226.

Article  CAS  Google Scholar 

Joober R, Schmitz N, Annable L, Boksa P. Publication bias: what are the challenges and can they be overcome? J Psychiatry Neurosci. 2012;37:149.

Article  PubMed  PubMed Central  Google Scholar 

Ioannidis JPA. Effectiveness of antidepressants: an evidence myth constructed from a thousand randomized trials? Philos Ethics Humanit Med. 2008;3:14.

Article  PubMed  PubMed Central  Google Scholar 

Lin L, Chu H. Quantifying publication bias in meta-analysis. Biometrics. 2018;74:785.

Article  PubMed  Google Scholar 

Chen J, Tam A, Kebets V, Orban C, Ooi LQR, Asplund CL, et al. Shared and unique brain network features predict cognitive, personality, and mental health scores in the ABCD study. Nat Commun. 2022;13:2217.

Article  CAS  PubMed  PubMed Central  Google Scholar 

Makowski C, Brown TT, Zhao W, Hagler DJ Jr, Parekh P, Garavan H, et al. Leveraging the adolescent brain cognitive development study to improve behavioral prediction from neuroimaging in smaller replication samples. Cereb Cortex. 2024;34:bhae223.

Spisak T, Bingel U, Wager TD. Multivariate BWAS can be replicable with moderate sample sizes. Nature. 2023;615:E4–E7.

Article  CAS  PubMed  PubMed Central  Google Scholar 

Varoquaux G. Cross-validation failure: small sample sizes lead to large error bars. Neuroimage. 2018;180:68–77.

Article  PubMed  Google Scholar 

OSF. https://osf.io/preprints/psyarxiv/c8xwe. Accessed 11 July 2024.

Schulz M-A, Bzdok D, Haufe S, Haynes J-D, Ritter K. Performance reserves in brain-imaging-based phenotype prediction. Cell Rep. 2024;43:113597.

Article  CAS  PubMed  Google Scholar 

Zhao W, Makowski C, Hagler DJ, Garavan HP, Thompson WK, Greene DJ, et al. Task fMRI paradigms may capture more behaviorally relevant information than resting-state functional connectivity. Neuroimage. 2023;270:119946.

Article  PubMed  Google Scholar 

Greene AS, Gao S, Scheinost D, Constable RT. Task-induced brain state manipulation improves prediction of individual traits. Nat Commun. 2018;9:2807.

Article  PubMed  PubMed Central  Google Scholar 

Task-induced brain connectivity promotes the detection of individual differences in brain-behavior relationships. Neuroimage. 2020;207:116370.

Aboraya A, France C, Young J, Curci K, LePage J. The validity of psychiatric diagnosis revisited: the clinician’s guide to improve the validity of psychiatric diagnosis. Psychiatry. 2005;2:48–55.

Feczko E, Miranda-Dominguez O, Marr M, Graham AM, Nigg JT, Fair DA. The heterogeneity problem: approaches to identify psychiatric subtypes. Trends Cogn Sci. 2019;23:584–601.

Tiego J, Martin EA, DeYoung CG, Hagan K, Cooper SE, Pasion R, et al. Precision behavioral phenotyping as a strategy for uncovering the biological correlates of psychopathology. Nat Ment Health. 2023;1:304.

Article  PubMed  PubMed Central  Google Scholar 

Nikolaidis A, Chen AA, He X, Shinohara R, Vogelstein J, Milham M, et al. Suboptimal phenotypic reliability impedes reproducible human neuroscience. bioRxiv. 2022:2022.07.22.501193.

Dhamala E, Yeo BT, Holmes AJ. One size does not fit all: methodological considerations for brain-based predictive modeling in psychiatry. Biol Psychiatry. 2023;93:717–28.

Li J, Bzdok D, Chen J, Tam A, Ooi LQR, Holmes AJ, et al. Cross-ethnicity/race generalization failure of behavioral prediction from resting-state functional connectivity. Sci Adv. 2022;8:eabj1812.

Article  PubMed  PubMed Central  Google Scholar 

Dhamala E, Ooi LQR, Chen J, Kong R, Anderson KM, Chin R, et al. Proportional intracranial volume correction differentially biases behavioral predictions across neuroanatomical features, sexes, and development. Neuroimage. 2022;260:119485.

Article  PubMed  Google Scholar 

Westreich D, Edwards JK, Lesko CR, Cole SR, Stuart EA. Target validity and the hierarchy of study designs. Am J Epidemiol. 2019;188:438.

Article  PubMed  Google Scholar 

Henrich J. The WEIRDest People in the World: how the West became psychologically peculiar and particularly prosperous. Farrar, Straus and Giroux, 2020.

Ricard JA, Parker TC, Dhamala E, Kwasa J, Allsop A, Holmes AJ. Confronting racially exclusionary practices in the acquisition and analyses of neuroimaging data. Nat Neurosci. 2023;26:4–11.

Article  CAS  PubMed 

Comments (0)

No login
gif