In today's day and age, the role of the physician-scientist is paramount to advancing the field of medicine. Although most people often associate the duties of physicians with their responsibilities in a clinical realm, it is of the utmost importance that physicians actively contribute to bettering the field through original research. It often is understood that those who hold positions at large academic medical centers are expected to produce research, and oftentimes, the trajectory of their careers is held contingent on their very ability to do so.1

One of the most convenient proxies for measuring such productivity often stems down to one's ability to produce. However, because of this, many academic physicians have focused on quantity rather than quality when it comes to publishing their work.2 Consequently, to circumvent meaningless and mass production of research articles, the National Institute of Health developed a new metric termed the relative citation ratio (RCR), which many view as a favorable alternative to other indices such as total number of publications, journal impact factors, and the h-index.3,4 This metric utilizes the cocitation network of an article to standardize its overall impact relative to publications within the same field and, ultimately, across different fields.5,6 Recent studies have investigated RCR analyses and their impact on career trajectory among hip-and-knee fellowship-trained graduates.2 Nonetheless, no studies have yet reported how RCR scores apply to graduates of American Orthopaedic Foot and Ankle Society (AOFAS)–accredited programs. Our study seeks to provide benchmark data for RCR scores within the field of foot and ankle surgery and to identify correlations between these scores and various physician factors including sex, presence of a PhD, presence of a DO or MD degree, whether they were a foreign medical graduate, and academic title (ie, assistant, associate, and professor). The results of this study hope to provide us with a more accurate measure of research impact within foot and ankle surgery and can be used for interpersonal and interdepartmental evaluation, and cross-fellowship comparisons. Our data reveal important implications in the assessment of grant outcomes, departmental promotions, and overall evaluation of research influence within the foot and ankle community.

Methods Study Cohort

Fellow names listed on the AOFAS website (https://www.aofas.org/education/fellowship-match-program/orthopaedic-foot-and-ankle-fellows) were collected. This website collects graduated Orthopaedic Foot and Ankle Fellowship fellows' names from the year 2008 to 2009 to the year 2022 to 2023. Names were cross-referenced through PubMed and internet search to make certain of the spelling. Using internet search, information on the fellows such as sex, degrees, and international medical graduate status was collected. In addition, internet searches were conducted to determine a graduated fellow's academic position. This was classified as either “Assistant,” “Associate,” and “Full Professor” for the respective levels of professorships. When a graduated fellow was found to not have an academic position, they were classified as “Other.” No effective method was found to determine occupation history to account for changes from academic to nonacademic positions, or vice versa. Sources of this search included hospital websites, CVs of graduate fellows, and profiles such as LinkedIn profiles (validated with a more detailed search to find academic title).

Next, all PMIDs associated with the graduated fellows' publications were collected. In addition, the year of publication, title of the publication, authors, and journal were also collected. On the collection of this information, the PMIDs were used in the iCite Application Programming Interface (https://icite.od.nih.gov/api) to collect RCR information on each of the publications.

Relative Citation Ratio

RCR values are calculated when an article is 2 years old and are calculated by dividing the article citation rate for an individual publication by the field normalized citation rate. An RCR value of 1.0 is the “standard” values; thereby, articles with an RCR below 1.0 have a lower academic impact and articles with an RCR value greater than 1.0 have a higher academic impact.5,6 Of note, an article citation rate was calculated by number of times an article was cited divided by the number of years since publication, while the field normalized citation rate was deemed the average of the journal citation rates for National Institute of Health funded articles published in the same field.2 The value of an RCR lies within its ability to provide an objective, unbiased metric of research productivity without the confounding variable of time.2

The following four measures were calculated using this information: (1) The mean RCR (mRCR) was calculated by taking the mRCR of all of the graduated fellows' publications. (2) Total publications were calculated by summing the number of publications per graduated fellow. (3) Weighted RCR (wRCR) is defined as the sum of the RCR's by the National Institute of Health iCite website; thus, the wRCR is obtained by adding each RCR of each of a graduated fellows' publications. (4) Finally, the change in RCR after fellowship graduation was calculated by subtracting the mRCR before fellow graduated from the mRCR after they graduated.

Statistical Analysis

Univariate analysis was conducted to determine notable difference between male and female sex, earning of a PhD, a career in academics, earning a DO or MD, or holding a foreign medical degree in the four publication measures described earlier. Multivariate analysis was conducted to predict these four measures as well. In each analysis, the covariates were male versus female sex, academic position versus nonacademic position, PhD versus no PhD, M.D. versus DO, foreign medical graduate versus nonforeign medical graduate, and the years since graduation from fellowship. Years from fellowship graduation was calculated as the year that the fellows graduated according to the AOFAS website subtracted from the current year, 2023. Significance was set at P < 0.05.

All statistical analyses were conducted using R version 4.2.3 software (R Core Team [2023]; R: a language and environment for statistical computing; R Foundation for Statistical Computing [https://www.R-project.org/]).

Results General Characteristics

Through our data collection, we were able to identify 820 fellows, 674 (82%) of whom are male, 146 (18%) are female. Each year anywhere between 43 (5%) and 65 (8%) fellows graduated except for 2020, during which 96 fellows (12%) graduated. Eight fellows (1%) held PhDs, while 814 (99%) did not. In total, 587 (71%) of these fellows did not go on to hold academic positions, while 168 (20%) held the position of Assistant Professor, 65 (8%) held the position of Associate Professor, and 2 (<1%) held a full professorship. In addition, 65 fellows (8%) held DO degrees, while 757 (92%) held a non-DO degree. Finally, 17 fellows (2%) held foreign medical degrees, while 805 (98%) did not. This information is presented in Table 1. This cohort of graduated fellows had an average (SD) of 7 (8) total publications, 1.34 (1.45) mRCR, 8.44 (15.11) wRCR, and 0.06 (1.3) change in RCR after fellowship graduation (Table 2).

Table 1 - General Demographics of Graduated Foot and Ankle Fellows Demographics Number (%) Graduates per year  2009 46 (6)  2010 49 (6)  2011 43 (5)  2012 56 (7)  2013 53 (6)  2014 57 (7)  2015 58 (7)  2016 65 (8)  2017 60 (7)  2018 65 (8)  2019 62 (8)  2020 96 (12)  2021 58 (7)  2022 54 (7) Sex  Male 674 (82)  Female 146 (18) Earned PhD  Yes 8 (1)  No 814 (99) Academic title  Assistant 168 (20)  Associate 65 (8)  Full professor 2 (<1)  Other 587 (71) Earned DO  Yes 65 (8)  No 757 (92) Foreign medical degree  Yes 17 (2)  No 805 (98)
Table 2 - General Publication Statistics of Graduated Foot and Ankle Fellows Publication Statistics Mean ± SD Total publications 7 ± 8 mRCR 1.34 ± 1.45 wRCR 8.44 ± 15.11 Change in RCR after fellowship graduation 0.06 ± 1.3
Univariate Analysis

Univariate analysis on the total publications of this cohort of fellows revealed that male foot and ankle fellows had more publications than female foot and ankle fellows (7 versus 5 total publications, P = 0.041) and that doctors without a DO had more total publications as well (7 versus 4 total publications, P = 0.016) (Table 3). Univariate analysis on mRCR revealed that male foot and ankle fellows had a significantly higher mean RCR when compared with female foot and ankle fellows (1.38 versus 1.16, P < 0.001). In addition, fellows without PhDs had a lower mRCR when compared with fellows with PhDs (1.25 versus 1.34, P = 0.025). Fellows with a career in academics had a significant lower mRCR than fellows with a career outside of academics (1.28 versus 1.37, P < 0.001). Finally, fellows with a DO had a mean mRCR of 0.96, while fellows without a DO have a mean mRCR (P = 0.001) (Table 4). Comparison of wRCR showed that males (9.16 versus 5.16), fellows with a career in academics (11.19 versus 7.33), and fellows who do not hold a DO (8.69 versus 5.48) had a significantly higher wRCR (P < 0.05). This is presented in Table 5. No significant results were obtained when conducting univariate analysis on change of mRCR before graduation to after graduation (Table 6).

Table 3 - Comparison of Total Publications Mean Demographics Mean ± SD P Sex 0.041  Male 7 ± 9  Female 5 ± 6 Earned PhD 0.833  Yes 31 ± 25  No 6 ± 8 Career in academics 0.326  Yes 4 ± 6  No 7 ± 8 Earned DO 0.016  Yes 4 ± 6  No 7 ± 8 Foreign medical degree 0.515  Yes 10 ± 12  No 7 ± 8

Bold indicates statistical significance.

Table 4 - Comparison of Mean Relative Citation Ratio Demographics Mean ± SD P Sex <0.001  Male 1.38 ± 1.51  Female 1.16 ± 1.13 Earned PhD 0.025  Yes 1.25 ± 1.18  No 1.34 ± 1.45 Career in academics <0.001  Yes 1.28 ± 0.96  No 1.37 ± 1.60 Earned DO 0.001  Yes 0.96 ± 1.27  No 1.37 ± 1.46 Foreign medical degree 0.304  Yes 1.58 ± 1.53  No 1.33 ± 1.45

Bold indicates statistical significance.

Table 5 - Comparison of Weighted Relative Citation Ratio Demographics Mean ± SD P Sex <0.001  Male 9.16 ± 16.28  Female 5.16 ± 6.99 Earned PhD 0.309  Yes 14.99 ± 17.02  No 8.37 ± 15.08 Career in academics <0.001  Yes 11.19 ± 12.8  No 7.33 ± 15.81 Earned DO 0.035  Yes 5.48 ± 11.2  No 8.69 ± 15.38 Foreign medical degree 0.566  Yes 11.19 ± 19.63  No 8.38 ± 15.01

Bold indicates statistical significance.

Table 6 - Comparison of Change in Relative Citation Ratio Before Graduation With After Graduation From Fellowship Demographics Mean ± SD P Sex 0.745  Male 0.07 ± 1.37  Female 0.03 ± 0.94 Earned PhD 0.246  Yes −0.23 ± 0.64  No 0.06 ± 1.31 Career in academics 0.221  Yes −0.01 ± 0.97  No 0.09 ± 1.41 Earned DO 0.875  Yes 0.05 ± 0.74  No 0.06 ± 1.34 Foreign medical degree 0.053  Yes −0.17 ± 0.44  No 0.07 ± 1.31
Multivariate Analysis

Multivariate analysis revealed that male sex (β = 2.32, P < 0.001), holding an academic position (β = 6.44, P < 0.001), holding a PhD (β = 22.96, P < 0.001), and a shorter length time since graduation (β = −0.50, P < 0.001) were independent predictors of number of total publications (Table 7). Holding a DO degree was an independent predictor of decreased mRCR (β = 0.39, P = 0.039) (Table 8). Finally, multivariate analysis revealed that male sex (β = 4.05, P = 0.003), a career in academics (β = 4.61, P < 0.001), and a shorter time since graduation (β = −0.45, P = 0.001) were associated with a larger wRCR (Table 9). No significant predictors were found on multivariate analysis on change in mRCR before fellowship graduation to after fellowship graduation (Table 10).

Table 7 - Multivariate Regression Predicting Total Publications Predictor Beta 95% Confidence Interval P Male versus female ref. 2.32 1.02 to 3.61 <0.001 Academic versus nonacademic ref. 6.44 5.32 to 7.57 <0.001 PhD versus no PhD 22.96 17.89 to 28.03 <0.001 MD versus no MD 1.23 −0.62 to 3.08 0.193 FMG versus no FMG 2.93 −0.56 to 6.42 0.100 Years since graduation −0.50 −0.63 to −0.37 <0.001

Bold indicates statistical significance.

Table 8 - Multivariate Regression Predicting Relative Citation Ratio Mean Predictor Beta 95% Confidence Interval P Male versus female ref. 0.21 −0.05 to 0.47 0.114 Academic versus nonacademic ref. −0.08 −0.31 to 0.15 0.487 PhD versus no PhD −0.10 −1.11 to 0.92 0.854 No DO versus DO 0.39 0.02 to 0.76 0.039 FMG versus no FMG 0.20 −0.50 to 0.90 0.572 Years since graduation −0.01 −0.03 to 0.02 0.540

Bold indicates statistical significance.

Table 9 - Multivariate Regression Predicting Weighted Relative Citation Ratio Predictor Beta 95% Confidence Interval P Male versus female ref. 4.05 1.40 to 6.70 0.003 Academic versus nonacademic ref. 4.61 2.30 to 6.920 <0.001 PhD versus no PhD 5.14 −5.25 to 15.53 0.332 MD versus no MD 2.11 −1.68 to 5.90 0.274 FMG versus no FMG 2.19 −4.97 to 9.36 0.548 Years since graduation −0.45 −0.72 to −0.19 0.001

Bold indicates statistical significance.

Table 10 - Multivariate Regression Predicting Change in Relative Citation Ratio Mean Predictor Beta 95% Confidence Interval P Male versus female ref. 0.03 −0.21 to 0.26 0.830 Academic versus nonacademic ref. −0.09 −0.3 to 0.11 0.371 PhD versus no PhD −0.26 −1.17 to 0.65 0.573 MD versus no MD 0.02 −0.31 to 0.36 0.886 FMG versus no FMG −0.23 −0.86 to 0.39 0.465 Years since graduation 0.00 −0.03 to 0.02 0.740
Discussion

In this study, we investigated the research productivity and impact within the field of foot and ankle surgery by analyzing the RCR and publication metrics of 820 graduates from AOFAS-accredited programs from 2009 to 2022. The findings shed light on factors that influence research productivity and the potential implications for academic careers. Our study provides a unique perspective by analyzing research productivity in close temporal proximity to graduate medical education, which allows for a more immediate understanding of research contributions and comprehensive analysis of early career trajectories.

Research has consistently shown that female representation in orthopaedics lags behind that of other medical specialties, and this remains true for research productivity within various fields of orthopaedics.7–11 Our study, which consisted of 82% male fellows, revealed that male foot and ankle fellows had markedly greater research productivity and impact than their female counterparts as evidenced by nearly twice the number of publications and wRCR. Similarly, a study of 2,511 orthopaedic surgeons from 132 residency programs found that male surgeons had higher wRCR scores than female surgeons (29.3 versus 15.6, P < 0.001); however, the difference persisted only among assistant professors (P = 0.01) after stratification by academic rank.12 Gupta et al12 attributed these findings to time normalization in which female academic surgeons may be publishing less frequently early in their careers, and they experience a period of “catch-up productivity,” matching their male colleagues later on. Given the number of foot and ankle fellows (n = 235, 29%) pursuing academic medicine and inclusion of recent fellow graduates, we are limited in stratification by academic title; however, there was no difference by sex in change in RCR before and after fellowship graduation.

Addressing these disparities is important because research productivity is often linked to vertical mobility within the academic hierarchy.1,10,11,13–15 Elahi et al16 demonstrated that 93% of orthopaedic foot and ankle fellowship directors are men, with minimal minority representations, distinguished by their high level of research productivity . Therefore, addressing these gender-based discrepancies before foot and ankle fellowship completion is essential for promoting diversity and equality in the field and ensuring a robust research workforce.

Foot and ankle surgeons pursuing academics had higher wRCR scores but lower mRCR than those not pursuing academics, indicating that surgeons in academia publish more prolifically. Furthermore, a career in academics was an independent predictor of the total number of publications. As expected, the wRCR score of recent foot and ankle fellows pursuing academics was markedly smaller than established academic surgeons in other orthopaedic specialties, such as hand, sports, and joint reconstruction.2,10,11 These studies found that RCR was associated with advanced academic rank and longer career duration; therefore, comparisons are limited because our study cohort included fellows who graduated from a program after 2009. Although our study is the first to examine RCR as a bibliometric tool among foot and ankle surgeons, previous research has shown that a higher total number of publications are associated with a higher academic rank for foot and ankle fellowship-affiliated surgeons.17,18 Thus, the juxtaposition of early publications after fellowship completion can serve as an important benchmark for evaluating one's career trajectory within the academic setting. Additional investigations are warranted to survey research perceptions among foot and ankle surgeons who pursue a career in nonacademic positions in efforts to identify any barriers, or incentives and disincentives, to research productivity.

Our study highlights differences in research productivity based on degree. As expected, fellows with PhDs had a markedly greater number of total publications than those without. The association between research productivity and PhD acquisition has been previously reported in various medical and surgical specialties.1,12,14,15 Within orthopaedics, Gupta et al12 found that academic orthopaedic surgeons with PhDs had greater mRCR and wRCR. Conversely, we found that DO fellows had a markedly lower mRCR compared with fellows without DO degrees. These observations might reflect variations in research opportunities and emphasis on research during medical training among different degree programs. Additional investigation is warranted to explore the underlying factors influencing research productivity among medical graduates with varying degrees.

In contrast to studies of other orthopaedic specialties, our study observed a negative correlation between time since fellowship completion and research productivity.2,10,11 Similarly, Casciato et al assessed 37 foot and ankle fellowships and found that despite a growing number of fellowships, a small proportion of fellows continue publishing postfellowship. In the evaluation of 132 fellows, 27% published 3 years after graduation, whereas only 18% published more than 10 years after graduation.17 These findings were attributed to the type of employment because fellows in private and multispecialty groups were less likely to publish after graduation.17 Notably, only 29% of fellows within our study pursued a career in academics. The early years after graduation may be crucial for establishing a solid research portfolio, and continued support and mentorship during this period may be essential to sustain research productivity in the long term. Future research is warranted to develop resources to support continued research productivity for young foot and ankle fellows interested in research careers.

The use of RCR as a metric for research impact offers several advantages over standard measures such as total publications or journal impact factors. RCR accounts for the citation frequency of articles relative to the field, thereby providing a more standardized and unbiased assessment of research impact. However, it is important to recognize that RCR has its limitations and may not capture other aspects of research quality or collaboration impact. First, the data were collected from publicly available sources, and potential inaccuracies or missing information may exist. Second, the analysis is retrospective in nature, and causality cannot be established for the observed associations. Third, RCR does not account for industry or grant funding received for a particular publication, which is typically more common in the academic setting and may influence the research impact. Finally, like most bibliometric indicators, the RCR does not distinguish author seniority.

In conclusion, this study contributes to understanding research productivity and impact within foot and ankle surgery. The findings highlight the importance of addressing gender disparities and promoting research opportunities across different programs. Moreover, academic institutions should provide adequate support and mentorship to early-career physicians to foster sustained research productivity. These insights can inform policies for evaluating research output, facilitate academic promotions, and enhance research culture within the orthopaedic community. By continuously advancing the understanding of research productivity, we can empower physician-scientists to make meaningful contributions to the field of medicine and drive innovations in foot and ankle surgery.

MRCR = mean relative citation ratio, RCR = relative citation ratio, WRCR = weighted relative citation ratio

References 1. Rock CB, Prabhu AV, Fuller CD, Thomas CR, Holliday EB: Evaluation of the relative citation ratio, a new national institutes of health-supported bibliometric measure of research productivity, among academic radiation oncologists. J Am Coll Radiol 2018;15:469-474. 2. Pizzo DM, Imam N, Sudah SY, et al.: Evaluation of the national institutes of health-supported relative citation ratio among fellowship-trained American orthopaedic joint reconstruction surgery faculty: A new bibliometric measure of scientific influence. J Arthroplasty 2023;38:165-170. 3. Pulverer B: Impact fact-or fiction? EMBO J 2013;32:1651-1652. 4. Seglen PO: Why the impact factor of journals should not be used for evaluating research. BMJ 1997;314:498-502. 5. Surkis A, Spore S: The relative citation ratio: What is it and why should medical librarians care? J Med Libr Assoc 2018;106:508-513. 6. Hutchins BI, Yuan X, Anderson JM, Santangelo GM: Relative citation ratio (RCR): A new metric that uses citation rates to measure influence at the article level. PLoS Biol 2016;14:e1002541. 7. Xu RF, Varady NH, Chen AF: Trends in gender disparities in authorship of arthroplasty research. J Bone Joint Surg Am 2020;102:E131. &