Patient Retention in Research

CenExel is proud to highlight the poster presented at the Annual CNS Summit, November 2021, Boston, MA.

ABSTRACT

Introduction: Psychiatric clinical trials are beset by study participant dropouts post-randomization, with significant negative study implications. These include reduced power, internal validity, and generalizability, as well as expanded trial durations causing greater expense or the opposite which is potential early study termination. This investigation adds further insight to dropout correlates previously empirically explored (e.g., study length) as well as new variables formerly not assessed (e.g., number of scales administered) to enhance attrition understanding.

Methods: Across 6 US research sites in the east, midwest, and west coasts, 176 outpatient indication clinical trials (e.g., schizophrenia, depression, bipolar depression, post-traumatic stress disorder, attention-deficit/hyperactivity disorder, etc.) were evaluated. Analyses were conducted on completed placebo-control (PC), open-label (OL), and extension studies. Dropouts post-baseline were evaluated directly due to participant decisions or factors (e.g., lost to follow-up and withdrew consent) rather than matters out of their control (e.g., investigator discretion due to labs or adverse events).

Results: Pearson correlation analyses revealed, for PC studies across all indications and regardless of site location, retention was significantly correlated with higher visit frequencies (r=.33; p=.001), shorter study duration (r=-.36; p<.001), less scales (r=-.29; p<.01), and higher compensation (r=.25; p<.05). For extension studies, regardless if they were PC, premature discontinuation was significantly correlated with higher study visits (r=.60; p<.05) and study completion significantly correlated with higher compensation (r=.59.; p<.05).

Conclusions: The poster will discuss how the current study results compliment as well as add to previous dropout research and recommendations for protocols to increase retention while simultaneously enhancing data integrity. ​

BACKGROUND

• The need to maintain low clinical trial participant dropout cannot be underestimated because poor retention leads to:​
  • Missing data which leads to costly delays in study completion and expanded trial durations (Liu et al., 2018; Ross et al., 1999), less generalizability, premature trial termination with potentially effective drugs not reaching patients (Gul & Ali, 2010; Kadam et al., 2016), negative research site moral (Sullivan, 2004), and questionable validity of the findings (Gul & Ali, 2010; Kadam et al., 2016; Levine et al., 2015)​
• Statistical measures aimed to address poor subject accrual have also been noted as bias (Leon et al., 2006; Rabinowitz & Davidov, 2008)​
• CNS (psychiatric) clinical trials yield alarmingly low retention rates. For example:​
  • Within psychosis indication (i.e., schizophrenia and schizoaffective disorder) clinical trials, dropouts range from 33% (Wahlbeck et al., 2001), 48% (Kemmler et al. 2005), and even exceeding 50% (Martin et al., 2006)​
  • Dropout rates within placebo-controlled depression trials range from 23% to 49% (Rohden et al., 2017; Rutherford et al., 2010; Torous et al., 2020).​
• Studies exploring the reasons for high CNS trial attrition are fraught with limitations:​
  • Meta-analyses are typically conducted solely on published studies (Rabinowitz et al., 2009)​
  • Lack of exploring a variety of hypothesized dropout variables due to inadequate access to such data (e.g., participant compensation and number of measures administered in the study) (Rutherford et al., 2013)​
• Cohen et al. (2021a, 2021b) specifically surveyed study participants’ reasons for their study retention, including being treated with respect from site staff, provided free study visit transportation, and having the Informed Consent Form carefully explained by study staff​
• The goals of the current investigation were to (a) obtain a more comprehensive understanding of participant dropout reasons across a variety of CNS indications, and (b) subsequently recommend retention strategies by analyzing such clinical trials’ study design (regardless of their publication status or efficacy findings) and possessing a multitude of Independent Variables (IVs) theorized to be associated with attrition and some which have not been previously empirically explored (e.g., number of study scales and participant compensation). ​

METHODS

• Table 1 lists the research site locations who participated in the current investigation and the number and type of outpatient CNS clinical trials analyzed from those sites.​
• The trials analyzed for the present study were conducted any time in 2018 and completed (Last Patient Out) by May 2020​
• Table 2 lists the IV and Dependent Variables (DV) analyzed in this study. ​
• Dropouts were operationalized as participants withdrawing post-baseline (a) as obtained from the Closeout Report Forms submitted to Institutional Review Boards and sponsors and (b) directly due to participant decisions or factors (e.g., lost to follow-up and withdrew consent) rather than matters out of their control (e.g., investigator discretion due to labs and adverse events).​
• The IVs were obtained from the study protocol and checked by two separate persons.

​

RESULTS

• 176 studies of various psychiatric indications (see Table 1) conducted at 6 research sites across the US were analyzed.​
• Bivariate associations between the IV and DV were assessed via Pearson correlation analysis for continuous variables and point-biserial correlation analysis for a mixture of binary and continuous variables.​
  • Note we use IV and DP terminology ‘loosely’ as the statistical analyses in this study were not regressions, but rather correlational.​
• Percent Dropout (and Total n Randomized / n Completed) from all participating research sites for PC trials across CNS indications investigated in the current study were 31% (1221/848), for OL trials 36% (1091/693), and for Ext trials 34% (82/54).​
• Results indicated a significant correlation for PC studies across all indications between retention post-baseline and higher visit frequencies (r=.33; p=.001), shorter study duration (r=-.36; p<.001), higher compensation (r=.25; p<.01), and less scales (r=-.29; p<.05); see Figures 1, 2, 3, and 4, respectively. IVs not reported here were non-significantly associated with the dropout DVs.​
• For Ext studies, regardless if they were PC, premature discontinuation was significantly correlated with higher study visits (r=.60; p<.05) and study completion significantly correlated with higher compensation (r=.59.; p<.05).​
  • The PC and Ext findings are consistent in correlational direction across the study’s US sites, although definitively concluding these finds are consistent regardless of site location is difficult because the East and Midwest sites had low sample sizes (of trials) compared to the West coast sites, thus impacting statistical power.​
• No significant correlations were found within OL trial DVs and the IVs.​
• Only 6 studies provided participants with gifts. This was too low a number to accurately interpret if such gifts correlated with retention.​

DISCUSSION

• Applied to developing protocols and managing study procedures aimed at participant retention, the current study results indicate that within PC studies, regardless of indication, participants remain in a clinical trial when:​
  • They come into the research site more frequently. This finding is new to participant dropout research. Protocol developers can increase retention by having participants come onsite more frequently, as this seems to enhance their study engagement. ​
  • The trial is shorter in duration. This is consistent with Rabinowitz et al.’s (2009) results. This current finding further validates that participants ‘tire’ or lose interest when the study is too long and as such may discontinue their study involvement. Protocol developers are thus encouraged to be purposeful when designing a study’s length and visit structure so that they balance the need for data collection and retention. If reducing study duration and visits is not possible, and given the finding from the current study as well as our previous survey of patients (Cohen et al., 2021a, 2021b) who reported the importance of compensation to retention, researchers may also consider implementing interim completion compensation ‘bonuses’ that may help reduce attrition. ​
  • The study requires less administered scales. Our research team have long anecdotally noticed this too, as participants seemed to be frustrated with several study measures, often perceived by the participants as asking redundant questions. Protocol developers should consider what instruments are necessary to collect the trial’s endpoint data as opposed to over exploring.​
  • They are compensated more for their time and travel. This mirrors the findings of the Ext studies we analyzed as well as direct reporting from study participants we previously surveyed (Cohen et al., 2021a, 2021b). While this finding illustrates the importance for the research team to carefully discuss adequate compensation that would help maintain retention, it is simultaneously critical for sites to continually remind participants (such as through standardized scripts) that the success of the study hinges on their honest reporting of symptoms.​
• Current study LIMITATIONS: (a) The low number of US sites (n=6) whose studies were analyzed may reduce the generalizability of the current findings; (b) future research should delve into whether the current results may differ by indication; and (c) new findings reported in this poster (e.g., visit frequency and number of scales) should be further investigated for confirmation and a reminder that correlational data is not causational and other factors may have contributed to the current results. 

Elan A. Cohen,¹ Howard A. Hassman,¹ David P. Walling,² Vera M. Grindell,² John G. Sonnenberg,^3,4 Katarzyna Wyka,³ Brett A. English,^1,2 Jaclyn M. Lobb,¹ Djouher Hough,¹ Cassie L. Blanchard,¹ Mia Robb Stahler,¹ and Larry Ereshefsky^1,2,4

Hassman Research Institute¹; Collaborative Neuroscience Network²; Uptown Research Institute³; Northwestern University Feinberg School of Medicine⁴; The City University of New York, Graduate School of Public Health and Health Policy⁵; Retired Professor, The University of Texas⁶​