Advertisement

The Emergency Medical Services Sleep Health Study: A cluster-randomized trial

Open AccessPublished:November 10, 2022DOI:https://doi.org/10.1016/j.sleh.2022.09.013

      Abstract

      Background

      Greater than half of emergency medical services (EMS) clinician shift workers report poor sleep, fatigue, and inadequate recovery between shifts. We hypothesized that EMS clinicians randomized to receive tailored sleep health education would have improved sleep quality and less fatigue compared to wait-list controls after 3 months.

      Methods

      We used a cluster-randomized, 2-arm, wait-list control study design (clinicaltrials.gov identifier: NCT04218279). Recruitment of EMS agencies (clusters) was nationwide. Our study was powered at 88% to detect a 0.4 standard deviation difference in sleep quality with 20 agencies per arm and a minimum of 10 individuals per agency. The primary outcome was measured using the Pittsburgh Sleep Quality Index (PSQI) at 3-month follow-up. Our intervention was accessible in an online, asynchronous format and comprised of 10 brief education modules that address fatigue mitigation topics prescribed by the American College of Occupational Environmental Medicine.

      Results

      In total, 36 EMS agencies and 678 individuals enrolled. Attrition at 3 months did not differ by study group (Intervention = 17.4% vs. Wait-list control = 18.2%; p = .37). Intention-to-treat analyses detected no differences in PSQI and fatigue scores at 3 months. Per protocol analyses showed the greater the number of education modules viewed, the greater the improvement in sleep quality and the greater the reduction in fatigue (p < .05).

      Conclusions

      While intention-to-treat analyses revealed no differences in sleep quality or fatigue at 3 months, per protocol findings identified select groups of EMS clinician shift workers who may benefit from sleep health education. Our findings may inform fatigue risk management programs.

      Keywords

      Background

      Severe mental and physical fatigue in shift work occupations is widespread, and according to previous research, workplace fatigue impacts greater than half of U.S.-based Emergency Medical Services (EMS) clinician first responders.
      • Patterson PD
      • Suffoletto BP
      • Kupas DF
      • Weaver MD
      • Hostler D.
      Sleep quality and fatigue among prehospital providers.
      • Patterson PD
      • Weaver MD
      • Frank RC
      • et al.
      Association between poor sleep, fatigue, and safety outcomes in Emergency Medical Services providers.
      • Patterson PD
      • Weaver MD
      • Hostler D
      EMS provider wellness.
      In addition, half of EMS clinicians in the U.S. report poor sleep quality and insufficient sleep.
      • Patterson PD
      • Suffoletto BP
      • Kupas DF
      • Weaver MD
      • Hostler D.
      Sleep quality and fatigue among prehospital providers.
      • Patterson PD
      • Weaver MD
      • Frank RC
      • et al.
      Association between poor sleep, fatigue, and safety outcomes in Emergency Medical Services providers.
      • Patterson PD
      • Weaver MD
      • Hostler D
      EMS provider wellness.
      Fatigue and disruptions in sleep have been reported in EMS/ambulance workers located in Europe and other nations,
      • Flaa TA
      • Bjorvatn B
      • Pallesen S
      • et al.
      Subjective and objective sleep among air ambulance personnel.
      ,
      • Pyper Z
      • Paterson JL.
      Fatigue and mental health in Australian rural and regional ambulance personnel.
      however, direct comparisons between nations may be difficult given differences in EMS organizational structure and deployment, scheduling, and licensure.
      • VanRooyen MJ
      • Thomas TL
      • Clem KJ.
      International Emergency Medical Services: assessment of developing prehospital systems abroad.
      In the U.S., long duration shifts, variable shift scheduling, limited time off between shifts, and other factors are linked to fatigue and poor sleep amongst EMS shift workers.
      • Patterson PD
      • Suffoletto BP
      • Kupas DF
      • Weaver MD
      • Hostler D.
      Sleep quality and fatigue among prehospital providers.
      • Patterson PD
      • Weaver MD
      • Frank RC
      • et al.
      Association between poor sleep, fatigue, and safety outcomes in Emergency Medical Services providers.
      • Patterson PD
      • Weaver MD
      • Hostler D
      EMS provider wellness.
      ,
      • Patterson PD
      • Runyon MS
      • Higgins JS
      • et al.
      Shorter versus longer shift duration to mitigate fatigue and fatigue related risks in Emergency Medical Services: a systematic review.
      ,
      • Patterson PD
      • Buysse DJ
      • Weaver MD
      • Callaway CW
      • Yealy DM.
      Recovery between work shifts among Emergency Medical Services clinicians.
      Odds of injury, error, and adverse events are higher among U.S.-based EMS clinicians who report fatigue vs. those who do not.
      • Patterson PD
      • Weaver MD
      • Frank RC
      • et al.
      Association between poor sleep, fatigue, and safety outcomes in Emergency Medical Services providers.
      Recent guidance for addressing fatigue in the EMS workplace recommends educating and training EMS clinicians on the importance of sleep and dangers of fatigue.
      • Patterson PD
      • Higgins JS
      • Van Dongen HPA
      • et al.
      Evidence-based guidelines for fatigue risk management in Emergency Medical Services.
      Unfortunately, there are few known resources for meeting the sleep health education needs of EMS shift workers in the U.S. and other nations.
      The type of work and work environment of EMS clinicians is unique, regardless of their nationality. Their volume of work is unpredictable, involves high-stress situations, long duration shifts (oftentimes 24 hours),
      • Patterson PD
      • Runyon MS
      • Higgins JS
      • et al.
      Shorter versus longer shift duration to mitigate fatigue and fatigue related risks in Emergency Medical Services: a systematic review.
      and high-risk medical procedures that are time-sensitive and play an important role in the life or death of patients. These unique work conditions have been linked to reports of work related stress, burnout, and fatigue.
      • Crowe RP
      • Fernandez AR
      • Pepe PE
      • et al.
      The association of job demands and resources with burnout among Emergency Medical Services professionals.
      ,
      • Reardon M
      • Abrahams R
      • Thyer L
      • Simpson P.
      Review article: Prevalence of burnout in paramedics: a systematic review of prevalence studies.
      This may have a pervasive impact on the safety and quality of care delivered in emergencies. A recent report shows that the U.S. EMS system is comprised of more than 20,000 agencies and nearly 1 million personnel.

      NASEMSO. 2020 National Emergency Medical Services Assessment. National Association of State EMS Officials (NASEMSO). Available at: https://nasemso.org/wp-content/uploads/2020-National-EMS-Assessment_Reduced-File-Size.pdf Accessed July 14, 2021.

      They play a dual role of emergency healthcare provider and public safety responder. The EMS clinician is often the first point of medical contact with the health care system when the chief medical complaint is an acute illness or injury. Approximately 16% of all U.S.-based hospital Emergency Department (ED) volume involves EMS transport.

      Cairns C, Kang K, Santo L. National Hospital Ambulatory Medical Care Survey: 2018 emergency department summary tables. U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics. Available at: https://www.cdc.gov/nchs/data/nhamcs/web_tables/2018-ed-web-tables-508.pdf Accessed July 14, 2021.

      The recent development of evidence-based guidelines for fatigue mitigation in the EMS setting revealed an absence of resources, programs, and interventions tailored to the EMS worker.
      • Patterson PD
      • Higgins JS
      • Van Dongen HPA
      • et al.
      Evidence-based guidelines for fatigue risk management in Emergency Medical Services.
      ,
      • Barger LK
      • Runyon MS
      • Renn ML
      • et al.
      Effect of fatigue training on safety, fatigue, and sleep in Emergency Medical Services personnel and other shift workers: a systematic review and meta-analysis.
      Sleep health education and training tailored to the unique occupational characteristics of EMS shift work may have a more positive impact than would programs targeting the general adult population or other occupational groups.
      An effort to fill this void began in 2013 when the National EMS Advisory Council, a council formed in 2007 by the U.S. Department of Transportation, recommended that federal partners examine the best available evidence and distribute resources for fatigue mitigation to EMS services nationwide. In 2015, the National Association of State EMS Officials, the University of Pittsburgh Department of Emergency Medicine, and the U.S. National Highway Traffic Safety Administration partnered on the Fatigue in EMS Project. Comprised of 3 phases, this project began with Phase 1, a combination of 7 systematic reviews of the evidence germane to fatigue mitigation and resulted in the 2018 Evidence-Based Guideline for Fatigue Mitigation in EMS.
      • Patterson PD
      • Higgins JS
      • Van Dongen HPA
      • et al.
      Evidence-based guidelines for fatigue risk management in Emergency Medical Services.
      The focus of this paper is reporting on Phase 2, an experimental study. The aim of Phase 2 was to determine if providing education and training on the importance of sleep health and dangers of fatigue impacts key indicators of sleep quality and fatigue. We hypothesized that education and training focused on sleep health and fatigue, delivered in an asynchronous manner and tailored to EMS shift workers, would result in improvements in sleep quality and a reduction in self-reported fatigue after a 3-month study interval.

      Methods

      Overview of experimental study design

      Our hypothesis was tested using a cluster-randomized, 2-arm randomized trial with a wait-list control group. All screened and eligible EMS agencies (clusters) were randomized to either the intervention group or a wait-list control group. The total study duration was 6 months to allow for crossover of wait-list participants. At 3 months of participation, the wait-list control group EMS agencies and participants were offered the intervention. Experimental designs with a wait-list comparison group are commonly used when testing the impact of sleep health related education programs.
      • Murawski B
      • Wade L
      • Plotnikoff RC
      • Lubans DR
      • Duncan MJ.
      A systematic review and meta-analysis of cognitive and behavioral interventions to improve sleep health in adults without sleep disorders.
      The study protocol was approved by the University of Pittsburgh Institutional Review Board, the Office of Management and Budget [Control Number: 2127-0742; ICR Reference Number: 201811-2127-003], and registered on ClinicalTrials.gov (NLM Identifier: NCT04218279). All study subjects provided informed consent. We report our findings in accordance with the CONSORT 2010 statement, extension to cluster randomized trials.
      • Campbell MK
      • Piaggio G
      • Elbourne DR
      • Altman DG.
      Consort 2010 statement: extension to cluster randomised trials.

      Experimental trial design

      We recruited EMS agencies (clusters) nationally and randomized each cluster to 1 of 2 possible study arms: [1] the immediate access to intervention (IAI) materials group; or the [2] wait list comparison (WLC) group. Within each cluster, we enrolled individual EMS clinician shift workers over a 30-day open enrollment period with a per-cluster goal enrollment of n = 30 and maximum enrollment of n = 50. The total duration of participation for individuals was 6 months. The WLC group was offered the intervention at 3 months. Our primary analyses compare outcomes at the 3-month timepoint.

      Setting

      Our population of interest was EMS agencies located in the United States (inclusive of Alaska and Hawaii) and their frontline EMS clinician shift workers.

      Participant eligibility

      An EMS agency was eligible if it: [1] provided 911 response and/or transport in the United States; [2] provided ground-based EMS services 24-hours-a-day (agencies limited to air-medical services only were not eligible); [3] employed n = 50 or more paid staff (agencies that used all-volunteer staffing were not eligible); and [4] did not restrict use of personal mobile phones/smartphones during shift work. The eligibility criteria for an individual EMS clinician was: [1] the clinician must be 18 years of age or older; [2] was currently working as an EMS clinician, not in an administrative only role; [3] worked a minimum of 1 shift per week; [4] worked and resided in the United States; [5] worked at an EMS agency that agreed to participate in this study; [6] had a cellular, mobile/smartphone that was capable of sending and receiving text messages; and [7] agreeable to answering online surveys and text message queries intermittently over a 6 month study period.

      Recruitment

      We recruited EMS agencies by distributing a one-page flyer to members of professional EMS organizations (eg, the National EMS Management Association, and others). We further shared the study flyer on social media and advertised the study on popular EMS news websites. Recruitment of individual EMS clinicians within EMS agencies (clusters) involved agency leaders circulating study-related information directly to potentially eligible EMS clinicians. Participants were offered remuneration totaling $35 dollars over the study period.

      Intervention

      Our intervention was developed specifically for this trial based on findings from a recent evidence review that showed [1] education and training in sleep health and fatigue has a positive impact on sleep quality among shift workers; and [2] there are few programs tailored to this population.
      • Barger LK
      • Runyon MS
      • Renn ML
      • et al.
      Effect of fatigue training on safety, fatigue, and sleep in Emergency Medical Services personnel and other shift workers: a systematic review and meta-analysis.
      We designed the Fatigue Risk Management in Emergency Medical Services Education Program based on guidance from the American College of Occupational and Environmental Medicine (ACOEM) Task Force on Fatigue Risk Management.
      • Lerman SE
      • Eskin E
      • Flower DJ
      • et al.
      Fatigue risk management in the workplace.
      We used the ACOEM recommendations as a template for designing an education program specific to the EMS occupation. Our program was comprised of 10 education modules targeting multiple topics as outlined by the ACOEM. The 10 topics included: [1] hazards of fatigue; [2] sleep physiology; [3] sleep health; [4] work related stress; [5] sleep disorders; [6] fatigue recognition; [7] adequate sleep; [8] diet and exercise; [9] alertness strategies; and [10] managing fatigue. The content of each module is comprised of: [1] a summary presentation of published evidence; [2] quiz questions about the evidence and information presented; and [3] video recordings of interviews with experts in sleep medicine, EMS administrators, and frontline EMS clinicians. We limited the duration of each module to approximately 10-15 minutes. Administrators of EMS organizations, individual EMS clinicians, and other stakeholders in the EMS work environment provided feedback, which we used to edit intervention content for clarity prior to dissemination. The Commission on Accreditation for Prehospital Continuing Education reviewed each module and approved 2.25 hours of continuing education for any individual who viewed the modules, answered a standard assessment and quiz, and applied for education credits.

      Data collection procedures

      Participants answered survey questions at baseline, 3 months, and 6 months. Participants also answered momentary assessment text message queries, which involved the previously tested, interactive SleepTrackTXT platform.
      • Patterson PD
      • Moore CG
      • Weaver MD
      • et al.
      Mobile phone text messaging intervention to improve alertness and reduce sleepiness and fatigue during shiftwork among emergency medicine clinicians: study protocol for the SleepTrackTXT pilot randomized controlled trial.
      • Patterson PD
      • Buysse DJ
      • Weaver MD
      • et al.
      Real-time fatigue reduction in emergency care clinicians: the SleepTrackTXT randomized trial.
      • Patterson PD
      • Moore CG
      • Guyette FX
      • et al.
      Fatigue mitigation with SleepTrackTXT2 in air medical emergency care systems: study protocol for a randomized controlled trial.
      • Patterson PD
      • Moore CG
      • Guyette FX
      • et al.
      Real-time fatigue mitigation with air-medical personnel: The SleepTrackTXT2 randomized trial.
      Momentary assessments occurred during shifts (intra-shift) and in between shifts (inter-shift) 7 consecutive days, followed by 2 weeks with no text message-based queries. This pattern was repeated for the duration of the study. The platform was used for queries only. We did not use the SleepTrackTXT platform for purposes of delivering intervention materials or messages. The findings from the text message queries will be reported separately.

      Outcomes

      Our primary outcome of interest was sleep quality measured by the Pittsburgh Sleep Quality Index (PSQI).
      • Buysse DJ
      • Reynolds CF
      • Monk TH
      • Berman SR
      • Kupfer DJ.
      The Pittsburgh Sleep Quality Index: a new instrument for psychiatric practice and research.
      The PSQI is a widely used 21-item survey with the total score ranging from 0-21. Respondents are asked to reflect on their sleep habits over the previous 30 days. A score greater than 5 was used to classify respondents as having poor sleep quality,
      • Buysse DJ
      • Reynolds CF
      • Monk TH
      • Berman SR
      • Kupfer DJ.
      The Pittsburgh Sleep Quality Index: a new instrument for psychiatric practice and research.
      whereas a decrease of 3 points from the first assessment to follow-up indicates a clinically meaningful improvement in sleep quality.
      • Buysse DJ
      • Germain A
      • Moul DE
      • et al.
      Efficacy of brief behavioral treatment for chronic insomnia in older adults.
      Secondary outcomes included the Chalder Fatigue Questionnaire (CFQ),
      • Chalder T
      • Berelowitz G
      • Pawlikowska T
      • et al.
      Development of a fatigue scale.
      the Epworth Sleepiness Scale (ESS),
      • Johns MW.
      A new method for measuring daytime sleepiness: the Epworth sleepiness scale.
      the EMS Safety Attitudes Questionnaire (EMS-SAQ),
      • Patterson PD
      • Huang DT
      • Fairbanks RJ
      • Wang HE.
      The Emergency Medical Services safety attitudes questionnaire.
      5 items from the Schedule Attitudes Survey (SAS),
      • Dunham RB
      • Pierce JL.
      Attitudes toward work schedules: construct definition, instrument development, and validation.
      and the 15-item Occupational Fatigue Exhaustion Recovery (OFER) survey.
      • Winwood PC
      • Winefield AH
      • Dawson D
      • Lushington K.
      Development and validation of a scale to measure work-related fatigue and recovery: the Occupational Fatigue Exhaustion/Recovery Scale (OFER).
      Survey assessments were administered online at baseline, at 3 months, and again at 6 months with a study-specific website that required participant specific login credentials. We chose 3-month follow-up as our primary time period for comparing intervention impact based on previous research,
      • Barger LK
      • Runyon MS
      • Renn ML
      • et al.
      Effect of fatigue training on safety, fatigue, and sleep in Emergency Medical Services personnel and other shift workers: a systematic review and meta-analysis.
      which tested similar interventions over a similar time period in diverse occupational settings. The survey tools selected are reliable and/or valid, and previously tested in EMS clinician shift workers.
      • Patterson PD
      • Suffoletto BP
      • Kupas DF
      • Weaver MD
      • Hostler D.
      Sleep quality and fatigue among prehospital providers.
      ,
      • Patterson PD
      • Buysse DJ
      • Weaver MD
      • Callaway CW
      • Yealy DM.
      Recovery between work shifts among Emergency Medical Services clinicians.
      ,
      • Patterson PD
      • Huang DT
      • Fairbanks RJ
      • Wang HE.
      The Emergency Medical Services safety attitudes questionnaire.
      ,
      • Patterson PD
      • Weaver MD
      • Fabio A
      • et al.
      Reliability and validity of survey instruments to measure work-related fatigue in the Emergency Medical Services setting: a systematic review.

      Power calculation

      We used preliminary data as a guide and originally powered this study at 80% to detect a 0.4 standard deviation difference in PSQI score with n = 10 EMS agencies in each group (intervention and wait-list control), n = 30 individual EMS clinicians participants per agency, and intracluster correlation (ICC) = 0.07 (2 sided α = 0.05).
      • Patterson PD
      • Suffoletto BP
      • Kupas DF
      • Weaver MD
      • Hostler D.
      Sleep quality and fatigue among prehospital providers.
      • Patterson PD
      • Weaver MD
      • Frank RC
      • et al.
      Association between poor sleep, fatigue, and safety outcomes in Emergency Medical Services providers.
      • Patterson PD
      • Weaver MD
      • Hostler D
      EMS provider wellness.
      At the study's mid-point, we determined that the number of EMS clinicians per agency was likely not attainable and re-evaluated the power with an increase in the number of EMS agencies and fewer participants per agency. With 20 EMS agencies in each group and 10 EMS clinicians per agency we retained 88% power to detect a 0.4 effect size in PSQI score. Assuming that the prevalence of fatigue in the WLC group was 50%, we had 90% power to detect an absolute 20% reduction in CFQ-measured fatigue. A power calculation was not performed for secondary measures ESS, EMS-SAQ, SAS, or OFER.

      Randomization and allocation concealment and implementation

      We randomized EMS agencies (clusters) with a biased coin minimization procedure.
      • Saghaei M
      • Saghaei S.
      Implementation of an open-source customizable minimization program for allocation of patients to parallel groups in clinical trials.
      This procedure was intended to balance the randomization by the type and size of each cluster adaptively with each additional enrollment. Type was defined as [1] fire-based model, [2] hospital-based / third-service model, and [3] air-based / other type of EMS agency model. There is no publicly available national database that characterizes the number of employees at the EMS agency level. We therefore used our collective knowledge and experience in EMS to define agency size as [1] EMS agencies with ≥300 employees vs. [2] EMS agencies with <300 employees. The study team responsible for enrollment and intervention implementation were not informed of the probabilities of assignment. Our senior statistician managed treatment allocation until each EMS agency (cluster) was judged eligible. Notification of treatment allocation was sent via email directly to the study team from the senior statistician. The EMS agency primary point of contact (ie, the EMS agency administrator) was then informed of allocation assignment. Notification of intervention (the IAI group) assignment was followed with immediate access to the intervention. Notification of assignment to the wait-list group (the WLC group) was followed with a release date and time when access to intervention materials was granted.

      Blinding

      Given the study design, participant awareness of randomization status post assignment could not be concealed. While our principal investigator and study staff were also aware of cluster and individual participant randomization status, our statistician was removed from allocation procedures and activities related to data collection of primary and secondary outcomes.

      Statistical methods

      We followed intent-to-treat principles in all primary analyses.
      • Tripepi G
      • Chesnaye NC
      • Dekker FW
      • Zoccali C
      • Jager KJ.
      Intention to treat and per protocol analysis in clinical trials.
      We initially report descriptive statistics without adjustment for agency-level clustering. We used t-tests, tests of medians, chi-square tests, and Fisher's exact tests to examine baseline differences in the characteristics of EMS agencies (clusters) and individual participants stratified by IAI and WLC group status. We used hierarchical mixed effects models to test the impact of the intervention on the changes in the primary and secondary outcomes. We accounted for clustering at the agency level (nesting participants within agencies) with random intercepts. Formal tests and visual inspections indicated the models satisfied all necessary assumptions (eg, normality, homoscedasticity, linearity). We defined exposure to an education module as any viewing of a module. Module viewing was assessed as a continuous measure and categorically. For purposes of categorical assessment, we stratified module viewing into 4 categories based on participant viewing patterns evaluated at the end of the study period: [1] No modules viewed; [2] Low—with 1-3 modules viewed; [3] Moderate—with 4-7 modules viewed; and [4] High—with 8-10 modules viewed. In addition, we performed per-protocol analyses to determine if variation in intervention exposure was associated with outcomes.
      • Tripepi G
      • Chesnaye NC
      • Dekker FW
      • Zoccali C
      • Jager KJ.
      Intention to treat and per protocol analysis in clinical trials.
      ,
      • Hernan MA
      • Hernandez-Diaz S.
      Beyond the intention-to-treat in comparative effectiveness research.
      For these analyses, we used hierarchical mixed effects models that accounted for clustering at the agency level and the dependence between repeated measures at the participant level. We used these models to characterize the relationship between exposure and the outcomes of interest, specifically among participants who viewed the education modules and completed the survey outcome assessments up to 3 and/or 6 months, separately. We examined the relative change (percentage change relative to baseline) in outcome measures stratified by module viewing status and by study group. In addition, we used the Cohen's d effect size measure and corresponding 95% confidence intervals to assess the difference in mean change of the PSQI and CFQ scores by group from baseline to 3-month follow-up.
      • Sullivan GM
      • Feinn R.
      Using effect size-or why the p value is not enough.
      We performed all analyses with the SAS statistical software version 9.4 (Cary, NC).

      Results

      In total, we screened 54 EMS agencies and confirmed 48 as eligible. Of these 48, we enrolled 36 EMS agencies (Fig. 1). Sixteen agencies were randomized to the IAI group and 20 to the WLC group (Table 1). Most agencies were moderately sized with 51-199 total employees (58.4%) and responded to 20,000 or more dispatches in 2019 (61.1%). Approximately two-thirds reported no formal fatigue management program (66.7%). Among the 36 EMS agencies enrolled, 678 individual EMS clinicians consented to participate (mean enrollments per agency 18.8, SD 14.3; min = 2, max = 51). Participant mean age was 37.4 years (SD 10.1), the mean years of experience was 12.6 years (SD 8.7), and the majority worked 12-hour or 24-hour shifts (89.4%). The demographic characteristics of individual enrollees in this study are similar to the characteristics documented in previous observational and experimental research studies involving EMS shift workers (Table 2).
      Fig 1
      Fig. 1CONSORT flow diagram. Notes: Shading signifies access to intervention materials. Some analyses and sample sizes reported in the document will involve the Immediate Access to Intervention (IAI) group and the Wait List Control (WLC) group sample sizes that differ slightly from the totals for 3-month and 6-month survey follow-up reported in this figure. These small differences are due to the fact that some participants did not complete the 3-month or 6-month follow-up, but were still considered “active participants” given their responses to other study related assessments (ie, text message queries), and given that some participants partially (or fully) completed the baseline survey, 3-month survey, and/or the 6- month survey. *n = 37 individuals assigned to the WLC group were identified as viewing one or more modules immediately prior to completing the 3-month follow-up survey.
      Table 1Agency demographics by study group
      VariableTotal agencies(N = 36)N (%)IAI group(N = 16)N (%)WLC group(N = 20)N (%)p-value
      Agency type.13
      Fisher exact test was used in categorical variables with over 20% expected cell counts less than 5.
      Fire-based12 (33.3%)5 (31.3%)7 (35.0%)
      Third-service7 (19.4%)1 (6.3%)6 (30.0%)
      Air/ground combination2 (5.6%)0 (0.0%)2 (10.0%)
      Hospital-based7 (19.4%)5 (31.3%)2 (10.0%)
      Other
      Other agency type includes non-profit and private agencies.
      8 (22.2%)5 (31.3%)3 (15.0%)
      Census region.38
      Fisher exact test was used in categorical variables with over 20% expected cell counts less than 5.
      Mid-West12 (33.3%)7 (43.8%)5 (25.0%)
      Northeast5 (13.9%)3 (18.8%)2 (10.0%)
      South12 (33.3%)3 (18.8%)9 (45.0%)
      West7 (19.4%)3 (18.8%)4 (20.0%)
      Paid employees (full-time and part-time).47
      Fisher exact test was used in categorical variables with over 20% expected cell counts less than 5.
      51-9910 (27.8%)5 (31.3%)5 (25.0%)
      100-19911 (30.6%)4 (25.0%)7 (35.0%)
      200-2992 (5.6%)2 (12.5%)0 (0.0%)
      300+13 (36.1%)5 (31.3%)8 (40.0%)
      Paid employees (full-time and part-time).59
      Smaller (<300)23 (63.9%)11 (68.8%)12 (60.0%)
      Larger (≥300)13 (36.1%)5 (31.3%)8 (40.0%)
      Total dispatch CY 2019.83
      Fisher exact test was used in categorical variables with over 20% expected cell counts less than 5.
      <10,0006 (16.7%)3 (18.8%)3 (15.0%)
      10,000-19,9998 (22.2%)3 (18.8%)5 (25.0%)
      20,000-49,99911 (30.6%)4 (25.00%)7 (35.0%)
      50,000+11 (30.6%)6 (37.5%)5 (25.0%)
      Total transports CY 2019.47
      Fisher exact test was used in categorical variables with over 20% expected cell counts less than 5.
      <10,0009 (25.0%)4 (25.0%)5 (25.0%)
      10,000-19,99912 (33.3%)4 (25.0%)8 (40.0%)
      20,000-49,9997 (19.4%)5 (31.3%)2 (10.0%)
      50,000+8 (22.2%)3 (18.8%)5 (25.0%)
      Formal fatigue program.63
      Yes12 (33.3%)6 (37.5%)6 (30.0%)
      No24 (66.7%)10 (62.5%)14 (70.0%)
      IAI = Immediate access to intervention group. WLC, waist list control group; CY, calendar year.
      The chi-square test was used for testing for group differences with categorical variables.
      & Other agency type includes non-profit and private agencies.
      # Fisher exact test was used in categorical variables with over 20% expected cell counts less than 5.
      Table 2Individual participant baseline demographics by study group
      VariableIAI(N = 302)N (%)WLC(N = 348)N (%)p-value
      Sex
      Male205 (67.9%)255 (73.3%).13
      Fisher exact test was used in categorical variables with over 20% expected cell counts less than 5. The differences in continuous variables were tested using the Mann-Whitney U test.
      Female97 (32.1%)92 (26.4%)
      Unknown0 (0%)1 (0.3%)
      Certification/license
      EMT-Basic98 (32.5%)112 (32.2%).85
      Fisher exact test was used in categorical variables with over 20% expected cell counts less than 5. The differences in continuous variables were tested using the Mann-Whitney U test.
      Paramedic187 (61.9%)214 (61.5%)
      Nurse5 (1.7%)4 (1.1%)
      Other12 (4.0%)18 (5.2%)
      Where do most work as EMS clinician
      Air-medical-based EMS7 (2.3%)12 (3.4%).74
      Hospital ED17 (5.6%)23 (6.6%)
      Ground-based EMS241 (79.8%)269 (77.3%)
      Hospital ICU0 (0%)1 (0.3%)
      Other37 (12.3%)43 (12.4%)
      Work multiple jobs
      Yes90 (29.8%)76 (21.8%).01
      p-value < .05.
      No212 (70.2%)272 (78.2%)
      Employment status
      Full-time286 (94.7%)338 (97.1%).06
      Part-time16 (5.3%)10 (2.9%)
      Type of shift commonly worked
      24-hour185 (61.5%)231 (66.6%).11
      12-hour83 (27.6%)79 (22.8%)
      8-hour11 (3.7%)4 (1.2%)
      Other22 (7.3%)33 (9.5%)
      Health
      Excellent57 (18.9%)92 (26.4%).10
      Fisher exact test was used in categorical variables with over 20% expected cell counts less than 5. The differences in continuous variables were tested using the Mann-Whitney U test.
      Good211 (69.9%)223 (64.1%)
      Fair33 (10.9%)31 (8.9%)
      Poor1 (0.3%)2 (0.6%)
      Race
      American Indian/Alaskan Native3 (1.0%)7 (2.0%).58
      Fisher exact test was used in categorical variables with over 20% expected cell counts less than 5. The differences in continuous variables were tested using the Mann-Whitney U test.
      Asian0 (0%)2 (0.6%)
      Black or African American1 (0.3%)4 (1.1%)
      White284 (94.0%)320 (92.0%)
      More than one race8 (2.6%)8 (2.3%)
      I prefer not to answer6 (2.0%)7 (2.0%)
      Ethnicity
      Hispanic or Latino16 (5.3%)18 (5.2%).46
      Not Hispanic or Latino274 (90.7%)322 (92.5%)
      I prefer not to answer12 (4.0%)8 (2.3%)
      Young children at home
      Yes155 (51.3%)177 (50.9%).45
      No147 (48.7%)171 (49.1%)
      Conditions
      Arthritis21 (7.0%)19 (5.5%).22
      Depression52 (17.2%)54 (15.6%).29
      Weight problems65 (21.5%)68 (20.0%).28
      Diabetes20 (6.6%)16 (4.6%).13
      High blood pressure73 (24.2%)58 (16.7%).01
      p-value < .05.
      Sleep apnea42 (13.9%)42 (12.1%).25
      Migraine headaches32 (10.6%)21 (6.0%).02
      p-value < .05.
      Lung/breathing problems19 (6.3%)17 (4.9%).22
      Heart problems15 (5.0%)9 (2.6%).06
      Other27 (8.9%)40 (11.5%).14
      None123 (40.7%)164 (47.1%).05
      Sleep disorders
      Yes77 (20.5%)76 (19.5%).33
      None240 (79.5%)280 (80.5%)
      Participant demographic means (± SD)
      Height (in)69.0 (3.7)69.9 (3.7).00
      p-value < .05.
      Weight (lbs.)207.5 (49.2)200.9 (45.9).06
      Years of experience15.0 (9.6)12.6 (8.7).00
      p-value < .05.
      Number of shifts worked last month13.3 (6.7)12.4 (4.7).16
      Alcoholic drinks (per week)4.5 (6.4)3.8 (4.9).08
      Cigarettes (per week)2.4 (15.1)2.8 (16.1).41
      Age (years)39.3 (9.6)37.4 (10.1).00
      p-value < .05.
      Baseline survey means (± SD)
      PSQI8.77 (3.3)8.96 (3.6).26
      % with poor sleep246 (82.0%)288 (83.7%).56
      CFQ6.64 (2.6)6.55 (2.8).37
      % with severe fatigue264 (88.3%)287 (83.7%).09
      ESS8.68 (4.2)7.99 (4.0).03
      p-value < .05.
      SAS
      Five select items from the Schedule Attitudes Survey (SAS).27
      44.2 (26.5)43.0 (22.5).33
      OFER - Chronic fatigue35.8 (25.3)36.3 (25.5).42
      OFER - Acute fatigue60.4 (22.5)60.1 (23.2).39
      OFER - Intershift recovery47.8 (25.1)48.9 (25.4).29
      EMS-SAQ - Teamwork climate69.9 (20.9)68.6 (22.3).32
      EMS-SAQ - Safety climate72.2 (19.5)73.3 (20.6).14
      EMS-SAQ - Stress recognition56.4 (21.9)59.5 (24.4).02
      p-value < .05.
      EMS-SAQ - Perceptions of management60.8 (24.8)56.9 (23.6).01
      p-value < .05.
      EMS-SAQ - Working conditions65.2 (22.7)64.0 (23.0).32
      EMS-SAQ - Job satisfaction73.1 (22.7)71.7 (23.2).19
      IAI, Immediate access to intervention group; WLC, wait list control group; EMS, emergency medical services; ED, emergency department; EMT, emergency medical technician; ICU, intensive care unit; SD, standard deviation.
      n = 678 individuals enrolled; however, n = 14 individuals randomized to the IAI group and n = 14 randomized to the WLC group did not answer the demographic component of the baseline survey. For purposes of this table, data from n = 650 total individuals are presented. Chi-square test was conducted for categorical variables.
      # Fisher exact test was used in categorical variables with over 20% expected cell counts less than 5. The differences in continuous variables were tested using the Mann-Whitney U test.
      low asterisk p-value < .05.
      ^ Five select items from the Schedule Attitudes Survey (SAS).
      • Dunham RB
      • Pierce JL.
      Attitudes toward work schedules: construct definition, instrument development, and validation.
      Individual participant baseline characteristics did not differ by study arm for most demographic variables (Table 2); however, baseline group-level differences were observed for participant height, years of experience, age, working multiple jobs, and history of select health conditions (ie, high blood pressure). In unadjusted analyses, baseline differences were also observed in mean ESS, a secondary outcome, with a higher mean score among participants in the IAI group vs. the WLC group (p = .03), and 2 domains of the EMS-SAQ (“stress recognition” and “perceptions of management”; Table 2). In analyses adjusted for clustering, baseline differences in the IAI and WLC groups in age, high blood pressure, ESS, and 2 SAQ domains disappeared.
      Over the 6-month study period, 142 participants withdrew and 109 were classified as LTFU (Fig. 1). During the first 3 months, the rate of withdrawals and LTFU combined (also known as attrition) was similar between groups (IAI = 17.4% vs. WLC = 18.2%, p = .37). Attrition during the first 3 months of the study was highest among fire-based (20.0%) and third-service model EMS agencies (20.3%), among agencies in the Southern U.S. Census region (20.1%), among agencies with 300+ employees (20.8%), among individuals that reported working full-time work (14.4%), and among individuals that regularly worked 24-hour shifts (15.9%). Attrition over the full 6-month study period did not differ by study arm (IAI = 34.5% vs. WLC = 39.2%; p = .20). Factors associated with attrition over the 6-month study period were agency type, highest among air-ground combined agencies (41.7%), region, agencies located in the Southern U.S. (39.4%), size, agencies with 300+ employees (40.8%), certification, participants with EMT-Basic certification/license (41.9%), full-time vs. part-time status, participants with full-time employment (35.1%), and shift type, participants that commonly worked 24-hour shifts (37.5%).
      Our primary comparison of interest was sleep quality as measured by the PSQI instrument at baseline and at 3 months by group status. The available data for 3-month comparisons included 15 IAI agencies with 210 individual participants, and 20 WLC agencies with 225 participants who partially or fully completed the 3-month survey.

      Primary outcome: Intent-to-treat analysis

      In intention-to-treat analyses, the mean PSQI at 3-month follow-up did not differ by group (3-month p = .74). The mean change in PSQI among IAI participants is -0.17 standard deviations greater than the mean change among WLC participants (Cohen's d = -0.17; 95% CI -0.36, 0.02; "a small effect"). The percent change from baseline also did not differ by group at 3-month follow-up (p = .10; Fig. 2). The proportion of participants with poor sleep quality did not differ by IAI or WLC group status at 3-month follow-up (p = .86 See Online Supplementary Figures). The proportion with a clinically meaningful improvement in sleep quality at 3 months, compared to baseline, did not differ by group (p = .80; See Online Supplementary Figures).
      Fig 2
      Fig. 2Percentage and raw change in Pittsburgh Sleep Quality Index (PSQI) score from baseline to 3-month follow-up. Notes: IAI, immediate access to intervention group. WLC, wait list control group. PSQI, Pittsburgh Sleep Quality Index. Standard deviation (SD) represented by error bars. The directionality of bars showing the mean percentage change and mean raw change in PSQI for the total group and WLC group differ based on [1] the change in PSQI raw scores from baseline to 3 months was 0 for 14% of participants, with decreases ranging from -1 to -13 and increases ranging from 1 to 7; and [2] the percentage change in PSQI from baseline to 3 months showed decreases ranging from -5.9% to -100% and increases ranging from 6.3% to 200%. The high percentage change scores relative to baseline values resulted in the difference in directionality of some point estimates (a point estimate of positive mean percentage change and negative mean raw change) shown in this figure.

      Secondary outcomes: Intent-to-treat analysis

      Our leading secondary measure of interest was mental and physical fatigue as measured by the CFQ. The mean CFQ score did not differ by group at 3-month follow-up (p = .45). The mean change in CFQ among IAI participants is 0.08 standard deviations greater than the mean change among WLC participants (Cohen's d = 0.08; 95% CI -0.11, 0.27; "a small effect"). The proportion of participants classified as fatigued by the CFQ did not differ by group status at 3-month follow-up (p = .86). The percentage change in CFQ score from baseline to 3-month follow-up did not differ by group status (p = .49; Fig. 3). The mean scores on secondary outcomes ESS, OFER, EMS-SAQ, and SAS did not differ by group status at 3-month follow-up (Table 3; p > .05 all comparisons).
      Fig 3
      Fig. 3Percentage and raw change in Chalder Fatigue Questionnaire (CFQ) score from baseline to 3-month follow-up. Notes: IAI, immediate access to intervention group. WLC, wait list control group. CFQ, Chalder Fatigue Questionnaire. Standard deviation (SD) represented by error bars.
      Table 3Secondary outcomes and sub-group analyses at 3-month follow-up stratified by study group status
      Survey instrument3-month follow-up
      Sub-group analysesWLC (N = 225)IAI (N = 210)
      Poor sleep quality WLC (N = 180)Poor sleep quality IAI (N = 173)
      Mean (SD) or N (%)Mean (SD) or N (%)Mean (SD) or N (%)Mean (SD) or N (%)
      PSQI9.3 (3.3)9.0 (3.5)8.5 (3.5)8.4 (3.6)
      % with poor sleepN = 158 (87.8%)N = 148 (85.6%)N = 176 (78.9%)N = 162 (77.1%)
      Change in PSQI
      p-value for sub-group analysis comparing change in PSQI p = .35.
      -0.38 (2.6)-0.73 (2.8)-0.08 (2.6)-0.53 (2.6)
      CFQ6.7 (2.6)7.0 (2.7)6.5 (2.7)
      n = 224.
      6.6 (2.9)
      % with severe fatigueN = 162 (90%)N = 153 (88.4%)N = 192 (85.3%)N = 178 (84.8%)
      ESS8.8 (4.8)8.9 (3.8)8.5 (4.1)
      n = 222.
      8.8 (3.8)
      OFER
      -Chronic fatigue39.7 (28.1)40.4 (26.1)37.8 (28.8)
      n = 219.
      37.8 (26.1)
      -Acute fatigue59.4 (24.9)60.0 (22.6)57.8 (25.6)
      n = 219.
      57.6 (24.2)
      -Intershift recovery47.4 (26.0)45.9 (25.4)49.8 (26.6)
      n = 219.
      50.7 (26.1)
      EMSSAQ
      -Teamwork climate65.3 (23.6)66.9 (22.3)66.3 (23.5)
      n = 219.
      67.7 (21.8)
      -Safety climate68.8 (23.1)69.1 (19.5)69.4 (22.9)
      n = 219.
      69.2 (19.5)
      -Stress recognition57.5 (24.1)55.5 (24.9)58.1 (24.7)
      n = 219.
      55.8 (25.4)
      -Perceptions of mgmt.51.7 (26.2)57.5 (26.1)52.1 (26.4)
      n = 219.
      58.5 (25.9)
      -Working conditions58.4 (26.1)61.3 (22.9)59.2 (25.8)
      n = 219.
      62.3 (22.6)
      -Job satisfaction67.1 (25.5)66.7 (24.3)67.2 (25.4)
      n = 219.
      68.2 (24.0)
      SAS
      Five select items from the Schedule Attitudes Survey (SAS).27
      43.4 (24.5)46.0 (28.4)41.7 (25.1)
      n = 219.
      44.0 (28.4)
      IAI, immediate access to intervention group; WLC, waist list control group; PSQI, Pittsburgh Sleep Quality Index; CFQ, Chalder Fatigue Questionnaire; ESS, Epworth Sleepiness Scale; OFER, Occupational Fatigue Exhaustion Recovery survey; EMSSAQ, Emergency Medical Services Safety Attitudes Questionnaire; SAS, Schedule Attitudes Survey.
      Tests for differences in secondary measures by group status (IAI vs. WLC) use linear mixed models (at 3 months) with the secondary measure as the outcome, group status as the predictor, and a random intercept to adjust for clustering. p > .05 for all comparisons.
      Five items out of 21 total items from the SAS^ tool were used in this study. Sample sizes are denoted by the following:
      a n = 224.
      b n = 222.
      c n = 219.
      low asterisklow asterisk p-value for sub-group analysis comparing change in PSQI p = .35.
      ^ Five select items from the Schedule Attitudes Survey (SAS).
      • Dunham RB
      • Pierce JL.
      Attitudes toward work schedules: construct definition, instrument development, and validation.

      Primary outcome: Per protocol analysis

      During the first 3 months, 33% (n = 103) of all enrolled IAI participants viewed at least one education module. In total, n = 140 participants viewed one or more education modules during the first 3 months with 57% classified as Low module viewers, 11% as Moderate, and 32% as High. At 3-month follow-up, module viewing was associated with change in the mean PSQI (p = .02). For every 1 module viewed, PSQI-measured sleep quality improved by 0.12 points (p = .0048). The improvement was small and likely not clinically meaningful. Compared to no module viewing, participants viewing 8-10 modules experienced the greatest improvement in sleep quality (Overall p = .0024; Bonferonni-corrected p = .01). Participants in the High module-viewing group had a greater percentage change in PSQI from baseline compared to participants with no module viewing (Fig. 4; Overall p = .02; Bonferonni correct p = .06). The PSQI score decreased (sleep quality improved) by 0.12 points for every module viewed (Fig. 5a; p = .003). Thirty-seven individuals assigned to the WLC group were identified as viewing one or more modules immediately prior to completing the 3-month follow-up survey. Among these participants, the mean PSQI score worsened or did not change when compared to baseline.
      Fig 4
      Fig. 4Percentage and raw change in Pittsburgh Sleep Quality Index (PSQI) from baseline to 3-month follow-up by module viewing group. Notes: Low = refers to participants viewing 1 to 3 education modules. Moderate = refers to participants viewing 4 to 7 education modules. High = refers to participants viewing 8 to 10 education modules. PSQI, Pittsburgh Sleep Quality Index. Standard deviation (SD) represented by error bars. The directionality of bars showing the mean percentage change and mean raw change in PSQI for the None group differ based on [1] the change in PSQI raw scores from baseline to 3 months was 0 for 14% of all participants, with decreases ranging from -1 to -13 and increases ranging from 1 to 7; and [2] the percentage change in PSQI from baseline to 3 months showed decreases ranging from -5.9% to -100% and increases ranging from 6.3% to 200%. The high percentage change scores relative to baseline values resulted in the difference in directionality of point estimates (a point estimate of positive mean percentage change and negative mean raw change) for the no module-viewing group in this figure.
      Fig 5
      Fig. 5(a) Number of education modules viewed and change in Pittsburgh Sleep Quality Index (PSQI) scores. (b) Number of education modules viewed and change in Chalder Fatigue Questionnaire (CFQ) scores. Notes: Y-axis = change in scores from baseline to 3 months for the PSQI and CFQ. X-axis = number of education modules viewed by study participants. Solid line on graph = slope.

      Secondary outcomes: Per protocol analysis

      At 3-month follow-up, module viewing was associated with an improvement (reduction) in self-reported fatigue as measured by the CFQ. Fatigue decreased (improved) by 0.074 points for every module viewed (Fig. 5b; p = .04). The association between module-viewing status (Low, Moderate, and High) and fatigue was non-significant (Fig. 6; mean CFQ p = .08; percentage change in CFQ p = .16). Participants in the High module-viewing group experienced a greater mean change (reduction) in fatigue compared to participants with no module viewing, however, the association did not reach statistical significance (Overall p = .05; Bonferonni-corrected p = .15). Those in the High module-viewing group also experienced a greater percentage change in fatigue from baseline compared to participants with no module viewing, although this comparison was non-significant (Overall p = .06; Bonferonni-corrected p = .17; Fig. 6).
      Fig 6
      Fig. 6Percentage and raw change in Chalder Fatigue Questionnaire (CFQ) measured fatigue by module viewing group. Notes: Low = refers to participants viewing 1-3 education modules. Moderate = refers to participants viewing 4-7 education modules. High = refers to participants viewing 8-10 education modules. Standard deviation (SD) represented by error bars. The directionality of bars showing the mean percentage change and mean raw change in CFQ for the None and Moderate groups differ based on [1] the change in CFQ raw scores from baseline to 3 months was 0 for 25% of participants, with decreases in raw scores ranging from -1 to -7 and increases ranging from 1 to 6; and [2] the percentage change in CFQ from baseline to 3 months showed decreases ranging from -9% to -88% and increases ranging from 10% to 600%. The high percentage change scores relative to baseline values resulted in the difference in directionality of point estimates (a point estimate of positive mean percentage change and negative mean raw change) for the moderate module-viewing group in this figure.
      For all other secondary outcomes ESS, OFER, EMS-SAQ, and SAS, change in the raw score was used as the outcome of interest because of substantial outliers in percentage change. None of these measures differed in terms of mean change by module viewing status at 3 months (p > .05 all comparisons). In addition, we performed sub-group analyses with participants who reported poor sleep quality at baseline. Findings from the sub-group analyses are similar to findings for the main comparison (Table 3).

      Discussion

      At 3-month follow-up, intention-to-treat analyses showed no impact of the education intervention on sleep quality or fatigue. Findings from the per-protocol analyses revealed that improvements in sleep quality and reductions in fatigue were associated with viewing education modules. The greater the number of education modules viewed, the greater the improvements in both sleep quality and fatigue.
      The Fatigue Risk Management in Emergency Medical Services Education Program, developed specifically for this research study, is one of a few targeting public safety personnel,
      • Kuehl KS
      • Elliot DL
      • MacKinnon DP
      • et al.
      The SHIELD (Safety & Health Improvement: Enhancing Law Enforcement Departments) study: mixed methods longitudinal findings.
      and perhaps the first program specifically tailored to EMS clinician shift workers. Previous reviews of the evidence reveal wide variation in the structure and content of previously developed sleep health education programs.
      • Barger LK
      • Runyon MS
      • Renn ML
      • et al.
      Effect of fatigue training on safety, fatigue, and sleep in Emergency Medical Services personnel and other shift workers: a systematic review and meta-analysis.
      ,
      • Murawski B
      • Wade L
      • Plotnikoff RC
      • Lubans DR
      • Duncan MJ.
      A systematic review and meta-analysis of cognitive and behavioral interventions to improve sleep health in adults without sleep disorders.
      Findings from these reviews provide evidence that tailoring to unique groups is likely necessary to impact outcomes like sleep quality. Notable characteristics of the education program tested in this trial include: [1] the wide breadth of topics covered in ten different modules; [2] the presentation of evidence germane to each topic interspersed with video-recorded commentary and interviews with EMS workers and clinical experts in sleep medicine; [3] the brevity of each module (eg, approximately 13 minutes each); and [4] the availability of continuing education credits following program completion. Intention-to-treat analyses showed no differences between the intervention and wait-list control groups. However, per protocol analyses showed that the participants who viewed the most education modules experienced a benefit, including improved sleep quality and reduced levels of fatigue. Our findings may inform the decisions of local EMS agency directors, who in recent years have been directed to provide education and training directly to EMS clinicians on the topics of sleep health and fatigue as part of employment on-boarding and continuing education practice.
      • Patterson PD
      • Higgins JS
      • Van Dongen HPA
      • et al.
      Evidence-based guidelines for fatigue risk management in Emergency Medical Services.
      ,
      • Martin-Gill C
      • Higgins JS
      • Van Dongen HPA
      • et al.
      Proposed performance measures and strategies for implementation of the fatigue risk management guidelines for Emergency Medical Services.
      Relative to prior studies,
      • Patterson PD
      • Suffoletto BP
      • Kupas DF
      • Weaver MD
      • Hostler D.
      Sleep quality and fatigue among prehospital providers.
      ,
      • Patterson PD
      • Weaver MD
      • Frank RC
      • et al.
      Association between poor sleep, fatigue, and safety outcomes in Emergency Medical Services providers.
      ,
      • van der Ploeg E
      • Kleber RJ.
      Acute and chronic job stressors among ambulance personnel: predictors of health symptoms.
      a noticeably high proportion of participants (>80%) in this study reported being fatigued and/or having poor sleep quality at baseline. It is possible that the individuals who volunteered for this research study were, at the time of enrollment, more severely impacted by work-related fatigue than others who did not participate. It is also possible that work-related factors, such as patient volume, and other factors played a role in how participants reported fatigue at baseline. In other words, workload at the time of enrollment for many may have been high, and this was reflected in how individuals responded to sleep and fatigue questionnaires. Another potential explanation is that the COVID-19 pandemic played a unique role in the sleep patterns of public safety workers, which led to increased levels of fatigue. For many who enrolled, the COVID-19 pandemic altered day-to-day duties in public safety operations. Many EMS agencies instituted a mask (facial covering) mandate and use of other personal protective equipment to combat infection at the same time when our study was recruiting and enrolling. At the same time, call volumes (patient-related volume) fluctuated drastically in some parts of the U.S.
      • McVaney KE
      • Pepe PE
      • Maloney LM
      • et al.
      The relationship of large city out-of-hospital cardiac arrests and the prevalence of COVID-19.
      These factors and other work-related elements may have altered sleep behaviors and perceptions of fatigue, which manifested in a greater number of EMS clinicians, including those who participated in this study, reporting feelings of fatigue and poor sleep. Yet another potential explanation is that fatigue and poor sleep are so pervasive in EMS operations that the vast majority of personnel, regardless of the pandemic, are persistently fatigued (to some degree). Regardless of the reasons, which likely vary somewhat across location, fatigue and poor sleep are significant problems in EMS and public safety in general, and solutions that are tailored to EMS work conditions are needed.
      While many benefited from the education program, others did not engage with the intervention or did not adhere to protocol in other ways. We describe several potential explanations for these findings. First, we believe that the COVID-19 pandemic, which began March 11, 2020, fundamentally changed the behavior of many, including the individuals that signed up for this trial. Recruitment and enrollment for our study began February 2020 and was paused from March 2020 until June 2020. During this time, the total volume of EMS dispatches (workload) experienced by many EMS organizations changed.
      • Lerner EB
      • Newgard CD
      • Mann NC.
      Effect of the coronavirus disease 2019 (COVID-19) pandemic on the U.S. Emergency Medical Services system: a preliminary report.
      ,
      • Handberry M
      • Bull-Otterson L
      • Dai M
      • et al.
      Changes in Emergency Medical Services before and during the COVID-19 pandemic in the United States, January 2018-December 2020.
      In some locations volume decreased, in others it increased, and in some locations there were changes in the frequency of types of dispatches.
      • Lerner EB
      • Newgard CD
      • Mann NC.
      Effect of the coronavirus disease 2019 (COVID-19) pandemic on the U.S. Emergency Medical Services system: a preliminary report.
      ,
      • Handberry M
      • Bull-Otterson L
      • Dai M
      • et al.
      Changes in Emergency Medical Services before and during the COVID-19 pandemic in the United States, January 2018-December 2020.
      In addition, the sleeping habits and behaviors of most working aged adults were altered so that many (37% in some locations) obtained more sleep compared to pre-pandemic levels, whereas 17% (in some locations) reported less sleep compared to pre-pandemic levels.

      Batool-Anwar S, Robbins R, Ali SH, et al. Examining changes in sleep duration associated with the onset of the COVID-19 pandemic: who is sleeping and who is not? medRxiv; 2021. Available at: https://www.medrxiv.org/content/10.1101/2021.04.06.21254996v1.article-info Accessed July 14, 2021.

      Other studies confirm changes in sleep habits during early periods of the pandemic,
      • Leone MJ
      • Sigman M
      • Golombek DA.
      Effects of lockdown on human sleep and chronotype during the COVID-19 pandemic.
      which happened to overlap with the recruitment and data collection phases of this trial.
      Second, we believe a lack of participant blinding may have played a role. Our study protocol used an open-label, wait-list control study design, which is a commonly used study design in sleep related research focused on changing behavior.
      • Murawski B
      • Wade L
      • Plotnikoff RC
      • Lubans DR
      • Duncan MJ.
      A systematic review and meta-analysis of cognitive and behavioral interventions to improve sleep health in adults without sleep disorders.
      With this design, it was not feasible to blind participants or most of the study team to the randomization assignment of agencies and individuals within agencies post-randomization. Previous research has suggested that when participants become aware of their status in an experimental study, many may alter their behavior or responses to study related questions.
      • Adamson SJ
      • Bland JM
      • Hay EM
      • et al.
      Patients' preferences within randomised trials: systematic review and patient level meta-analysis.
      • McCambridge J
      • Kypri K
      • Elbourne D.
      In randomization we trust? There are overlooked problems in experimenting with people in behavioral intervention trials.
      • Silverman WA
      • Altman DG.
      Patients' preferences and randomised trials.
      While we did not receive any complaints from agency leaders or individuals upon notification of their randomization assignment, findings from previous research would suggest that some participants in the wait-list control condition may have been displeased with their assignment and changed their behavior proximal to select points of follow-up.
      Opportunities for improving the delivery and potential impact of this intervention include: [1] Integrating the education modules with new employee on-boarding and/or with annual or once every 2-year requirements for EMS continuing education. Most all EMS clinicians are required to complete a minimum set of continuing education hours annually in order to maintain state and/or national level certification/licensure. At present, there is no formal requirement for education and/or training focused on employee sleep health and fatigue risk management. Adding this type of education and training is feasible and may have a significant impact. [2] Awarding individuals credits for completing education and training that individuals can count towards annual requirements is a natural complement to adding sleep health and fatigue training to an existing EMS continuing education system. Education and training minus “credits” may have limited impact on education module engagement. [3] Frequent updates of the education materials based on the most recent and best available sleep health-focused research may elevate the novelty of the content and lead to increased engagement. [4] Finally, the delivery of education may need to evolve. For purposes of this research study, we deployed the modules on a password protected website. The website was compatible with mobile smartphone technology; thus participants could view the modules on their smartphones. However, website aesthetics or other aspects of our technology may have been limited in the eyes of many participants. As technology evolves, newer methods of dissemination may prove more engaging than what was used for in this research study.
      Despite several challenges experienced during the execution of this trial, participant feedback suggests that the structure and format of the educational intervention was acceptable and informative. None who provided comment following study participation expressly identified a module or the contents of a module as not relevant or not informative, though the feedback provided was isolated to only those individuals willing to provide it. Although feedback was positive, the educational program could be improved. First, module production was limited to a select type of technology and software readily available to the study team. The production quality, animation, and other aesthetically related elements may be improved and consequently attract more viewers in the future. Second, the evidence germane to all topics should be updated. Future reviews of the published evidence on sleep health and fatigue mitigation will likely identify novel strategies supported by the evidence that would further inform the targeted population. Finally, the content of each module was designed to inform front-line clinicians on behaviors that may lead to improved sleep quality and fatigue. Given that fatigue risk management in the workplace is a shared responsibility between the employee and employer,
      • Lerman SE
      • Eskin E
      • Flower DJ
      • et al.
      Fatigue risk management in the workplace.
      there is reason to believe that a companion set of education modules targeting the employer, and focused on employer related decisions, may be a worthwhile addition to the existing program.

      Limitations

      Our study is limited to those EMS agencies and individuals within these agencies that agreed to participate. While our sample may be biased to those who elected to enroll and participate, the demographic characteristics of those involved are similar to the characteristics of agencies and individual EMS clinicians involved in previous observational and experimental research.
      • Patterson PD
      • Buysse DJ
      • Weaver MD
      • et al.
      Real-time fatigue reduction in emergency care clinicians: the SleepTrackTXT randomized trial.
      ,
      • Patterson PD
      • Huang DT
      • Fairbanks RJ
      • Simeone S
      • Weaver MD
      • Wang HE.
      Variation in Emergency Medical Services workplace safety culture.
      Recruitment, enrollment, and attrition may have been impacted by the COVID-19 pandemic, which impacted sleep patterns when compared to pre-pandemic periods.

      Batool-Anwar S, Robbins R, Ali SH, et al. Examining changes in sleep duration associated with the onset of the COVID-19 pandemic: who is sleeping and who is not? medRxiv; 2021. Available at: https://www.medrxiv.org/content/10.1101/2021.04.06.21254996v1.article-info Accessed July 14, 2021.

      ,
      • Leone MJ
      • Sigman M
      • Golombek DA.
      Effects of lockdown on human sleep and chronotype during the COVID-19 pandemic.
      ,
      • Bann D
      • Villadsen A
      • Maddock J
      • et al.
      Changes in the behavioural determinants of health during the COVID-19 pandemic: gender, socioeconomic and ethnic inequalities in five British cohort studies.
      ,
      • Salfi F
      • Lauriola M
      • D'Atri A
      • et al.
      Demographic, psychological, chronobiological, and work-related predictors of sleep disturbances during the COVID-19 lockdown in Italy.
      Our protocol was open-label, which may have impacted behavior or responses to study related assessments post-randomization.
      • Adamson SJ
      • Bland JM
      • Hay EM
      • et al.
      Patients' preferences within randomised trials: systematic review and patient level meta-analysis.
      • McCambridge J
      • Kypri K
      • Elbourne D.
      In randomization we trust? There are overlooked problems in experimenting with people in behavioral intervention trials.
      • Silverman WA
      • Altman DG.
      Patients' preferences and randomised trials.
      Ten percent of individuals assigned to the wait-list control group (n = 37) viewed components of the intervention immediately prior to completing the 3-month survey. These individuals gained access to the materials early due to an error in the design of our online system that controlled access to surveys and education materials. Their responses to survey items and participation in the study may have been impacted. Similar to previous research,
      • Dodd S
      • White IR
      • Williamson P.
      Nonadherence to treatment protocol in published randomised controlled trials: a review.
      many with access to the intervention failed to comply with protocol, which may have impacted our findings. Low adherence to protocol may have resulted in an attenuated effect of the intervention.

      Conclusions

      Intention-to-treat analyses revealed no differences in measures of sleep quality or fatigue at 3 months or 6 months follow-up. Per protocol analyses showed that the greater the number of education modules viewed, the greater the improvement in sleep quality and the greater the reduction in fatigue at 3-month follow-up. These findings suggest that the program is beneficial when the target population engages with the education. These findings may be useful to EMS administrators when designing fatigue risk management programs.

      Declaration of conflict of interest

      Authors report no conflicts of interest.

      Funding

      Work performed on this manuscript was supported with funding from the U.S. Department of Transportation, National Highway Traffic Safety Administration to the National Association of State EMS Officials, sub-contracted to the University of Pittsburgh. Contract number: DTNH2215R00029. The views contained in this article are those of the authors and not necessarily those of the National Highway Traffic Safety Administration.

      Author contribution

      Authors PDP, MDW, CGP, and KR conceived of the study idea and aims. Authors PDP, SEM, BNB, WH, MDW, TSO, SNS, and CGP collected and analyzed study related data. All authors participated in the synthesis and interpretation of study findings and writing of the manuscript.

      Acknowledgments

      Findings from this study and described in this manuscript appear in a report submitted to the National Highway Traffic Safety Administration on September 2, 2021.

      Appendix. Supplementary materials

      References

        • Patterson PD
        • Suffoletto BP
        • Kupas DF
        • Weaver MD
        • Hostler D.
        Sleep quality and fatigue among prehospital providers.
        Prehosp Emerg Care. 2010; 14: 187-193
        • Patterson PD
        • Weaver MD
        • Frank RC
        • et al.
        Association between poor sleep, fatigue, and safety outcomes in Emergency Medical Services providers.
        Prehosp Emerg Care. 2012; 16: 86-97
        • Patterson PD
        • Weaver MD
        • Hostler D
        EMS provider wellness.
        in: Cone DC Brice JH Delbridge TR Myers JB Emergency Medical Services: Clinical Practice and Systems Oversight. John Wiley & Sons, Inc., 2015: 211-216 (chap 20)
        • Flaa TA
        • Bjorvatn B
        • Pallesen S
        • et al.
        Subjective and objective sleep among air ambulance personnel.
        Chronobiol Int. 2021; 38: 129-139
        • Pyper Z
        • Paterson JL.
        Fatigue and mental health in Australian rural and regional ambulance personnel.
        Emerg Med Australas. 2016; 28: 62-66
        • VanRooyen MJ
        • Thomas TL
        • Clem KJ.
        International Emergency Medical Services: assessment of developing prehospital systems abroad.
        J Emerg Med. 1999; 17: 691-696
        • Patterson PD
        • Runyon MS
        • Higgins JS
        • et al.
        Shorter versus longer shift duration to mitigate fatigue and fatigue related risks in Emergency Medical Services: a systematic review.
        Prehosp Emerg Care. 2018; 22: 28-36
        • Patterson PD
        • Buysse DJ
        • Weaver MD
        • Callaway CW
        • Yealy DM.
        Recovery between work shifts among Emergency Medical Services clinicians.
        Prehosp Emerg Care. 2015; 19: 365-375
        • Patterson PD
        • Higgins JS
        • Van Dongen HPA
        • et al.
        Evidence-based guidelines for fatigue risk management in Emergency Medical Services.
        Prehosp Emerg Care. 2018; 22: 89-101
        • Crowe RP
        • Fernandez AR
        • Pepe PE
        • et al.
        The association of job demands and resources with burnout among Emergency Medical Services professionals.
        J Am Coll Emerg Physicians Open. 2020; 1: 6-16
        • Reardon M
        • Abrahams R
        • Thyer L
        • Simpson P.
        Review article: Prevalence of burnout in paramedics: a systematic review of prevalence studies.
        Emerg Med Australas. 2020; 32: 182-189
      1. NASEMSO. 2020 National Emergency Medical Services Assessment. National Association of State EMS Officials (NASEMSO). Available at: https://nasemso.org/wp-content/uploads/2020-National-EMS-Assessment_Reduced-File-Size.pdf Accessed July 14, 2021.

      2. Cairns C, Kang K, Santo L. National Hospital Ambulatory Medical Care Survey: 2018 emergency department summary tables. U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics. Available at: https://www.cdc.gov/nchs/data/nhamcs/web_tables/2018-ed-web-tables-508.pdf Accessed July 14, 2021.

        • Barger LK
        • Runyon MS
        • Renn ML
        • et al.
        Effect of fatigue training on safety, fatigue, and sleep in Emergency Medical Services personnel and other shift workers: a systematic review and meta-analysis.
        Prehosp Emerg Care. 2018; 22: 58-68
        • Murawski B
        • Wade L
        • Plotnikoff RC
        • Lubans DR
        • Duncan MJ.
        A systematic review and meta-analysis of cognitive and behavioral interventions to improve sleep health in adults without sleep disorders.
        Sleep Med Rev. 2018; 40: 160-169
        • Campbell MK
        • Piaggio G
        • Elbourne DR
        • Altman DG.
        Consort 2010 statement: extension to cluster randomised trials.
        BMJ. 2012; 345: e5661
        • Lerman SE
        • Eskin E
        • Flower DJ
        • et al.
        Fatigue risk management in the workplace.
        J Occup Environ Med. 2012; 54: 231-258
        • Patterson PD
        • Moore CG
        • Weaver MD
        • et al.
        Mobile phone text messaging intervention to improve alertness and reduce sleepiness and fatigue during shiftwork among emergency medicine clinicians: study protocol for the SleepTrackTXT pilot randomized controlled trial.
        Trials. 2014; 15: 244
        • Patterson PD
        • Buysse DJ
        • Weaver MD
        • et al.
        Real-time fatigue reduction in emergency care clinicians: the SleepTrackTXT randomized trial.
        Am J Ind Med. 2015; 58: 1098-1113
        • Patterson PD
        • Moore CG
        • Guyette FX
        • et al.
        Fatigue mitigation with SleepTrackTXT2 in air medical emergency care systems: study protocol for a randomized controlled trial.
        Trials. 2017; 18: 254
        • Patterson PD
        • Moore CG
        • Guyette FX
        • et al.
        Real-time fatigue mitigation with air-medical personnel: The SleepTrackTXT2 randomized trial.
        Prehosp Emerg Care. 2019; 23: 465-478
        • Buysse DJ
        • Reynolds CF
        • Monk TH
        • Berman SR
        • Kupfer DJ.
        The Pittsburgh Sleep Quality Index: a new instrument for psychiatric practice and research.
        Psychiatr Serv. 1989; 28: 193-213
        • Buysse DJ
        • Germain A
        • Moul DE
        • et al.
        Efficacy of brief behavioral treatment for chronic insomnia in older adults.
        Arch Intern Med. 2011; 171: 887-895
        • Chalder T
        • Berelowitz G
        • Pawlikowska T
        • et al.
        Development of a fatigue scale.
        J Psychosom Res. 1993; 37: 147-153
        • Johns MW.
        A new method for measuring daytime sleepiness: the Epworth sleepiness scale.
        Sleep. 1991; 14: 540-545
        • Patterson PD
        • Huang DT
        • Fairbanks RJ
        • Wang HE.
        The Emergency Medical Services safety attitudes questionnaire.
        Am J Med Qual. 2010; 25: 109-115
        • Dunham RB
        • Pierce JL.
        Attitudes toward work schedules: construct definition, instrument development, and validation.
        Acad Manag J. 1986; 29: 170-182
        • Winwood PC
        • Winefield AH
        • Dawson D
        • Lushington K.
        Development and validation of a scale to measure work-related fatigue and recovery: the Occupational Fatigue Exhaustion/Recovery Scale (OFER).
        J Occup Environ Med. 2005; 47: 594-606
        • Patterson PD
        • Weaver MD
        • Fabio A
        • et al.
        Reliability and validity of survey instruments to measure work-related fatigue in the Emergency Medical Services setting: a systematic review.
        Prehosp Emerg Care. 2018; 22: 17-27
        • Saghaei M
        • Saghaei S.
        Implementation of an open-source customizable minimization program for allocation of patients to parallel groups in clinical trials.
        J Biomed Sci Eng. 2011; 4: 734-739
        • Tripepi G
        • Chesnaye NC
        • Dekker FW
        • Zoccali C
        • Jager KJ.
        Intention to treat and per protocol analysis in clinical trials.
        Nephrology. 2020; 25: 513-517
        • Hernan MA
        • Hernandez-Diaz S.
        Beyond the intention-to-treat in comparative effectiveness research.
        Clin Trials. 2012; 9: 48-55
        • Sullivan GM
        • Feinn R.
        Using effect size-or why the p value is not enough.
        J Grad Med Educ. 2012; 4: 279-282
        • Kuehl KS
        • Elliot DL
        • MacKinnon DP
        • et al.
        The SHIELD (Safety & Health Improvement: Enhancing Law Enforcement Departments) study: mixed methods longitudinal findings.
        J Occup Environ Med. 2016; 58: 492-498
        • Martin-Gill C
        • Higgins JS
        • Van Dongen HPA
        • et al.
        Proposed performance measures and strategies for implementation of the fatigue risk management guidelines for Emergency Medical Services.
        Prehosp Emerg Care. 2018; 22: 102-109
        • van der Ploeg E
        • Kleber RJ.
        Acute and chronic job stressors among ambulance personnel: predictors of health symptoms.
        Occup Environ Med. 2003; 60: i40-i46
        • McVaney KE
        • Pepe PE
        • Maloney LM
        • et al.
        The relationship of large city out-of-hospital cardiac arrests and the prevalence of COVID-19.
        EClinicalMedicine. 2021; 34100815
        • Lerner EB
        • Newgard CD
        • Mann NC.
        Effect of the coronavirus disease 2019 (COVID-19) pandemic on the U.S. Emergency Medical Services system: a preliminary report.
        Acad Emerg Med. 2020; 27: 693-699
        • Handberry M
        • Bull-Otterson L
        • Dai M
        • et al.
        Changes in Emergency Medical Services before and during the COVID-19 pandemic in the United States, January 2018-December 2020.
        Clin Infect Dis. 2021; 73: S84-S91
      3. Batool-Anwar S, Robbins R, Ali SH, et al. Examining changes in sleep duration associated with the onset of the COVID-19 pandemic: who is sleeping and who is not? medRxiv; 2021. Available at: https://www.medrxiv.org/content/10.1101/2021.04.06.21254996v1.article-info Accessed July 14, 2021.

        • Leone MJ
        • Sigman M
        • Golombek DA.
        Effects of lockdown on human sleep and chronotype during the COVID-19 pandemic.
        Curr Biol. 2020; 30: R930-R931
        • Adamson SJ
        • Bland JM
        • Hay EM
        • et al.
        Patients' preferences within randomised trials: systematic review and patient level meta-analysis.
        BMJ. 2008; 337: a1864
        • McCambridge J
        • Kypri K
        • Elbourne D.
        In randomization we trust? There are overlooked problems in experimenting with people in behavioral intervention trials.
        J Clin Epidemiol. 2014; 67: 247-253
        • Silverman WA
        • Altman DG.
        Patients' preferences and randomised trials.
        Lancet. 1996; 347: 171-174
        • Patterson PD
        • Huang DT
        • Fairbanks RJ
        • Simeone S
        • Weaver MD
        • Wang HE.
        Variation in Emergency Medical Services workplace safety culture.
        Prehosp Emerg Care. 2010; 14: 448-460
        • Bann D
        • Villadsen A
        • Maddock J
        • et al.
        Changes in the behavioural determinants of health during the COVID-19 pandemic: gender, socioeconomic and ethnic inequalities in five British cohort studies.
        J Epidemiol Community Health. 2021; 75: 1136-1142
        • Salfi F
        • Lauriola M
        • D'Atri A
        • et al.
        Demographic, psychological, chronobiological, and work-related predictors of sleep disturbances during the COVID-19 lockdown in Italy.
        Sci Rep. 2021; 11: 11416
        • Dodd S
        • White IR
        • Williamson P.
        Nonadherence to treatment protocol in published randomised controlled trials: a review.
        Trials. 2012; 13: 84