If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
Corresponding author: P. Daniel Patterson, PhD, NRP, Department of Emergency Medicine, University of Pittsburgh, Iroquois Bldg., 3600 Forbes Ave., Suite 400A, Pittsburgh, PA 15261, USA.
University of Pittsburgh, School of Medicine, Department of Emergency Medicine, Pittsburgh, Pennsylvania, USAUniversity of Pittsburgh, School of Health and Rehabilitation Sciences, Division of Community Health Services, Emergency Medicine Program, Pittsburgh, Pennsylvania, USA
Brigham and Women's Hospital, Division of Sleep and Circadian Disorders, Boston, Massachusetts, USAHarvard Medical School, Division of Sleep Medicine, Boston, Massachusetts, USA
Greater than half of emergency medical services (EMS) clinician shift workers report poor sleep, fatigue, and inadequate recovery between shifts. We hypothesized that EMS clinicians randomized to receive tailored sleep health education would have improved sleep quality and less fatigue compared to wait-list controls after 3 months.
Methods
We used a cluster-randomized, 2-arm, wait-list control study design (clinicaltrials.gov identifier: NCT04218279). Recruitment of EMS agencies (clusters) was nationwide. Our study was powered at 88% to detect a 0.4 standard deviation difference in sleep quality with 20 agencies per arm and a minimum of 10 individuals per agency. The primary outcome was measured using the Pittsburgh Sleep Quality Index (PSQI) at 3-month follow-up. Our intervention was accessible in an online, asynchronous format and comprised of 10 brief education modules that address fatigue mitigation topics prescribed by the American College of Occupational Environmental Medicine.
Results
In total, 36 EMS agencies and 678 individuals enrolled. Attrition at 3 months did not differ by study group (Intervention = 17.4% vs. Wait-list control = 18.2%; p = .37). Intention-to-treat analyses detected no differences in PSQI and fatigue scores at 3 months. Per protocol analyses showed the greater the number of education modules viewed, the greater the improvement in sleep quality and the greater the reduction in fatigue (p < .05).
Conclusions
While intention-to-treat analyses revealed no differences in sleep quality or fatigue at 3 months, per protocol findings identified select groups of EMS clinician shift workers who may benefit from sleep health education. Our findings may inform fatigue risk management programs.
Severe mental and physical fatigue in shift work occupations is widespread, and according to previous research, workplace fatigue impacts greater than half of U.S.-based Emergency Medical Services (EMS) clinician first responders.
however, direct comparisons between nations may be difficult given differences in EMS organizational structure and deployment, scheduling, and licensure.
In the U.S., long duration shifts, variable shift scheduling, limited time off between shifts, and other factors are linked to fatigue and poor sleep amongst EMS shift workers.
Recent guidance for addressing fatigue in the EMS workplace recommends educating and training EMS clinicians on the importance of sleep and dangers of fatigue.
Unfortunately, there are few known resources for meeting the sleep health education needs of EMS shift workers in the U.S. and other nations.
The type of work and work environment of EMS clinicians is unique, regardless of their nationality. Their volume of work is unpredictable, involves high-stress situations, long duration shifts (oftentimes 24 hours),
and high-risk medical procedures that are time-sensitive and play an important role in the life or death of patients. These unique work conditions have been linked to reports of work related stress, burnout, and fatigue.
This may have a pervasive impact on the safety and quality of care delivered in emergencies. A recent report shows that the U.S. EMS system is comprised of more than 20,000 agencies and nearly 1 million personnel.
They play a dual role of emergency healthcare provider and public safety responder. The EMS clinician is often the first point of medical contact with the health care system when the chief medical complaint is an acute illness or injury. Approximately 16% of all U.S.-based hospital Emergency Department (ED) volume involves EMS transport.
Cairns C, Kang K, Santo L. National Hospital Ambulatory Medical Care Survey: 2018 emergency department summary tables. U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics. Available at: https://www.cdc.gov/nchs/data/nhamcs/web_tables/2018-ed-web-tables-508.pdf Accessed July 14, 2021.
The recent development of evidence-based guidelines for fatigue mitigation in the EMS setting revealed an absence of resources, programs, and interventions tailored to the EMS worker.
Effect of fatigue training on safety, fatigue, and sleep in Emergency Medical Services personnel and other shift workers: a systematic review and meta-analysis.
Sleep health education and training tailored to the unique occupational characteristics of EMS shift work may have a more positive impact than would programs targeting the general adult population or other occupational groups.
An effort to fill this void began in 2013 when the National EMS Advisory Council, a council formed in 2007 by the U.S. Department of Transportation, recommended that federal partners examine the best available evidence and distribute resources for fatigue mitigation to EMS services nationwide. In 2015, the National Association of State EMS Officials, the University of Pittsburgh Department of Emergency Medicine, and the U.S. National Highway Traffic Safety Administration partnered on the Fatigue in EMS Project. Comprised of 3 phases, this project began with Phase 1, a combination of 7 systematic reviews of the evidence germane to fatigue mitigation and resulted in the 2018 Evidence-Based Guideline for Fatigue Mitigation in EMS.
The focus of this paper is reporting on Phase 2, an experimental study. The aim of Phase 2 was to determine if providing education and training on the importance of sleep health and dangers of fatigue impacts key indicators of sleep quality and fatigue. We hypothesized that education and training focused on sleep health and fatigue, delivered in an asynchronous manner and tailored to EMS shift workers, would result in improvements in sleep quality and a reduction in self-reported fatigue after a 3-month study interval.
Methods
Overview of experimental study design
Our hypothesis was tested using a cluster-randomized, 2-arm randomized trial with a wait-list control group. All screened and eligible EMS agencies (clusters) were randomized to either the intervention group or a wait-list control group. The total study duration was 6 months to allow for crossover of wait-list participants. At 3 months of participation, the wait-list control group EMS agencies and participants were offered the intervention. Experimental designs with a wait-list comparison group are commonly used when testing the impact of sleep health related education programs.
The study protocol was approved by the University of Pittsburgh Institutional Review Board, the Office of Management and Budget [Control Number: 2127-0742; ICR Reference Number: 201811-2127-003], and registered on ClinicalTrials.gov (NLM Identifier: NCT04218279). All study subjects provided informed consent. We report our findings in accordance with the CONSORT 2010 statement, extension to cluster randomized trials.
We recruited EMS agencies (clusters) nationally and randomized each cluster to 1 of 2 possible study arms: [1] the immediate access to intervention (IAI) materials group; or the [2] wait list comparison (WLC) group. Within each cluster, we enrolled individual EMS clinician shift workers over a 30-day open enrollment period with a per-cluster goal enrollment of n = 30 and maximum enrollment of n = 50. The total duration of participation for individuals was 6 months. The WLC group was offered the intervention at 3 months. Our primary analyses compare outcomes at the 3-month timepoint.
Setting
Our population of interest was EMS agencies located in the United States (inclusive of Alaska and Hawaii) and their frontline EMS clinician shift workers.
Participant eligibility
An EMS agency was eligible if it: [1] provided 911 response and/or transport in the United States; [2] provided ground-based EMS services 24-hours-a-day (agencies limited to air-medical services only were not eligible); [3] employed n = 50 or more paid staff (agencies that used all-volunteer staffing were not eligible); and [4] did not restrict use of personal mobile phones/smartphones during shift work. The eligibility criteria for an individual EMS clinician was: [1] the clinician must be 18 years of age or older; [2] was currently working as an EMS clinician, not in an administrative only role; [3] worked a minimum of 1 shift per week; [4] worked and resided in the United States; [5] worked at an EMS agency that agreed to participate in this study; [6] had a cellular, mobile/smartphone that was capable of sending and receiving text messages; and [7] agreeable to answering online surveys and text message queries intermittently over a 6 month study period.
Recruitment
We recruited EMS agencies by distributing a one-page flyer to members of professional EMS organizations (eg, the National EMS Management Association, and others). We further shared the study flyer on social media and advertised the study on popular EMS news websites. Recruitment of individual EMS clinicians within EMS agencies (clusters) involved agency leaders circulating study-related information directly to potentially eligible EMS clinicians. Participants were offered remuneration totaling $35 dollars over the study period.
Intervention
Our intervention was developed specifically for this trial based on findings from a recent evidence review that showed [1] education and training in sleep health and fatigue has a positive impact on sleep quality among shift workers; and [2] there are few programs tailored to this population.
Effect of fatigue training on safety, fatigue, and sleep in Emergency Medical Services personnel and other shift workers: a systematic review and meta-analysis.
We designed the Fatigue Risk Management in Emergency Medical Services Education Program based on guidance from the American College of Occupational and Environmental Medicine (ACOEM) Task Force on Fatigue Risk Management.
We used the ACOEM recommendations as a template for designing an education program specific to the EMS occupation. Our program was comprised of 10 education modules targeting multiple topics as outlined by the ACOEM. The 10 topics included: [1] hazards of fatigue; [2] sleep physiology; [3] sleep health; [4] work related stress; [5] sleep disorders; [6] fatigue recognition; [7] adequate sleep; [8] diet and exercise; [9] alertness strategies; and [10] managing fatigue. The content of each module is comprised of: [1] a summary presentation of published evidence; [2] quiz questions about the evidence and information presented; and [3] video recordings of interviews with experts in sleep medicine, EMS administrators, and frontline EMS clinicians. We limited the duration of each module to approximately 10-15 minutes. Administrators of EMS organizations, individual EMS clinicians, and other stakeholders in the EMS work environment provided feedback, which we used to edit intervention content for clarity prior to dissemination. The Commission on Accreditation for Prehospital Continuing Education reviewed each module and approved 2.25 hours of continuing education for any individual who viewed the modules, answered a standard assessment and quiz, and applied for education credits.
Data collection procedures
Participants answered survey questions at baseline, 3 months, and 6 months. Participants also answered momentary assessment text message queries, which involved the previously tested, interactive SleepTrackTXT platform.
Mobile phone text messaging intervention to improve alertness and reduce sleepiness and fatigue during shiftwork among emergency medicine clinicians: study protocol for the SleepTrackTXT pilot randomized controlled trial.
Momentary assessments occurred during shifts (intra-shift) and in between shifts (inter-shift) 7 consecutive days, followed by 2 weeks with no text message-based queries. This pattern was repeated for the duration of the study. The platform was used for queries only. We did not use the SleepTrackTXT platform for purposes of delivering intervention materials or messages. The findings from the text message queries will be reported separately.
Outcomes
Our primary outcome of interest was sleep quality measured by the Pittsburgh Sleep Quality Index (PSQI).
The PSQI is a widely used 21-item survey with the total score ranging from 0-21. Respondents are asked to reflect on their sleep habits over the previous 30 days. A score greater than 5 was used to classify respondents as having poor sleep quality,
Survey assessments were administered online at baseline, at 3 months, and again at 6 months with a study-specific website that required participant specific login credentials. We chose 3-month follow-up as our primary time period for comparing intervention impact based on previous research,
Effect of fatigue training on safety, fatigue, and sleep in Emergency Medical Services personnel and other shift workers: a systematic review and meta-analysis.
which tested similar interventions over a similar time period in diverse occupational settings. The survey tools selected are reliable and/or valid, and previously tested in EMS clinician shift workers.
We used preliminary data as a guide and originally powered this study at 80% to detect a 0.4 standard deviation difference in PSQI score with n = 10 EMS agencies in each group (intervention and wait-list control), n = 30 individual EMS clinicians participants per agency, and intracluster correlation (ICC) = 0.07 (2 sided α = 0.05).
At the study's mid-point, we determined that the number of EMS clinicians per agency was likely not attainable and re-evaluated the power with an increase in the number of EMS agencies and fewer participants per agency. With 20 EMS agencies in each group and 10 EMS clinicians per agency we retained 88% power to detect a 0.4 effect size in PSQI score. Assuming that the prevalence of fatigue in the WLC group was 50%, we had 90% power to detect an absolute 20% reduction in CFQ-measured fatigue. A power calculation was not performed for secondary measures ESS, EMS-SAQ, SAS, or OFER.
Randomization and allocation concealment and implementation
We randomized EMS agencies (clusters) with a biased coin minimization procedure.
This procedure was intended to balance the randomization by the type and size of each cluster adaptively with each additional enrollment. Type was defined as [1] fire-based model, [2] hospital-based / third-service model, and [3] air-based / other type of EMS agency model. There is no publicly available national database that characterizes the number of employees at the EMS agency level. We therefore used our collective knowledge and experience in EMS to define agency size as [1] EMS agencies with ≥300 employees vs. [2] EMS agencies with <300 employees. The study team responsible for enrollment and intervention implementation were not informed of the probabilities of assignment. Our senior statistician managed treatment allocation until each EMS agency (cluster) was judged eligible. Notification of treatment allocation was sent via email directly to the study team from the senior statistician. The EMS agency primary point of contact (ie, the EMS agency administrator) was then informed of allocation assignment. Notification of intervention (the IAI group) assignment was followed with immediate access to the intervention. Notification of assignment to the wait-list group (the WLC group) was followed with a release date and time when access to intervention materials was granted.
Blinding
Given the study design, participant awareness of randomization status post assignment could not be concealed. While our principal investigator and study staff were also aware of cluster and individual participant randomization status, our statistician was removed from allocation procedures and activities related to data collection of primary and secondary outcomes.
Statistical methods
We followed intent-to-treat principles in all primary analyses.
We initially report descriptive statistics without adjustment for agency-level clustering. We used t-tests, tests of medians, chi-square tests, and Fisher's exact tests to examine baseline differences in the characteristics of EMS agencies (clusters) and individual participants stratified by IAI and WLC group status. We used hierarchical mixed effects models to test the impact of the intervention on the changes in the primary and secondary outcomes. We accounted for clustering at the agency level (nesting participants within agencies) with random intercepts. Formal tests and visual inspections indicated the models satisfied all necessary assumptions (eg, normality, homoscedasticity, linearity). We defined exposure to an education module as any viewing of a module. Module viewing was assessed as a continuous measure and categorically. For purposes of categorical assessment, we stratified module viewing into 4 categories based on participant viewing patterns evaluated at the end of the study period: [1] No modules viewed; [2] Low—with 1-3 modules viewed; [3] Moderate—with 4-7 modules viewed; and [4] High—with 8-10 modules viewed. In addition, we performed per-protocol analyses to determine if variation in intervention exposure was associated with outcomes.
For these analyses, we used hierarchical mixed effects models that accounted for clustering at the agency level and the dependence between repeated measures at the participant level. We used these models to characterize the relationship between exposure and the outcomes of interest, specifically among participants who viewed the education modules and completed the survey outcome assessments up to 3 and/or 6 months, separately. We examined the relative change (percentage change relative to baseline) in outcome measures stratified by module viewing status and by study group. In addition, we used the Cohen's d effect size measure and corresponding 95% confidence intervals to assess the difference in mean change of the PSQI and CFQ scores by group from baseline to 3-month follow-up.
We performed all analyses with the SAS statistical software version 9.4 (Cary, NC).
Results
In total, we screened 54 EMS agencies and confirmed 48 as eligible. Of these 48, we enrolled 36 EMS agencies (Fig. 1). Sixteen agencies were randomized to the IAI group and 20 to the WLC group (Table 1). Most agencies were moderately sized with 51-199 total employees (58.4%) and responded to 20,000 or more dispatches in 2019 (61.1%). Approximately two-thirds reported no formal fatigue management program (66.7%). Among the 36 EMS agencies enrolled, 678 individual EMS clinicians consented to participate (mean enrollments per agency 18.8, SD 14.3; min = 2, max = 51). Participant mean age was 37.4 years (SD 10.1), the mean years of experience was 12.6 years (SD 8.7), and the majority worked 12-hour or 24-hour shifts (89.4%). The demographic characteristics of individual enrollees in this study are similar to the characteristics documented in previous observational and experimental research studies involving EMS shift workers (Table 2).
Fig. 1CONSORT flow diagram. Notes: Shading signifies access to intervention materials. Some analyses and sample sizes reported in the document will involve the Immediate Access to Intervention (IAI) group and the Wait List Control (WLC) group sample sizes that differ slightly from the totals for 3-month and 6-month survey follow-up reported in this figure. These small differences are due to the fact that some participants did not complete the 3-month or 6-month follow-up, but were still considered “active participants” given their responses to other study related assessments (ie, text message queries), and given that some participants partially (or fully) completed the baseline survey, 3-month survey, and/or the 6- month survey. *n = 37 individuals assigned to the WLC group were identified as viewing one or more modules immediately prior to completing the 3-month follow-up survey.
Fisher exact test was used in categorical variables with over 20% expected cell counts less than 5. The differences in continuous variables were tested using the Mann-Whitney U test.
Fisher exact test was used in categorical variables with over 20% expected cell counts less than 5. The differences in continuous variables were tested using the Mann-Whitney U test.
Fisher exact test was used in categorical variables with over 20% expected cell counts less than 5. The differences in continuous variables were tested using the Mann-Whitney U test.
Fisher exact test was used in categorical variables with over 20% expected cell counts less than 5. The differences in continuous variables were tested using the Mann-Whitney U test.
IAI, Immediate access to intervention group; WLC, wait list control group; EMS, emergency medical services; ED, emergency department; EMT, emergency medical technician; ICU, intensive care unit; SD, standard deviation.
n = 678 individuals enrolled; however, n = 14 individuals randomized to the IAI group and n = 14 randomized to the WLC group did not answer the demographic component of the baseline survey. For purposes of this table, data from n = 650 total individuals are presented. Chi-square test was conducted for categorical variables.
# Fisher exact test was used in categorical variables with over 20% expected cell counts less than 5. The differences in continuous variables were tested using the Mann-Whitney U test.
p-value < .05.
^ Five select items from the Schedule Attitudes Survey (SAS).
Individual participant baseline characteristics did not differ by study arm for most demographic variables (Table 2); however, baseline group-level differences were observed for participant height, years of experience, age, working multiple jobs, and history of select health conditions (ie, high blood pressure). In unadjusted analyses, baseline differences were also observed in mean ESS, a secondary outcome, with a higher mean score among participants in the IAI group vs. the WLC group (p = .03), and 2 domains of the EMS-SAQ (“stress recognition” and “perceptions of management”; Table 2). In analyses adjusted for clustering, baseline differences in the IAI and WLC groups in age, high blood pressure, ESS, and 2 SAQ domains disappeared.
Over the 6-month study period, 142 participants withdrew and 109 were classified as LTFU (Fig. 1). During the first 3 months, the rate of withdrawals and LTFU combined (also known as attrition) was similar between groups (IAI = 17.4% vs. WLC = 18.2%, p = .37). Attrition during the first 3 months of the study was highest among fire-based (20.0%) and third-service model EMS agencies (20.3%), among agencies in the Southern U.S. Census region (20.1%), among agencies with 300+ employees (20.8%), among individuals that reported working full-time work (14.4%), and among individuals that regularly worked 24-hour shifts (15.9%). Attrition over the full 6-month study period did not differ by study arm (IAI = 34.5% vs. WLC = 39.2%; p = .20). Factors associated with attrition over the 6-month study period were agency type, highest among air-ground combined agencies (41.7%), region, agencies located in the Southern U.S. (39.4%), size, agencies with 300+ employees (40.8%), certification, participants with EMT-Basic certification/license (41.9%), full-time vs. part-time status, participants with full-time employment (35.1%), and shift type, participants that commonly worked 24-hour shifts (37.5%).
Our primary comparison of interest was sleep quality as measured by the PSQI instrument at baseline and at 3 months by group status. The available data for 3-month comparisons included 15 IAI agencies with 210 individual participants, and 20 WLC agencies with 225 participants who partially or fully completed the 3-month survey.
Primary outcome: Intent-to-treat analysis
In intention-to-treat analyses, the mean PSQI at 3-month follow-up did not differ by group (3-month p = .74). The mean change in PSQI among IAI participants is -0.17 standard deviations greater than the mean change among WLC participants (Cohen's d = -0.17; 95% CI -0.36, 0.02; "a small effect"). The percent change from baseline also did not differ by group at 3-month follow-up (p = .10; Fig. 2). The proportion of participants with poor sleep quality did not differ by IAI or WLC group status at 3-month follow-up (p = .86 See Online Supplementary Figures). The proportion with a clinically meaningful improvement in sleep quality at 3 months, compared to baseline, did not differ by group (p = .80; See Online Supplementary Figures).
Fig. 2Percentage and raw change in Pittsburgh Sleep Quality Index (PSQI) score from baseline to 3-month follow-up. Notes: IAI, immediate access to intervention group. WLC, wait list control group. PSQI, Pittsburgh Sleep Quality Index. Standard deviation (SD) represented by error bars. The directionality of bars showing the mean percentage change and mean raw change in PSQI for the total group and WLC group differ based on [1] the change in PSQI raw scores from baseline to 3 months was 0 for 14% of participants, with decreases ranging from -1 to -13 and increases ranging from 1 to 7; and [2] the percentage change in PSQI from baseline to 3 months showed decreases ranging from -5.9% to -100% and increases ranging from 6.3% to 200%. The high percentage change scores relative to baseline values resulted in the difference in directionality of some point estimates (a point estimate of positive mean percentage change and negative mean raw change) shown in this figure.
Our leading secondary measure of interest was mental and physical fatigue as measured by the CFQ. The mean CFQ score did not differ by group at 3-month follow-up (p = .45). The mean change in CFQ among IAI participants is 0.08 standard deviations greater than the mean change among WLC participants (Cohen's d = 0.08; 95% CI -0.11, 0.27; "a small effect"). The proportion of participants classified as fatigued by the CFQ did not differ by group status at 3-month follow-up (p = .86). The percentage change in CFQ score from baseline to 3-month follow-up did not differ by group status (p = .49; Fig. 3). The mean scores on secondary outcomes ESS, OFER, EMS-SAQ, and SAS did not differ by group status at 3-month follow-up (Table 3; p > .05 all comparisons).
Fig. 3Percentage and raw change in Chalder Fatigue Questionnaire (CFQ) score from baseline to 3-month follow-up. Notes: IAI, immediate access to intervention group. WLC, wait list control group. CFQ, Chalder Fatigue Questionnaire. Standard deviation (SD) represented by error bars.
Tests for differences in secondary measures by group status (IAI vs. WLC) use linear mixed models (at 3 months) with the secondary measure as the outcome, group status as the predictor, and a random intercept to adjust for clustering. p > .05 for all comparisons.
Five items out of 21 total items from the SAS^ tool were used in this study. Sample sizes are denoted by the following:
a n = 224.
b n = 222.
c n = 219.
p-value for sub-group analysis comparing change in PSQI p = .35.
^ Five select items from the Schedule Attitudes Survey (SAS).
During the first 3 months, 33% (n = 103) of all enrolled IAI participants viewed at least one education module. In total, n = 140 participants viewed one or more education modules during the first 3 months with 57% classified as Low module viewers, 11% as Moderate, and 32% as High. At 3-month follow-up, module viewing was associated with change in the mean PSQI (p = .02). For every 1 module viewed, PSQI-measured sleep quality improved by 0.12 points (p = .0048). The improvement was small and likely not clinically meaningful. Compared to no module viewing, participants viewing 8-10 modules experienced the greatest improvement in sleep quality (Overall p = .0024; Bonferonni-corrected p = .01). Participants in the High module-viewing group had a greater percentage change in PSQI from baseline compared to participants with no module viewing (Fig. 4; Overall p = .02; Bonferonni correct p = .06). The PSQI score decreased (sleep quality improved) by 0.12 points for every module viewed (Fig. 5a; p = .003). Thirty-seven individuals assigned to the WLC group were identified as viewing one or more modules immediately prior to completing the 3-month follow-up survey. Among these participants, the mean PSQI score worsened or did not change when compared to baseline.
Fig. 4Percentage and raw change in Pittsburgh Sleep Quality Index (PSQI) from baseline to 3-month follow-up by module viewing group. Notes: Low = refers to participants viewing 1 to 3 education modules. Moderate = refers to participants viewing 4 to 7 education modules. High = refers to participants viewing 8 to 10 education modules. PSQI, Pittsburgh Sleep Quality Index. Standard deviation (SD) represented by error bars. The directionality of bars showing the mean percentage change and mean raw change in PSQI for the None group differ based on [1] the change in PSQI raw scores from baseline to 3 months was 0 for 14% of all participants, with decreases ranging from -1 to -13 and increases ranging from 1 to 7; and [2] the percentage change in PSQI from baseline to 3 months showed decreases ranging from -5.9% to -100% and increases ranging from 6.3% to 200%. The high percentage change scores relative to baseline values resulted in the difference in directionality of point estimates (a point estimate of positive mean percentage change and negative mean raw change) for the no module-viewing group in this figure.
Fig. 5(a) Number of education modules viewed and change in Pittsburgh Sleep Quality Index (PSQI) scores. (b) Number of education modules viewed and change in Chalder Fatigue Questionnaire (CFQ) scores. Notes: Y-axis = change in scores from baseline to 3 months for the PSQI and CFQ. X-axis = number of education modules viewed by study participants. Solid line on graph = slope.
At 3-month follow-up, module viewing was associated with an improvement (reduction) in self-reported fatigue as measured by the CFQ. Fatigue decreased (improved) by 0.074 points for every module viewed (Fig. 5b; p = .04). The association between module-viewing status (Low, Moderate, and High) and fatigue was non-significant (Fig. 6; mean CFQ p = .08; percentage change in CFQ p = .16). Participants in the High module-viewing group experienced a greater mean change (reduction) in fatigue compared to participants with no module viewing, however, the association did not reach statistical significance (Overall p = .05; Bonferonni-corrected p = .15). Those in the High module-viewing group also experienced a greater percentage change in fatigue from baseline compared to participants with no module viewing, although this comparison was non-significant (Overall p = .06; Bonferonni-corrected p = .17; Fig. 6).
Fig. 6Percentage and raw change in Chalder Fatigue Questionnaire (CFQ) measured fatigue by module viewing group. Notes: Low = refers to participants viewing 1-3 education modules. Moderate = refers to participants viewing 4-7 education modules. High = refers to participants viewing 8-10 education modules. Standard deviation (SD) represented by error bars. The directionality of bars showing the mean percentage change and mean raw change in CFQ for the None and Moderate groups differ based on [1] the change in CFQ raw scores from baseline to 3 months was 0 for 25% of participants, with decreases in raw scores ranging from -1 to -7 and increases ranging from 1 to 6; and [2] the percentage change in CFQ from baseline to 3 months showed decreases ranging from -9% to -88% and increases ranging from 10% to 600%. The high percentage change scores relative to baseline values resulted in the difference in directionality of point estimates (a point estimate of positive mean percentage change and negative mean raw change) for the moderate module-viewing group in this figure.
For all other secondary outcomes ESS, OFER, EMS-SAQ, and SAS, change in the raw score was used as the outcome of interest because of substantial outliers in percentage change. None of these measures differed in terms of mean change by module viewing status at 3 months (p > .05 all comparisons). In addition, we performed sub-group analyses with participants who reported poor sleep quality at baseline. Findings from the sub-group analyses are similar to findings for the main comparison (Table 3).
Discussion
At 3-month follow-up, intention-to-treat analyses showed no impact of the education intervention on sleep quality or fatigue. Findings from the per-protocol analyses revealed that improvements in sleep quality and reductions in fatigue were associated with viewing education modules. The greater the number of education modules viewed, the greater the improvements in both sleep quality and fatigue.
The Fatigue Risk Management in Emergency Medical Services Education Program, developed specifically for this research study, is one of a few targeting public safety personnel,
and perhaps the first program specifically tailored to EMS clinician shift workers. Previous reviews of the evidence reveal wide variation in the structure and content of previously developed sleep health education programs.
Effect of fatigue training on safety, fatigue, and sleep in Emergency Medical Services personnel and other shift workers: a systematic review and meta-analysis.
Findings from these reviews provide evidence that tailoring to unique groups is likely necessary to impact outcomes like sleep quality. Notable characteristics of the education program tested in this trial include: [1] the wide breadth of topics covered in ten different modules; [2] the presentation of evidence germane to each topic interspersed with video-recorded commentary and interviews with EMS workers and clinical experts in sleep medicine; [3] the brevity of each module (eg, approximately 13 minutes each); and [4] the availability of continuing education credits following program completion. Intention-to-treat analyses showed no differences between the intervention and wait-list control groups. However, per protocol analyses showed that the participants who viewed the most education modules experienced a benefit, including improved sleep quality and reduced levels of fatigue. Our findings may inform the decisions of local EMS agency directors, who in recent years have been directed to provide education and training directly to EMS clinicians on the topics of sleep health and fatigue as part of employment on-boarding and continuing education practice.
a noticeably high proportion of participants (>80%) in this study reported being fatigued and/or having poor sleep quality at baseline. It is possible that the individuals who volunteered for this research study were, at the time of enrollment, more severely impacted by work-related fatigue than others who did not participate. It is also possible that work-related factors, such as patient volume, and other factors played a role in how participants reported fatigue at baseline. In other words, workload at the time of enrollment for many may have been high, and this was reflected in how individuals responded to sleep and fatigue questionnaires. Another potential explanation is that the COVID-19 pandemic played a unique role in the sleep patterns of public safety workers, which led to increased levels of fatigue. For many who enrolled, the COVID-19 pandemic altered day-to-day duties in public safety operations. Many EMS agencies instituted a mask (facial covering) mandate and use of other personal protective equipment to combat infection at the same time when our study was recruiting and enrolling. At the same time, call volumes (patient-related volume) fluctuated drastically in some parts of the U.S.
These factors and other work-related elements may have altered sleep behaviors and perceptions of fatigue, which manifested in a greater number of EMS clinicians, including those who participated in this study, reporting feelings of fatigue and poor sleep. Yet another potential explanation is that fatigue and poor sleep are so pervasive in EMS operations that the vast majority of personnel, regardless of the pandemic, are persistently fatigued (to some degree). Regardless of the reasons, which likely vary somewhat across location, fatigue and poor sleep are significant problems in EMS and public safety in general, and solutions that are tailored to EMS work conditions are needed.
While many benefited from the education program, others did not engage with the intervention or did not adhere to protocol in other ways. We describe several potential explanations for these findings. First, we believe that the COVID-19 pandemic, which began March 11, 2020, fundamentally changed the behavior of many, including the individuals that signed up for this trial. Recruitment and enrollment for our study began February 2020 and was paused from March 2020 until June 2020. During this time, the total volume of EMS dispatches (workload) experienced by many EMS organizations changed.
In addition, the sleeping habits and behaviors of most working aged adults were altered so that many (37% in some locations) obtained more sleep compared to pre-pandemic levels, whereas 17% (in some locations) reported less sleep compared to pre-pandemic levels.
which happened to overlap with the recruitment and data collection phases of this trial.
Second, we believe a lack of participant blinding may have played a role. Our study protocol used an open-label, wait-list control study design, which is a commonly used study design in sleep related research focused on changing behavior.
With this design, it was not feasible to blind participants or most of the study team to the randomization assignment of agencies and individuals within agencies post-randomization. Previous research has suggested that when participants become aware of their status in an experimental study, many may alter their behavior or responses to study related questions.
While we did not receive any complaints from agency leaders or individuals upon notification of their randomization assignment, findings from previous research would suggest that some participants in the wait-list control condition may have been displeased with their assignment and changed their behavior proximal to select points of follow-up.
Opportunities for improving the delivery and potential impact of this intervention include: [1] Integrating the education modules with new employee on-boarding and/or with annual or once every 2-year requirements for EMS continuing education. Most all EMS clinicians are required to complete a minimum set of continuing education hours annually in order to maintain state and/or national level certification/licensure. At present, there is no formal requirement for education and/or training focused on employee sleep health and fatigue risk management. Adding this type of education and training is feasible and may have a significant impact. [2] Awarding individuals credits for completing education and training that individuals can count towards annual requirements is a natural complement to adding sleep health and fatigue training to an existing EMS continuing education system. Education and training minus “credits” may have limited impact on education module engagement. [3] Frequent updates of the education materials based on the most recent and best available sleep health-focused research may elevate the novelty of the content and lead to increased engagement. [4] Finally, the delivery of education may need to evolve. For purposes of this research study, we deployed the modules on a password protected website. The website was compatible with mobile smartphone technology; thus participants could view the modules on their smartphones. However, website aesthetics or other aspects of our technology may have been limited in the eyes of many participants. As technology evolves, newer methods of dissemination may prove more engaging than what was used for in this research study.
Despite several challenges experienced during the execution of this trial, participant feedback suggests that the structure and format of the educational intervention was acceptable and informative. None who provided comment following study participation expressly identified a module or the contents of a module as not relevant or not informative, though the feedback provided was isolated to only those individuals willing to provide it. Although feedback was positive, the educational program could be improved. First, module production was limited to a select type of technology and software readily available to the study team. The production quality, animation, and other aesthetically related elements may be improved and consequently attract more viewers in the future. Second, the evidence germane to all topics should be updated. Future reviews of the published evidence on sleep health and fatigue mitigation will likely identify novel strategies supported by the evidence that would further inform the targeted population. Finally, the content of each module was designed to inform front-line clinicians on behaviors that may lead to improved sleep quality and fatigue. Given that fatigue risk management in the workplace is a shared responsibility between the employee and employer,
there is reason to believe that a companion set of education modules targeting the employer, and focused on employer related decisions, may be a worthwhile addition to the existing program.
Limitations
Our study is limited to those EMS agencies and individuals within these agencies that agreed to participate. While our sample may be biased to those who elected to enroll and participate, the demographic characteristics of those involved are similar to the characteristics of agencies and individual EMS clinicians involved in previous observational and experimental research.
Recruitment, enrollment, and attrition may have been impacted by the COVID-19 pandemic, which impacted sleep patterns when compared to pre-pandemic periods.
Changes in the behavioural determinants of health during the COVID-19 pandemic: gender, socioeconomic and ethnic inequalities in five British cohort studies.
Ten percent of individuals assigned to the wait-list control group (n = 37) viewed components of the intervention immediately prior to completing the 3-month survey. These individuals gained access to the materials early due to an error in the design of our online system that controlled access to surveys and education materials. Their responses to survey items and participation in the study may have been impacted. Similar to previous research,
many with access to the intervention failed to comply with protocol, which may have impacted our findings. Low adherence to protocol may have resulted in an attenuated effect of the intervention.
Conclusions
Intention-to-treat analyses revealed no differences in measures of sleep quality or fatigue at 3 months or 6 months follow-up. Per protocol analyses showed that the greater the number of education modules viewed, the greater the improvement in sleep quality and the greater the reduction in fatigue at 3-month follow-up. These findings suggest that the program is beneficial when the target population engages with the education. These findings may be useful to EMS administrators when designing fatigue risk management programs.
Declaration of conflict of interest
Authors report no conflicts of interest.
Funding
Work performed on this manuscript was supported with funding from the U.S. Department of Transportation, National Highway Traffic Safety Administration to the National Association of State EMS Officials, sub-contracted to the University of Pittsburgh. Contract number: DTNH2215R00029. The views contained in this article are those of the authors and not necessarily those of the National Highway Traffic Safety Administration.
Author contribution
Authors PDP, MDW, CGP, and KR conceived of the study idea and aims. Authors PDP, SEM, BNB, WH, MDW, TSO, SNS, and CGP collected and analyzed study related data. All authors participated in the synthesis and interpretation of study findings and writing of the manuscript.
Acknowledgments
Findings from this study and described in this manuscript appear in a report submitted to the National Highway Traffic Safety Administration on September 2, 2021.
in: Cone DC Brice JH Delbridge TR Myers JB Emergency Medical Services: Clinical Practice and Systems Oversight. John Wiley & Sons, Inc.,
2015: 211-216 (chap 20)
Cairns C, Kang K, Santo L. National Hospital Ambulatory Medical Care Survey: 2018 emergency department summary tables. U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics. Available at: https://www.cdc.gov/nchs/data/nhamcs/web_tables/2018-ed-web-tables-508.pdf Accessed July 14, 2021.
Effect of fatigue training on safety, fatigue, and sleep in Emergency Medical Services personnel and other shift workers: a systematic review and meta-analysis.
Mobile phone text messaging intervention to improve alertness and reduce sleepiness and fatigue during shiftwork among emergency medicine clinicians: study protocol for the SleepTrackTXT pilot randomized controlled trial.
Changes in the behavioural determinants of health during the COVID-19 pandemic: gender, socioeconomic and ethnic inequalities in five British cohort studies.
Previous presentations of this work: Work performed on this manuscript was supported with funding from the U.S. Department of Transportation, National Highway Traffic Safety Administration to the National Association of State EMS Officials, sub-contracted to the University of Pittsburgh. Contract number: DTNH2215R00029. Findings from this study and described in this manuscript appear in a report submitted to the National Highway Traffic Safety Administration on September 2, 2021.
p-value < .05.
00181-4&id=gr1.jpg){kind=link}
00181-4&id=gr2.jpg){kind=link}
00181-4&id=gr3.jpg){kind=link}
00181-4&id=gr4.jpg){kind=link}
00181-4&id=gr5.jpg){kind=link}
00181-4&id=gr6.jpg){kind=link}