Reliability of Canadian Emergency Department Triage and Acuity Scale (CTAS) in Saudi Arabia
© Alquraini et al. 2015
Received: 20 May 2015
Accepted: 30 July 2015
Published: 7 August 2015
The Canadian Emergency Department Triage and Acuity Scale (CTAS) is an integral part of the Canadian emergency medicine triaging system. There is growing interest and implementation of CTAS worldwide. However, little is known about its reliability outside Canada. The aim of this study was to determine the reliability agreement of CTAS in a tertiary care emergency center in Saudi Arabia.
Ten triage nurses (five senior and five junior nurses) utilized CTAS guidelines to independently assign a triage level for 160 real case-based scenarios. Quadratic weighted kappa statistics were used to measure raters’ agreements.
Raters provided 1600 triage category assignments to case scenarios for analysis. Intra-rater agreement was similar for both senior and junior nurses; for senior nurses (SN1) kappa 0.871 95 % CI (0.840–0.897), and for junior nurses (SN2) kappa 0.871 95 % CI (0.839–0.898). Inter-rater agreement for the SN1 versus SN2 nurses had statistically meaningful agreement across different triage levels (weighted kappa = 0.770) 95 % CI (0.742–0.797).
CTAS has good reliability among emergency department (ED) triage nurses in King Abdulaziz Medical City (KAMC), Saudi Arabia. The findings suggest that CTAS might be a reliable instrument when applied in countries outside Canada.
Triage systems aim to distribute medical resources according to patients’ needs . Trained triage nurses are usually responsible for giving a triage level for patients’ conditions based on an established triage system within a short timeframe. Applying triage can lead to safe and efficient utilization of an emergency department (ED) . The foundations of triage systems take into consideration the values of human life, health care resources, and fairness in distribution . A valid and reliable triage instrument would serve these values. Most modern triage systems are reported to be valid; however, there are wide variations in their reliability (k = 0.25 to 0.91) . Reliability of a triage system is of paramount importance toward achieving its stated goals. Different raters should come to the same decisions regarding patient prioritization to avoid unnecessary delay of treatment, especially as it has been shown that early treatment of certain conditions can improve outcomes .
There are several triage systems implemented around the world, such as the Canadian Emergency Department Triage and Acuity Scale (CTAS), the Emergency Severity Index (ESI), the Australian Triage Scale (ATS), and the Manchester Triage System (MTS) . The CTAS triage system has been utilized in several leading health care institutes in Saudi Arabia for almost 10 years, including King Abdulaziz Medical City (KAMC), National Guard Health Affairs (NGHA). There are no national unified standards for triage system in Saudi Arabia. Currently, many Ministry of Health (MOH), non-MOH, and private Hospitals in Saudi Arabia are using CTAS .
CTAS time objectives
Time for triage assignment
Time to nursing reassessment
Time to physician assessment
Every 15 min
Every 30 min
Every 60 min
Every 120 min
CTAS operational objectives can be achieved and implemented outside Canada . Based on their study in Saudi Arabia, Elkum et al. found that CTAS could be implemented with achievable objectives, however the study was in a center with a special population, and it has not assessed CTAS reliability . In this study, we aim to assess the inter-rater reliability of CTAS in our current clinical setting, which can serve as a base for benchmarking and future quality improvement in the triage process in KAMC as well as other hospitals in Saudi Arabia.
The local institutional review board approved this study. We utilized electronically registered triage data for adult patients visiting the Emergency Care Center (ECC) at KAMC. KAMC is a tertiary care facility in Riyadh, Saudi Arabia. Its emergency care center is one of the largest in the Middle East with a capacity of 100 beds and more than 200,000 annual ED visits. Ten triage nurses (five senior and five junior nurses) were randomly selected by stratified randomization according to nurses rank and invited to participate in the study. Nursing ranking (senior vs. junior) in our institution depends on qualifications and developed countries experiences and not only to number of years of practice. All participants expected to have completed the mandatory triage course and be re-certified every 2 years. The triage course is based on CTAS 2008 guidelines and is delivered by our hospital. Case scenarios were extracted from real data of ED patient visits registered in QuadraMed’s Computerized Patient Record (QCPR) in the preceding 3 months before the start of the study. A total of 160 emergency patients’ assessment data was included (chief complaint, mode of arrival, vital signs, pain score, and general appearance), collected, and summarized as case scenarios on paper sheets. Patients’ visits were randomly selected through a two-stage stratified randomization by date and shifts in a retrospective pattern. Two investigators reviewed the case scenario answer sheets for data and language consistency and appropriateness. Pilot testing of answer sheets was done by a senior ED nurse who was not invited to participate in our study. After revision, we drafted the final version for the purpose of the study. Each rater independently assigned a triage level for 160 cases based on the information provided.
Altman’s kappa strength interpretation criteria
Strength of agreement
Demographics and training background of the study cohort
Triage course taken
Last triage course taken
Number of years of practice [mean ± SE]
12.80 ± 1.93
11.40 ± 2.87
Kappa values for CTAS inter-observer agreement
95 % confidence interval (lower bound–upper bound)
Senior nurses (SN1) vs. junior nurses (SN2)
In our study, we assessed CTAS reliability by utilizing real case, paper-based scenarios rated by both senior and junior nurses in KAMC. Most of the cases triaged as urgent and less urgent (CTAS levels III and IV), which depict our ED census. However, our trial showed CTAS score distribution had highest agreement (61.13 %) for the urgent level (CTAS III). This finding is not congruent with previous studies that report the highest agreement across the extreme categories (e.g., CTAS level I) . This could be explained by the fact that patients rated CTAS levels I and II, in our institution, are always assigned and shifted to the ED Critical Care (CC) area and seen almost always immediately. CTAS category V and most of level IV patients, on the other hand, are sent to the Urgent Care Center (UCC) in the ED. In addition, due to overcrowding, our nurses are not infrequently questioned by staff about their assignment, which places an implicit pressure on their decision. This ED design and patient flow may play a role in triage. Nonetheless, our results showed overall good reliability of the CTAS instrument.
For the second decade since its inception, CTAS is still being tested for its reliability. Most of the studies have been conducted in Canadian EDs with very few outside Canada. Our results suggest that CTAS has good reliability when applied in one of the major EDs in Saudi Arabia. Our local setting trial outcome is similar to previous Canadian studies that showed good reliability of CTAS [16, 17]. However, after CTAS’s major revision in 2004 followed by another in 2008 [8, 9], further studies showed less reliable results. Dallaire et al. found moderate agreement in their trial that compared base hospital nurses versus ED triage nurses , where base hospital nurses triaged the patients in the pre-hospital phase and then ED nurse re-triage them in ED. Their results could contradict our findings, perhaps due to time lag from pre-hospital to hospital arrival with the possibility of a change in patients’ condition during transport. In another study, Dallaire showed that CTAS inter-rater agreement was moderate when involving experienced nurses . In our current trial, we had five senior and five junior nurses with considerable years of practice and experience, and more than half of them did not have recent triage course re-certification. Although there are differences in practice settings, this might open an avenue for further studies to assess the influence of rater’s experiences upon CTAS reliability. In addition, Fernandes found that applying the 2008 CTAS guideline yields moderate reliability compared to good reliability when using the former version of the triage guideline . Our finding proved that CTAS has good reliability; however, we did not aim to compare previous CTAS guidelines to determine the effect on reliability.
In our study, we used paper-based case scenarios instead of real ED patient encounters. Although we copied the verbatim written visual clues in the triage notes to try matching the holistic patient assessment, real patient interaction may still influence triaging. Nonetheless, Worster et al. showed that using case scenarios is an acceptable alternative for triage reliability testing . Another limitation for generalizability is conducting the study in one academic-affiliated ED. However, we believe that our setting represents an ideal ED in the region.
In conclusion, CTAS has good reliability among ED nurses in KAMC, Saudi Arabia. The findings in our study suggest that CTAS might be a reliable instrument when applied in countries outside Canada. Further investigations may be warranted to explore the influence of rater’s experiences on the reliability of the updated CTAS guidelines.
We would like to thank all the participant nurses for their contributions and express our thanks for Dr. Ineke van der Wulp and the biostatistics and modeling section at King Abdullah International Medical Research Center.
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Iserson KV, Moskop JC. Triage in medicine, part I: concept, history, and types. Ann Emerg Med. 2007;49(3):275–81.PubMedView ArticleGoogle Scholar
- Hamamoto J, Yamase H, Yamase Y. Impacts of the introduction of a triage system in Japan: a time series study. Int Emerg Nurs. 2014;22(3):153–8.PubMedView ArticleGoogle Scholar
- Moskop JC, Iserson KV. Triage in medicine, part II: underlying values and principles. Ann Emerg Med. 2007;49(3):282–7.PubMedView ArticleGoogle Scholar
- Christ M, Grossmann F, Winter D, Bingisser R, Platz E. Modern triage in the emergency department. Dtsch Arztebl Int. 2010;107(50):892–8.PubMed CentralPubMedGoogle Scholar
- Rivers EP, Katranji M, Jaehne KA, Brown S, Abou Dagher G, Cannon C, et al. Early interventions in severe sepsis and septic shock: a review of the evidence one decade later. Minerva Anestesiol. 2012;78(6):712–24.PubMedGoogle Scholar
- Qureshi NA. Triage systems: a review of the literature with reference to Saudi Arabia. East Mediterr Health J. 2010;16(6):690–8.PubMedGoogle Scholar
- Beveridge R, Clarke B, Janes L, Savage N, Thompson J, Dodd G. Canadian emergency department triage and acuity scale: implementation guidelines. CJEM. 1999;1(3 Suppl):002.Google Scholar
- Murray M, Bullard M, Grafstein E. CEDIS National Working Group. Revisions to the Canadian emergency department triage and acuity scale implementation guidelines. CJEM. 2004;6(6):421–7.PubMedGoogle Scholar
- Bullard MJ, Unger B, Spence J, Grafstein E. CTAS National Working Group. Revisions to the Canadian emergency department triage and acuity scale (CTAS) adult guidelines. CJEM. 2008;10(2):136–51.PubMedGoogle Scholar
- Bullard MJ, Chan T, Brayman C, Warren D, Musgrave E, Unger B. Revisions to the Canadian emergency department triage and acuity scale (CTAS) guidelines. CJEM. 2014;16:1–5.Google Scholar
- Dallaire C, Poitras J, Aubin K, Lavoie A, Moore L. Emergency department triage: do experienced nurses agree on triage scores? J Emerg Med. 2012;42(6):736–40.PubMedView ArticleGoogle Scholar
- Jiménez JG, Murray MJ, Beveridge R, Pons JP, Cortés EA, Garrigós JB, et al. Implementation of the Canadian emergency department triage and acuity scale (CTAS) in the principality of Andorra: can triage parameters serve as emergency department quality indicators? CJEM. 2003;5(5):315–22.PubMedGoogle Scholar
- Elkum NB, Barrett C, Al-Omran H. Canadian emergency department triage and acuity scale: implementation in a tertiary care center in Saudi Arabia. BMC Emerg Med. 2011;11:3.PubMed CentralPubMedView ArticleGoogle Scholar
- Wulp I. Reliability and validity of emergency department triage systems. 2010.Google Scholar
- Altman D. Practical statistics for medical research. London (UK): Chapman and Hall; 1991.Google Scholar
- Manos D, Petrie DA, Beveridge RC, Walter S, Ducharme J. Inter-observer agreement using the Canadian emergency department triage and acuity scale. CJEM. 2002;4(1):16–22.PubMedGoogle Scholar
- Beveridge R, Ducharme J, Janes L, Beaulieu S, Walter S. Reliability of the Canadian emergency department triage and acuity scale: interrater agreement. Ann Emerg Med. 1999;34(2):155–9.PubMedView ArticleGoogle Scholar
- Dallaire C, Poitras J, Aubin K, Lavoie A, Moore L, Audet G. Interrater agreement of Canadian emergency department triage and acuity scale scores assigned by base hospital and emergency department nurses. CJEM. 2010;12(1):45–9.PubMedGoogle Scholar
- Fernandes CM, McLeod S, Krause J, Shah A, Jewell J, Smith B, et al. Reliability of the Canadian triage and acuity scale: interrater and intrarater agreement from a community and an academic emergency department. CJEM. 2013;15(4):227–32.PubMedGoogle Scholar
- Worster A, Sardo A, Eva K, Fernandes CM, Upadhye S. Triage tool inter-rater reliability: a comparison of live versus paper case scenarios. J Emerg Nurs. 2007;33(4):319–23.PubMedView ArticleGoogle Scholar