Open Access

Comment on Alquraini et al.: reliability of Canadian Emergency Department Triage and Acuity Scale (CTAS) in Saudi Arabia

International Journal of Emergency Medicine20158:40

https://doi.org/10.1186/s12245-015-0090-3

Received: 9 September 2015

Accepted: 21 October 2015

Published: 11 November 2015

Abstract

It is very common to examine reliability of triage scales using (weighted) kappa statistics. The point is that weighted kappa has grossly underestimated disagreements by one category and put more emphasis on extreme category disagreements; therefore, low prevalence of critically-ill and non-urgent patients has excluded the effect of extreme categories disagreement from calculated kappa coefficient and also contributed to significant overestimation. As a result, weighted kappa coefficient as an estimate of scale reliability is overestimated by the anchoring effect.

Keywords

Kappa Agreement Reliability Triage Emergency

Dear editor

With great interest, we have read the outstanding publication in the recent International Journal of Emergency Medicine from Alquraini et al. entitled “Reliability of Canadian Emergency Department Triage and Acuity Scale (CTAS) in Saudi Arabia” [1].

The Canadian Triage and Acuity Score (CTAS) has been structured using narrative description of the patient’s medical condition including alarming signs and symptoms for each high-risk complaint [2]. However, narrative triage scales let nurses to decide much more freely than they do by algorithm-based scales such as Emergency Severity Index (ESI) [3], and also, the CTAS does not force nurses to allocate patients into any particular category [4]; it should be noted that it may affect triage concordance and lessen agreement among triage nurses instead.

We would like to bring your attention to the discrepancy concerning agreement across the extreme categories. Raw agreement among nurses has been reported 52.31 % for resuscitation (CTAS I), 55.63 % emergent (CTAS II), 61.13 % urgent (CTAS III), 53.85 % less urgent (CTAS IV), and 56.14 % for non-urgent (CTAS V) level [1], so it means there was concordance among categories only in half of the cases. It has also resulted in kappa value of 0.871 indicating good agreement [1]. Whereas, generally, there is an expected agreement on extreme categories more than twice as much as other categories [4]. The vital question is that if the kappa coefficient represents the extent to which reliability exists.

Two main issues must be clearly addressed to answer this question. First of all, it has been clarified that most of the cases triaged as urgent and less urgent (CTAS levels III and IV) [1], so it means ED census has provided certain type of patients for nurses to be triaged especially level III and IV patients. Existing awareness of high prevalence of urgent patients will lead toward a particular decision, and the triage nurses tended to quickly focus on urgent patients; it leads to completely wildly optimistic results and unrealistic substantial kappa value. The anchoring effects can occur, while nurses are less likely to choose a low-end or end-point of triage scale. Anchoring effects are almost always present as there is usually a less extreme way to label the high and low ends of a scale. Anchoring effect is a fully recognized bias in which people start with an implicitly suggested reference point (the “anchor”) and make adjustments to it to reach their estimate. People start with an initial estimation (anchor) and after that make further adjustments based on additional information. Insufficient adjustments often give the initial anchor a great deal of influence over future assessments [5].

In addition, weighted kappa has grossly underestimated disagreements by one category and put more emphasis on extreme category disagreements [6]; therefore, low prevalence of critically-ill and non-urgent patients has excluded the effect of extreme categories disagreement from calculated kappa coefficient and also contributed to current results either. As a result, weighted kappa coefficient as an estimate of scale reliability is overestimated by the anchoring effect. Overall, it has been suggested that the results should be interpreted with extreme cautious.

We highly recommend further studies on the reliability of the five-level triage scales and desired outcomes including time-related indices, mistriage rates (over-triage and under-triage), weighted and un-weighted kappa coefficients concerning different populations, and emergency clinicians in Saudi Arabia.

Abbreviations

CTAS: 

The Canadian Triage and Acuity Score

ESI: 

Emergency severity index

Declarations

Acknowledgements

I appreciate the assistance of Springer Open team.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors’ Affiliations

(1)
Evidence-Based Caring Research Center, Department of Medical-Surgical Nursing, School of Nursing and Midwifery, Mashhad University of Medical Sciences
(2)
Department of Medical-Surgical Nursing, School of Nursing and Midwifery, Chahrrah-e-Doktorha

References

  1. Alquraini M, Awad E, Hijazi R. Reliability of Canadian Emergency Department Triage and Acuity Scale (CTAS) in Saudi Arabia. Int J Emerg Med. 2015;8(1):80. doi:https://doi.org/10.1186/s12245-015-0080-5. Epub 2015 Aug 7.View ArticlePubMedGoogle Scholar
  2. Hamamoto J, Yamase H, Yamase Y. Impacts of the introduction of a triage system in Japan: a time series study. Int Emerg Nurs. 2014;22(3):153–8. doi:https://doi.org/10.1016/j.ienj.2013.10.006. Epub 2013 Oct 23.View ArticlePubMedGoogle Scholar
  3. Mirhaghi A, Heydari A, Mazlom R, Hasanzadeh F. Reliability of the emergency severity index: meta-analysis. Sultan Qaboos Univ Med J. 2015;15(1):e71–7. Epub 2015 Jan 21.PubMed CentralPubMedGoogle Scholar
  4. Mirhaghi A, Heydari A, Mazlom R, Ebrahimi M. The reliability of the Canadian Triage and Acuity Scale: meta-analysis. N Am J Med Sci. 2015;7(7):299–305. doi:https://doi.org/10.4103/1947-2714.161243.PubMed CentralView ArticlePubMedGoogle Scholar
  5. Friedman HH, Amoo T. Multiple biases in rating scale construction. J Int Marketing and Marketing Res. 1999;24:115–26.Google Scholar
  6. Ebrahimi M, Mirhaghi A. Re: Inter-rater reliability and validity of the Ministry of Health of Turkey’s mandatory emergency triage instrument. Emerg Med Australas. 2015. doi: https://doi.org/10.1111/1742-6723.12450.

Copyright

© Mirhaghi. 2015

Advertisement