Open Access

Board review course effect on resident in-training examination

International Journal of Emergency Medicine20081:68

DOI: 10.1007/s12245-008-0068-5

Received: 27 April 2008

Accepted: 10 September 2008

Published: 28 October 2008

Abstract

Background

The in-training examination is a national and yearly exam administered by the American Board of Emergency Medicine to all emergency medicine residents in the USA. The purpose of the examination is to evaluate a resident’s progress toward obtaining the fundamental knowledge to practice independent emergency medicine.

Aims

The purpose of this study was to determine the effects of a 40 hour board review lecture course on the resident in-training examination in emergency medicine.

Methods

A 40 hour board review lecture course was designed and implemented during the weekly 5 hour long resident conferences during the 8 weeks preceding the in-training examination date in 2006. Attendance was mandatory at the Accreditation Council for Graduate Medical Education (ACGME) standard of 70% or greater. A positive result was considered to be a 10% increase or greater in the resident’s individual national class percentile ranking among their national peers for their class year for the emergency medicine in-training examination. A resident was excluded from the study if there was no 2005 in-training examination score for self-comparison. The 95% confidence intervals (CI) were used to analyze the results.

Results

Of 16 residents, 1 (6.25%; 95% CI: 0–18%) showed a positive result of increasing their national class percentile ranking by 10% or greater. For the PGY2, one of the eight had a positive result (12.5%; 95% CI: 0–35.4%). For PGY3, no resident (0%; 95% CI: 0–35.4%) had a positive result.

Conclusions

A 40 hour board review lecture course has no positive effect on improving a resident’s in-training examination score.

Keywords

In-training examination Board review Emergency medicine resident

Introduction

The American Board of Emergency Medicine (ABEM) in-training examination is a yearly national cognitive test administered to emergency medicine residents in USA to evaluate progress toward obtaining the fundamental knowledge to be independent emergency physicians. According to the ABEM website, residents with higher in-training examination scores have a higher likelihood of passing the initial qualifying examination to be certified emergency medicine physicians than residents with lower examination scores [1]. While there are many commercial weeklong board review lecture courses for the initial qualifying certification exam for emergency medicine, there is currently no review lecture course for the emergency medicine in-training examination. The author sought to determine the effects of implementing a 40 hour board review lecture course for resident in-training examination scores.

Methods

A 40 hour board review lecture course was designed and implemented during the weekly 5 hour long resident conferences during the 8 weeks preceding the in-training examination date in 2006. The review course was based on the topics outlined in the “model of clinical practice of emergency medicine” [2]. The course consisted of lecture reviews of test taking skills, trauma, pediatric emergencies, toxicological emergencies, cardiac emergencies, pulmonary emergencies, gastroenterology emergencies, neurology-psychiatry emergencies, endocrine emergencies, hematology-oncology emergencies, otorhinolaryngology-ophthalmology emergencies, genitourinary emergencies, obstetric-gynecology emergencies, orthopedic emergencies, dermatology-environmental emergencies, and emergency medicine potpourri given by the emergency medicine faculty at a university medical center. To accommodate the inclusion of the 40 hour review lecture series, the 4 hour monthly journal club was suspended for 10 months. Furthermore, attendance was mandatory at these lectures at the standard Accreditation Council for Graduate Medical Education (ACGME) guideline of 70% or greater [3]. A positive result was chosen to be a 10% increase or greater in the resident’s individual national class percentile ranking among their national peers for their class year for the emergency medicine in-training examination. A resident was excluded from the study if there was no 2005 in-training examination score for self-comparison. The 95% confidence intervals (CI) were used to analyze the results.

Results

Of 16 residents, 1 (6.25%; 95% CI: 0–18%) showed a positive result of increasing their national class percentile ranking by 10% or greater. For the PGY2, one of the eight had a positive result (12.5%; 95% CI: 0–35.4%). For PGY3, no resident (0%; 95% CI: 0–35.4%) had a positive result.

Discussion

Embarking on a successful career in emergency medicine requires a lifelong commitment to learning that starts during the residency years. The ABEM administers the in-training examination during residency, the initial qualifying certification examination post-residency, the recertification examination every 10 years as well as the yearly Lifelong Learning and Self-Assessment (LLSA) examination. In an effort to help individuals pass these examinations, many commercial review lecture courses have been developed for the board examination as well as the LLSA. Whether these courses are beneficial has not been formally studied using a rigorous scientific method. Based on the review courses advertisement brochures, it seems a very large percentage of their course participants pass the examination. Furthermore, some commercial review courses offer a money back guarantee and given the longevity of the course, one may surmise that more course participants pass the examination than fail the examination as the course continues to be profitable.

Because of the success of these courses, a 40 hour board review lecture course was designed and given to the residents prior to their in-training examination. Since this is a non-blinded study, one would expect a biased positive effect as the residents knew that the lectures were given to improve their individual in-training examination score. Interestingly, only 1 of 16 had a positive result of increasing their national class percentile ranking by 10% or greater in the study. Gillen’s study of a structured board review consisting of monthly reading assignments in an emergency medicine textbook followed by a graded multiple choice written examination showed no favorable effect on the ABEM in-training examination for emergency medicine PGY 2 and PGY 3 residents. Instead, the study found a decrease in the mean EM-2 institution program mean scores from 66.8 to 65.4 and mean EM-3 institution program mean score of 74.4 to 67.4 though these decreases were not statistically significant in the study [4]. However, the study did show an improvement in the intern class institution program mean score from 50.7 to 68.9. In the present study, national class year percentile ranking rather than mean scores was chosen because the mean scores tend to increase per PGY years and these mean scores differ yearly, while one would expect an individual’s national class percentile ranking to be more constant. For example, if a resident had a score of 70 which placed him at the 50 percentile ranking among his national peers, one would expect the resident to increase his score next year secondary to increased emergency medicine training, but more likely to remain at the 50 percentile ranking next year among his national class year peers who would have also improved with their similar increased training. Interns were excluded from this study because they did not have a previous in-training examination for self-comparison. Furthermore, the mean examination score for the emergency medicine intern class varies widely year to year in our institution.

Currently, there is no explanation for the lack of increase in the resident’s national class percentile ranking with the board review course. In other medical specialties, Wigton gave 40 1-h lectures on gastroenterology over a 1-month period to 24 internal medicine residents as well as had them take a multiple choice examination before and 3 weeks after the lecture series with mean scores increasing less than 10% [5]. Similarly, Bull et al. compared didactic lectures given by faculty to reviews prepared by residents from a major thoracic surgery textbook and found a statistically significant percentile increase of 31% with the resident reviews compared to 11% with faculty lectures (P < 0.05) [6]. Also, Wade and Kaminski compared the influence of different educational methods [textbooks, grand rounds, preceptor guidance, review courses, Surgical Education and Self-Assessment Program (SESAP)] on the in-training examination of surgical residents and found that independent study methods were the most effective pathway to successful examination performance [7].

Finally, several limitations of this study need to be recognized. First, the design of the 40 hour board review lectures has never been validated nor has the 10% increase expectation been validated. The author felt that the 10% increase was a reasonable goal given the time and effort devoted to designing a 40 hour emergency medicine review. A 5% increase was felt to be too low a goal given the time and effort expended on the review course, while a 20% increase was too unreasonable. Another limitation is that this study occurred at a single institution. Furthermore, this study involves a small sample size. Also, confounding factors such as lack of sleep and test anxiety may play a larger hidden role. Finally, there is no incentive for performance on the in-training examination as opposed to the certifying examination where passing the examination allows ABEM board certifications as well as avoids additional fees for retaking the examination.

Conclusion

A 40 hour board review lecture course had no positive effect on improving a resident’s in-service exam score.

Notes

Declarations

Funding/conflict of interest

The author declares that he has no conflict of interest or disclosures.

Authors’ Affiliations

(1)
UH Emergency Department

References

  1. ABEM (2008) In-training examination overview. American Board of Emergency Medicine. Available at http://www.abem.org. Accessed 22 Apr 2008
  2. Thomas HA, Binder LS, Chapman DM et al (2006) The 2003 model of clinical practice of emergency medicine: the 2005 update. Acad Emerg Med 13(10):1070–1073PubMedView ArticleGoogle Scholar
  3. ACGME (2007) ACGME program requirements for graduate medical education in emergency medicine: Section II.A.4.s (p 8)Google Scholar
  4. Gillen JP (1997) Structured emergency medicine board review and resident in-service examination scores. Acad Emerg Med 4(7):715–717PubMedView ArticleGoogle Scholar
  5. Wigton RS (1981) Effect of lectures and increased experience in gastroenterology on examination scores of internal medicine residents. Gastroenterology 80(3):601–604PubMedGoogle Scholar
  6. Bull DA, Stringham JC, Karwande SV, Neumayer LA (2001) Effect of a resident self-study and presentation program on performance on the thoracic surgery in-training examination. Am J Surg 181(2):142–144PubMedView ArticleGoogle Scholar
  7. Wade TP, Kaminski DL (1995) Comparative evaluation of educational methods in surgical resident education. Arch Surg 130(1):83–87PubMedView ArticleGoogle Scholar

Copyright

© Springer-Verlag London Ltd 2008