The Southern Medical Journal (SMJ) is the official, peer-reviewed journal of the Southern Medical Association. It has a multidisciplinary and inter-professional focus that covers a broad range of topics relevant to physicians and other healthcare specialists.

SMJ // Article

Original Article

Mind the Gap: Rethinking Procedural Skill Assessment in Medical Residency

Authors: Pete Meliagros, MD, Adam Garber, MD, Matthew Ambrosio, MS, Le Kang, PhD, Rebecca Forrest, MD

Abstract

Objectives: The purpose of this study was to investigate the efficacy of various assessment tools in accurately evaluating resident proficiency in bedside procedures and to determine the potential value of incorporating self-assessment as a means to bridge the “competency gap” between perceived and actual procedural skill.

Methods: This retrospective, observational study evaluated resident performance of two common bedside procedures using evidence-based global rating scales (GRSs) and procedural checklists. We additionally compared supervisor assessments with resident self-assessments to identify discrepancies. Residents were assessed by a small group of expert faculty on a medicine procedure service at Virginia Commonwealth University from June 2016 to July 2020.

Results: Most residents (94% and 79%) performed all of the procedural checklist items. Only 79% and 48% of residents achieved both a GRS score ≥4 and completed all of the items on the checklist for each respective procedure. Residents were significantly more likely to perform all items on the checklist than obtain a GRS score ≥4 (P < 0.001) for both procedures. The mean number of procedures needed for resident and supervisor assessment to agree was 1.34 and 1.28.

Conclusions: The combined use of comprehensive performance checklists and GRSs, coupled with deliberate self-evaluation coaching and timely feedback, provides a more comprehensive and precise evaluation of procedural performance.

Full Article

Having trouble viewing the article content below? Click here to open it directly.

Images

Download Image

References

1. Lee GB, Chiu AM. Assessment and feedback methods in competency-based medical education. Ann Allergy Asthma Immunol 2021;128:256-262.
 
2. Chimea T, Kanji Z, Schmitz S. Assessment of clinical competence in competency-based education. Can J Dent Hyg 2020;54:83-91.
  Simons KB, et al. Implementation of competency-based medical education: are we addressing the concerns and challenges? Med Educ 2015;49:1086-1102.  
4. Ilgen JS, Ma IW, Hatala R, et al. A systematic review of validity evidence for checklists versus global rating scales in simulation-based assessment. Med Educ 2015;49:161-173.
 
5. Lenhard A, Moallem M, Marrie R, et al. An intervention to improve procedure education for internal medicine residents. J Gen Intern Med 2008;23:288-293.
 
6. Mourad M, Ranji S, Sliwka D. A randomized controlled trial of the impact of a teaching procedure service on the training of internal medicine residents. J Grad Med Educ 2012;4:170-175.
 
7. Meliagros P, Garber A, Sadr N, et al. Competency-based assessment of procedural skills in resident training. J Hosp Med   2023;18:703-718.
 
8. Yates N, Gough S, Brazil V. Self-assessment: with all its limitations, why are we still measuring and teaching it? Lessons from a scoping review. Med Teach 2022;44:1296-1302.
 
9. Gordon MJ. A review of the validity and accuracy of self-assessments in health professions training. Acad Med 1991;66:762-769.
 
10. Evans AW, McKenna C, Oliver M. Self-assessment in medical practice. J R Soc Med 2002;95:511-513..
 
11. Edgar L, , Hatlak K, , Haynes IL, , et al. Milestones guidebook. https://www.acgme.org/globalassets/MilestonesGuidebook.pdf?ver&#x003D;2020-06-11-100958-330. Published 2020. Accessed June 1, 2025.
 
12. Zheng B, He Q, Lei J. Informing factors and outcomes of self-assessment practices in medical education: a systematic review. Ann Med 2024;56:2421441.
 
13. Meliagros P, Forrest R. Introducing a comprehensive framework for competency-based procedure training. J Gen Intern Med 2025;40:3560-3565.
 
14. Pugh D, Cavalcanti RB, Halman S, et al. Using the entrustable professional activities framework in the assessment of procedural skills. J Grad Med Educ 2017;9:209-214.