The Southern Medical Journal (SMJ) is the official, peer-reviewed journal of the Southern Medical Association. It has a multidisciplinary and inter-professional focus that covers a broad range of topics relevant to physicians and other healthcare specialists.
SMJ // Article
Original Article
Aligning Competency Committee and Observing Faculty Needs in Pursuit of Useful Written Evaluations of Residents
Abstract
Objectives: End-of-rotation narrative evaluations by faculty of Internal Medicine residents provide feedback to residents directly and to the program leadership to assess resident competency; however, observing faculty often lack an understanding of what information is useful to the program leadership. Faculty also face barriers in completing evaluations and formulating high-quality evaluations. We sought to qualitatively evaluate the interplay of the needs of the program leadership and barriers faculty face in providing high-quality evaluations.Methods: We conducted a descriptive qualitative study through semistructured interviews with six faculty with program leadership positions and six observing clinical faculty at a single large Internal Medicine residency program.
Results: Interview themes elucidated a multifaceted process that observing faculty must navigate in writing narrative evaluations and the important features of a useful narrative evaluation itself. Faculty must navigate assessment committee needs and their own competing priorities and be skilled in observing residents and navigating emotional barriers with residents. Successfully navigating these factors allows them to produce useful narrative evaluations that ultimately inform assessment committee decisions.
Conclusions: Understanding the complex dynamics of observing faculty factors and assessment committee needs can help guide future interventions to improve the quality and utility of written evaluations in assessing residents.
This content is limited to qualifying members.
Existing members, please login first
If you have an existing account please login now to access this article or view purchase options.
Purchase only this article ($25)
Create a free account, then purchase this article to download or access it online for 24 hours.
Purchase an SMJ online subscription ($75)
Create a free account, then purchase a subscription to get complete access to all articles for a full year.
Purchase a membership plan (fees vary)
Premium members can access all articles plus recieve many more benefits. View all membership plans and benefit packages.
References
1. Accreditation Council for Graduate Medical Education. ACGME program requirements for graduate medical education in internal medicine. https://www.acgme.org/globalassets/pfassets/programrequirements/140_internalmedicine_2023.pdf. Published 2022. Accessed June 1, 2025.
2. Andolsek K, Padmore J, Hauer KE, et al. Clinical competency committees: a guidebook for programs, 3rd ed. https://www.acgme.org/globalassets/acgmeclinicalcompetencycommitteeguidebook.pdf. Published 2020. Accessed June 1, 2025.
3. Hauer KE, Chesluk B, Iobst W, et al. Reviewing residents' competence: a qualitative study of the role of clinical competency committees in performance assessment. Acad Med 2015;90: 1084–1092.
4. Dauphinee WD. Building a core competency assessment program for all stakeholders: the design and building of sailing ships can inform core competency frameworks. Adv Health Sci Educ 2020;25:189–193.
5. Ekpenyong A, Baker E, Harris I, et al. How do clinical competency committees use different sources of data to assess residents’ performance on the internal medicine milestones? A mixed methods pilot study. Med Teach 2017;39:1074–1083.
6. Ginsburg S, McIlroy J, Oulanova O, et al. Toward authentic clinical evaluation: pitfalls in the pursuit of competency. Acad Med 2010;85:780–786.
7. Ginsburg S, van der Vleuten CPM, Eva KW. The hidden value of narrative comments for assessment: a quantitative reliability analysis of qualitative data. Acad Med 2017;92:1617–1621.
8. Green ML, Holmboe E. Perspective. The ACGME toolbox: half empty or half full? Acad Med 2010;85:787–790.
9. Lye PS, Biernat KA, Bragg DS, et al. A pleasure to work with—an analysis of written comments on student evaluations. Ambul Pediatr 2001;1:128–131.
10. Ramani S, Krackov SK. Twelve tips for giving feedback effectively in the clinical environment. Med Teach 2012;34:787–791.
11. Schumacher DJ, Martini A, Bartlett KW, et al. Key factors in clinical competency committee members’ decisions regarding residents’ readiness to serve as supervisors: a national study. Acad Med 2019;94:251–258.
12. Kelleher M, Kinnear B, Sall DR, et al. Warnings in early narrative assessment that might predict performance in residency: signal from an internal medicine residency program. Perspect Med Educ 2021;10:334–340.
13. Holmboe ES, Sherbino J, Long DM, et al. The role of assessment in competency-based medical education. Med Teach 2010;32:676–682.
14. Holmboe ES, Yamazaki K, Hamstra SJ. The evolution of assessment: thinking longitudinally and developmentally. Acad Med 2020;95(Suppl 11 Association of American Medical Colleges Learn Serve Lead: Proceedings of the 59th Annual Research in Medical Education Presentations):S7–S9.
15. Holmboe ES, Fiebach NH, Galaty LA, et al. Effectiveness of a focused educational intervention on resident evaluations from faculty: a randomized controlled trial. J Gen Intern Med 2001;16:427–434.
16. Dudek NL, Marks MB, Wood TJ, et al. Quality evaluation reports: can a faculty development program make a difference? Med Teach 2012;34:e725–731.
17. Archer JC. State of the science in health professional education: effective feedback. Med Educ 2010;44:101–108.
18. Dudek N, Dojeiji S. Twelve tips for completing quality in-training evaluation reports. Med Teach 2014;36:1038–1042.
19. Hauer KE, Cate OT, Boscardin CK, et al. Ensuring resident competence: a narrative review of the literature on group decision making to inform the work of clinical competency committees. J Grad Med Educ 2016;8:156–164.
20. Bradshaw C, Atkinson S, Doody O. Employing a qualitative description approach in health care research. Glob Qual Nurs Res 2017;4:2333393617742282.
21. Varpio L, Paradis E, Uijtdehaage S, et al. The distinctions between theory, theoretical framework, and conceptual framework. Academic Med 2020;95:989–994.
22. Miller WL, Crabtree BF. Primary care research: a multimethod typology and qualitative road map. In: MillerWL, Crabtree BF, eds. Doing Qualitative Research. Newbury Park, CA: Sage; 1992:3–28.
23. Birks M, Chapman Y, Francis K. Memoing in qualitative research: probing data and processes. J Res Nurs 2008;13:68–75.
24. Patton MQ. Enhancing the quality and credibility of qualitative analysis. Health Serv Res 1999;34(5 Pt 2):1189–1208.
25. Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care 2007;19:349–357.
26. Guest G, Bunce A, Johnson L. How many interviews are enough? An experiment with data saturation and variability. Field Methods 2006;18:59–82.
27. Lockyer J, Carraccio C, Chan MK, et al. Core principles of assessment in competency-based medical education. Med Teach 2017;39:609–616.
28. Zelenski AB, Tischendorf JS, Kessler M, et al. Beyond "read more": an intervention to improve faculty written feedback to learners. J Grad Med Educ 2019;11:468–471.
29. Bakke BM, Sheu L, Hauer KE. Fostering a feedback mindset: a qualitative exploration of medical students’ feedback experiences with longitudinal coaches. Acad Med 2020;95: 1057–1065.
30. Hauer KE, O’Sullivan PS, Fitzhenry K, et al. Translating theory into practice: implementing a program of assessment. Acad Med 2018;93:444–450.
31. Hattie J, Timperley H. The Power of feedback. Rev Educ Res 2007;77:81–112.
32. Marcotte L, Egan R, Soleas E, et al. Assessing the quality of feedback to general internal medicine residents in a competency-based environment. Can Med Educ J 2019;10:e32–e47.
