Original Article

A Reliable and Valid Instrument to Assess Competency in Basic Surgical Skills in Second-Year Medical Students

Authors: Rachel Bramson, MD, Mark Sadoski, PhD, Charles W. Sanders, MD, Kim van Walsum, MS, Robert Wiprud, MD

Abstract

Background: Despite calls for competency based education, a dearth of validated instruments for measuring basic skills currently exists. We developed an instrument to assess competency in basic surgical skills in second-year medical students and tested it for psychometric reliability and validity.


Methods: From a review of the literature, an instrument comprised of numerically scaled items was constructed. After initial tests, several items were divided to produce a final instrument more specific and more appropriate for providing feedback to students. The final instrument was empirically tested for reliability and validity.


Results: The final 10-item instrument is presented here along with all of the empirical evidence including internal consistency reliability and interrater reliability, and content, criterion-related, and construct validity. Overall alpha reliability was 0.84 and interrater reliability was r = 0.83, P < 0.01 for the total scores. Factor analysis provided evidence of construct validity.


Conclusions: The instrument has psychometric properties adequate for use as one criterion for summative evaluation and is educationally practical enough to provide focused and detailed feedback for student improvement.


Key Points


* Despite calls for competency based medical education, there are few validated instruments for measuring basic skills.


* The development of a valid and reliable instrument to assess basic surgical skills is described.


* The final 10-item instrument is also practical for providing detailed, focused feedback for student remediation and improvement.

This content is limited to qualifying members.

Existing members, please login first

If you have an existing account please login now to access this article or view purchase options.

Purchase only this article ($25)

Create a free account, then purchase this article to download or access it online for 24 hours.

Purchase an SMJ online subscription ($75)

Create a free account, then purchase a subscription to get complete access to all articles for a full year.

Purchase a membership plan (fees vary)

Premium members can access all articles plus recieve many more benefits. View all membership plans and benefit packages.

References

1. Nelson MS, Traub S. Clinical skills training of US medical students. Acad Med 1993;68:926–928.
 
2. Medical Students Objectives Report #1. Acad Med 1999;74:13–18.
 
3. Sanders CW, Edwards JC, Burdenski TK. A survey of basic technical skills of medical students. Acad Med 2004;79:873–875.
 
4. Harden RM, Stevenson M, Downie WW, et al. Assessment of clinical competence using objective structured examination. Br Med J 1975;290:447–451.
 
5. Cizek G. Setting Performance Standards: Concepts, Methods, and Perspectives. Mahwah, NJ, Lawrence Erlbaum Associates, 2001.
 
6. Schueneman AL, Pickleman J, Freeark RJ. Age, gender, lateral dominance, and prediction of operative skill among general surgery residents. Surgery 1985;98:506–515.
 
7. Shueneman AL, Pickleman J, Hesslein R, et al. Neuropsychological predictors of operative skill among general surgery residents. Surgery 1984;97:288–293.
 
8. Reznick R, Regehr G, MacRae H, et al. Testing technical skill via an innovative ‘bench station’ examination. Am J Surg 1997;173:226–230.
 
9. Reznick RK. Teaching and testing technical skills. Am J Surg 1993;165:358–361.
 
10. Crocker L, Algina J. Introduction to Classical and Modern Test Theory. Fort Worth, Harcourt Brace Jovanovich College Publishers, 1986.
 
11. Likert R, Quasha W. The Revised Minnesota Paper Form Board Test. San Antonio, The Psychological Corporation, 1995.
 
12. Spielberger C, Gorsuch R, Lushene R, Vagg P, Jacobs G. Manual for State-Trait Anxiety Inventory.Consulting Psychologists Press, 1983.
 
13. Sanders CW, Sadoski M, Bramson R, et al. Comparing the effects of physical practice and mental imagery rehearsal on learning basic surgical skills by medical students. Am J Obstet Gynecol 2004;191:1811–1814.