Original Article

Curriculum Enrichment Across the Medical Education Continuum Using e-Delphi and the Community Priority Index

Authors: Hamisu M. Salihu, MD, PhD, Deepa Dongarwar, MS, Erik D. Malmberg, PhD, JD, Toi B. Harris, MD, Jennifer G. Christner, MD, William A. Thomson, PhD


Objective: To describe the use of the e-Delphi combined with the Community Priority Index (CPI) to support medical curriculum enrichment.

Methods: This mixed-methods study was conducted from December 2017 to May 2018 at Baylor College of Medicine in Houston, Texas. First, a nominal group identified a topical list. Second, to refine the curriculum content and achieve consensus, an e-Delphi was implemented with healthcare experts regarding the following target cohorts (N = 40): transformed postbaccalaureate premedical scholar students, medical students, clinical fellows, and junior faculty. Third, the CPI incorporated multicriteria decision making and calculation of standardized prioritization scores (range 0–1) with bootstrap 95% confidence intervals (CIs).

Results: Among transformed postbaccalaureate premedical scholar students, medical students, and clinical fellows, the e-Delphi-CPI system’s highest ranked topic was in the domain of leadership skills and competence for transformed postbaccalaureate premedical scholar students (CPI 0.87, 95% CI 0.58–0.94), medical students (CPI 0.85, 95% CI 0.36–0.91), and clinical fellows (CPI 0.86, 95% CI 0.32–0.92), respectively. For junior faculty, the highest ranked topic was introductory research methods (CPI 0.90, 95% CI 0.65–1.00). In each cohort, the top three ranked topics also contained leadership skills and competence and introductory research methods. The system ranked practical issues in health disparity as the third most valued domain among transformed postbaccalaureate premedical scholar students.

Conclusions: The integrated e-Delphi-CPI system identified the highest ranked options across all of the domains and established comparability across cohorts. We recommend the e-Delphi-CPI system to advance medical curriculum enrichment processes.

This content is limited to qualifying members.

Existing members, please login first.

If you have an existing account please login now to access this article or view your purchase options.

Purchase only this article ($15)

Create a free account, then purchase this article to download or access it online for 24 hours.

Purchase an SMJ online subscription ($75)

Create a free account, then purchase a subscription to get complete access to all articles for a full year.

Purchase a membership plan (fees vary)

Premium members can access all articles plus recieve many more benefits. View all membership plans and benefit packages.


1. Kern DE. Curriculum Development for Medical Education: A Six-Step Approach. Baltimore:Johns Hopkins University Press;2009.
2. McDonald FS, Schultz HJ, LaRusso NF. A learner-centered academic career development curriculum. Acad Med 2002;77:463.
3. Bowles N. The Delphi technique. Nurs Stand 1999;13:32-36.
4. Baker J, Lovell K, Harris N. How expert are the experts? An exploration of the concept of "expert" within Delphi panel techniques. Nurse Res 2006;14:59-70.
5. Stritter FT, Tresolini CP, Reeb KG. The Delphi technique in curriculum development. Teach Learn Med 1994;6:136-141.
6. Mitchell MP. Nursing education planning: a Delphi study. J Nurs Educ 1998;37:305-307.
7. Humphrey-Murto S, Varpio L, Wood TJ, et al. The use of the Delphi and other consensus group methods in medical education research: a review. Acad Med 2017;:1491-1498.
8. Dannenberg KA, Stroben F, Schr T, et al. The future of practical skills in undergraduate medical education-an explorative Delphi study. GMS J Med Educ 2016;33:Doc62.
9. Esmaily HM, Savage C, Vahidi R, et al. Identifying outcome-based indicators and developing a curriculum for a continuing medical education programme on rational prescribing using a modified Delphi process. BMC Med Educ 2008;8:33.
10. Broomfield D, Humphris GM. Using the Delphi technique to identify the cancer education requirements of general practitioners. Med Educ 2001;35:928-937.
11. Olson CA, Tooman TR, Leist JC. Contents of a core library in continuing medical education: a Delphi study. J Contin Educ Health Prof 2005;25:278-288.
12. Goodman CM. The Delphi technique: a critique. J Adv Nurs 1987;12:729-734.
13. Kennedy HP. Enhancing Delphi research: methods and results. J Adv Nurs 2004;45:504-511.
14. Chou C. Developing the e-Delphi system: a web-based forecasting tool for educational research. Br J Educ Technol 2002;33:233-236.
15. Donohoe H, Stellefson M, Tennant B. Advantages and limitations of the e-Delphi technique. Am J Health Educ 2012;43:38-46.
16. Bruggen E, Willems P. A critical comparison of offline focus groups, online focus groups and e-Delphi. Int J Market Res 2009;51:363-382.
17. Legare F, Borduas F, Jacques A, et al. Developing a theory-based instrument to assess the impact of continuing professional development activities on clinical practice: a study protocol. Implement Sci 2011;6:17.
18. Akins RB, Tolson H, Cole BR. Stability of response characteristics of a Delphi panel: application of bootstrap data expansion. BMC Med Res Methodol 2005;5:37.
19. Salihu HM, Salinas-Miranda AA, Paothong A, et al. Community-based decision making and priority setting using the R software: the community priority index. Comput Math Methods Med 2015;2015:347501.
20. Salihu HM, Salinas-Miranda AA, Wang W, et al. Community Priority Index: utility, applicability and validation for priority setting in community-based participatory research. J Public Health Res 2015;4:443.
21. Lie D, Boker J, Cleveland E. Using the tool for assessing cultural competence training (TACCT) to measure faculty and medical student perceptions of cultural competence instruction in the first three years of the curriculum. Acad Med 2006;81:557-564.
22. Lie DA, Boker J, Crandall S, et al. Revising the tool for assessing cultural competence training (TACCT) for curriculum evaluation: findings derived from seven US schools and expert consensus. Med Educ Online 2008;13:1-11.
23. Person SD, Jordan CG, Allison JJ, et al. Measuring diversity and inclusion in academic medicine: The Diversity Engagement Survey. Acad Med 2015;90:1675-1683.
25. Creswell JW, Plano Clark VL. Designing and Conducting Mixed Methods Research. Thousand Oaks, CA:SAGE Publications;2007.
26. Onwuegbuzie AJ, Bustamante RM, Nelson JA. Mixed research as a tool for developing quantitative instruments. J Mixed Methods Res 2010;4:56-78.
27. Tashakkori A, Teddlie C. Mixed Methodology: Combining Qualitative and Quantitative Approaches. Thousand Oaks, CA:Sage Publications;1998.
28. Teddlie C, Yu F. Mixed methods sampling: a typology with examples. J Mixed Methods Res 2007;1:77-100.
29. Salinas-Miranda AA, Nash MC, Salemi JL, et al. Cutting-edge technology for public health workforce training in comparative effectiveness research. Health Informatics J 2013;19:101-115.
30. Boulkedid R, Abdoul H, Loustau M, et al. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS One 2011;6:e20476.
31. Efron B, Tibshirani R. An Introduction to the Bootstrap. New York: Chapman & Hall; 1994.
32. Association of American Medical Colleges. Curriculum inventory (CI). aamc.org/initiatives/cir. Accessed September 25, 2018.
33. Association of American Medical Colleges. Initiatives. aamc.org/initiatives. Accessed September 25, 2018.
34. Hsu C-C, Sandford BA. The Delphi technique: making sense of consensus. Practical Assess Res Eval 2007;12:1-8.
35. Holey EA, Feeley JL, Dixon J, et al. An exploration of the use of simple statistics to measure consensus and stability in Delphi studies. BMC Med Res Methodol 2007;7:52.