Original Article

Early Performance of the Patients Over Paperwork Initiative among Family Medicine Physicians

Authors: Oliver T. Nguyen, MSHI, Karim Hanna, MD, Lisa J. Merlo, PhD, MPE, Arpan Parekh, BA, Amir Alishahi Tabriz, MD, PhD, Young-Rock Hong, PhD, MPH, Sue S. Feldman, RN, PhD, Kea Turner, PhD, MPH


Objectives: In 2019, the Centers for Medicare & Medicaid Services began implementing the Patients Over Paperwork (POP) initiative in response to clinicians reporting burdensome documentation regulations. To date, no study has evaluated how these policy changes have influenced documentation burden.

Methods: Our data came from the electronic health records of an academic health system. Using quantile regression models, we assessed the association between the implementation of POP and clinical documentation word count using data from family medicine physicians in an academic health system from January 2017 to May 2021 inclusive. Studied quantiles included the 10th, 25th, 50th, 75th, and 90th quantiles. We controlled for patient-level (race/ethnicity, primary language, age, comorbidity burden), visit-level (primary payer, level of clinical decision making involved, whether a visit was done through telemedicine, whether a visit was for a new patient), and physician-level (sex) characteristics.

Results: We found that the POP initiative was associated with lower word counts across all of the quantiles. In addition, we found lower word counts among notes for private payers and telemedicine visits. Conversely, higher word counts were observed in notes that were written by female physicians, notes for new patient visits, and notes involving patients with greater comorbidity burden.

Conclusions: Our initial evaluation suggests that documentation burden, as measured by word count, has declined over time, particularly following implementation of the POP in 2019. Additional research is needed to see whether the same occurs when examining other medical specialties, clinician types, and longer evaluation periods.

This content is limited to qualifying members.

Existing members, please login first.

If you have an existing account please login now to access this article or view your purchase options.

Purchase only this article ($15)

Create a free account, then purchase this article to download or access it online for 24 hours.

Purchase an SMJ online subscription ($75)

Create a free account, then purchase a subscription to get complete access to all articles for a full year.

Purchase a membership plan (fees vary)

Premium members can access all articles plus recieve many more benefits. View all membership plans and benefit packages.


1. Embi PJ, Weir C, Efthimiadis EN, et al. Computerized provider documentation: findings and implications of a multisite study of clinicians and administrators. J Am Med Inform Assoc 2013;20:718–726.
2. Ho YX, Gadd CS, Kohorst KL, et al. A qualitative analysis evaluating the purposes and practices of clinical documentation. Appl Clin Inform 2014;5:153–168.
3. Kroth PJ, Morioka-Douglas N, Veres S, et al. The electronic elephant in the room: physicians and the electronic health record. JAMIA Open 2018;1:49–56.
4. Kuhn T, Basch P, Barr M, et al. Medical Informatics Committee of the American College of Physicians. Clinical documentation in the 21st century: executive summary of a policy position paper from the American College of Physicians. Ann Intern Med 2015;162:301–303.
5. Colicchio TK, Cimino JJ. Clinicians’ reasoning as reflected in electronic clinical note-entry and reading/retrieval: a systematic review and qualitative synthesis. J Am Med Inform Assoc 2019;26:172–184.
6. Erickson SM, Rockwern B, Koltov M, et al. Putting patients first by reducing administrative tasks in health care: a position paper of the American College of Physicians. Ann Intern Med 2017;166:659–661.
7. Tutty MA, Carlasare LE, Lloyd S, et al. The complex case of EHRs: examining the factors impacting the EHR user experience. J Am Med Inform Assoc 2019;26:673–677.
8. Koopman RJ, Steege LMB, Moore JL, et al. Physician information needs and electronic health records (EHRs): time to reengineer the clinic note. J Am Board Fam Med 2015;28:316–323.
9. Cusack CM, Hripcsak G, Bloomrosen M, et al. The future state of clinical data capture and documentation: a report from AMIA’s 2011 Policy Meeting. J Am Med Inform Assoc 2013;20:134–140.
10. Moy AJ, Schwartz JM, Chen R, et al. Measurement of clinical documentation burden among physicians and nurses using electronic health records: a scoping review. J Am Med Inform Assoc 2021;28:998–1008.
11. Arndt BG, Beasley JW, Watkinson MD, et al. Tethered to the EHR: primary care physician workload assessment using EHR event log data and time-motion observations. Ann Fam Med 2017;15:419–426.
12. Rotenstein LS, Holmgren AJ, Downing NL, et al. Differences in total and after-hours electronic health record time across ambulatory specialties. JAMA Intern Med 2021;181:863–865.
13. Rotenstein LS, Holmgren JA, Downing NL, et al. Differences in clinician electronic health record use across adult and pediatric primary care specialties. JAMA Netw Open 2021;4:e2116375–e2116375.
14. Young RA, Bayles B, Hill JH, et al. Family physicians’ opinions on the primary care documentation, coding, and billing system: a qualitative study from the residency research network of Texas. Fam Med 2014;46:378–384.
15. Taylor KA, McQuilkin D, Hughes RG. Medical scribe impact on patient and provider experience. Mil Med 2019;184:388–393.
16. Sinsky C, Colligan L, Li L, et al. Allocation of physician time in ambulatory practice: a time and motion study in 4 specialties. Ann Intern Med 2016;165: 753–760.
17. Gardner RL, Cooper E, Haskell J, et al. Physician stress and burnout: the impact of health information technology. J Am Med Inform Assoc 2019;26: 106–114.
18. Hauer A, Waukau HJ, Welch P. Physician burnout in Wisconsin—an alarming trend affecting physician wellness. Wis Med J 2018;117:194–200.
19. Hilliard RW, Haskell J, Gardner RL. Are specific elements of electronic health record use associated with clinician burnout more than others? J Am Med Inform Assoc 2020;27:1401–1410.
20. Eschenroeder HC, Manzione LC, Adler-Milstein J, et al. Associations of physician burnout with organizational electronic health record support and after-hours charting. J Am Med Inform Assoc 2021;28:960–966.
21. Shanafelt TD, Boone S, Tan L, et al. Burnout and satisfaction with work-life balance among US physicians relative to the general US population. Arch Intern Med 2012;172:1377–1385.
22. Olson K, Sinsky C, Rinne ST, et al. Cross-sectional survey of workplace stressors associated with physician burnout measured by the Mini-Z and the Maslach Burnout Inventory. Stress Health 2019;35:157–175.
23. Holmes EG, Connolly A, Putnam KT, et al. Taking care of our own: a multispecialty study of resident and program director perspectives on contributors to burnout and potential interventions. Acad Psychiatry 2017; 41:159–166.
24. Shanafelt TD, Noseworthy JH. Executive leadership and physician wellbeing: nine organizational strategies to promote engagement and reduce burnout. Mayo Clin Proc 2017;92:129–146.
25. Williams ES, Konrad TR, Scheckler WE, et al. Understanding physicians’ intentions to withdraw from practice: the role of job satisfaction, job stress, mental and physical health. Health Care Manage Rev 2001;26:7–19.
26. Dewa CS, Loong D, Bonato S, et al. How does burnout affect physician productivity? A systematic literature review. BMC Health Serv Res 2014; 14:325.
27. Shanafelt TD, Balch CM, Bechamps G, et al. Burnout and medical errors among American surgeons. Ann Surg 2010;251:995–1000.
28. Salyers MP, Bonfils KA, Luther L, et al. The relationship between professional burnout and quality and safety in healthcare: a meta-analysis. J Gen Intern Med 2017;32:475–482.
29. Nguyen OT, Jenkins NJ, Khanna N, et al. A systematic review of contributing factors of and solutions to electronic health record-related impacts on physician well-being. J Am Med Inform Assoc 2021;28:974–984.
30. Office of the National Coordinator for Health Information Technology. Strategy on reducing regulatory and administrative burden relating to the use of health IT and EHRs. https://www.healthit.gov/sites/default/files/page/2020-02/BurdenReport_0.pdf. Accessed January 23, 2021.
31. Centers for Medicare & Medicaid Services. Patients Over Paperwork. https://www.cms.gov/About-CMS/story-page/patients-over-paperwork. Accessed January 23, 2021.
32. Accreditation Council for Graduate Medical Education. The ACGME’s approach to limit resident duty hours 12 months after implementation: a summary of achievements. https://www.acgme.org/globalassets/PFAssets/PublicationsPapers/dh_dutyhoursummary2003-04.pdf. Accessed October 29, 2021.
33. Centers for Medicare & Medicaid Services. Patients Over Paperwork. https://www.cms.gov/Outreach-and-Education/Outreach/Partnerships/Downloads/PoPDecember2017Newsletter.pdf. Accessed October 12, 2020.
34. Centers for Medicare & Medicaid Services. Final policy, payment, and quality provisions changes to the Medicare physician fee schedule for calendar year 2019. https://www.cms.gov/newsroom/fact-sheets/final-policy-payment-and-quality-provisions-changes-medicare-physician-fee-schedule-calendar-year. Published November 1, 2018. Accessed September 9, 2021.
35. Centers for Medicare & Medicaid Services 2019 Patients Over Paperwork newsletter. https://www.cms.gov/files/document/november-2019-patients-over-paperwork-newsletter.pdf. Accessed November 22, 2021.
36. Berenson RA, Basch P, Sussex A. Revisiting E&M visit guidelines—a missing piece of payment reform. N Engl J Med 2011;364:1892–1895.
37. von Elm E, Altman DG, Egger M, et al. Strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. BMJ 2007;335:806–808.
38. Barberio JA, Jenkins ML. Transitioning to telehealth: today’s guidelines for future sustainability. J Nurse Pract 2021;17:795–798.
39. Huang WY, Grigoryan L, Aggarwal A. Predictors of student use of an electronic record. Clin Teach 2019;16:131–137.
40. Gottlieb JD, Shapiro AH, Dunn A. The complexity of billing and paying for physician care. Health Aff 2018;37:619–626.
41. Dunn A, Gottlieb JD, Shapiro A, et al. A denial a day keeps the doctor away. https://www.nber.org/system/files/working_papers/w29010/w29010.pdf. Published July 2021. Accessed November 22, 2021.
42. Gupta K, Murray SG, Sarkar U, et al. Differences in ambulatory EHR use patterns for male vs. female physicians. https://catalyst.nejm.org/doi/full/10.1056/CAT.19.0690. Published November 13, 2019. Accessed November 26, 2019.
43. Akbar F, Mark G, Warton EM, et al. Physicians’ electronic inbox work patterns and factors associated with high inbox work duration. J Am Med Inform Assoc 2021;28:923–930.
44. Attipoe S, Huang Y, Schweikhart S, et al. Factors associated with electronic health record usage among primary care physicians after hours: retrospective cohort study. JMIR Hum Factors 2019;6:e13779.
45. Melnick ER, Ong SY, Fong A, et al. Characterizing physician EHR use with vendor derived data: a feasibility study and cross-sectional analysis. J Am Med Inform Assoc 2021;28:1383–1392.
46. van Walraven C, Austin PC, Jennings A, et al. A modification of the Elixhauser comorbidity measures into a point system for hospital death using administrative data. Med Care 2009;47:626–633.
47. Quan H, Sundararajan V, Halfon P, et al. Coding algorithms for defining comorbidities in ICD-9-CM and ICD-10 administrative data. Med Care 2005;43:1130–1139.
48. de Moissac D, Bowen S. Impact of language barriers on quality of care and patient safety for official language minority francophones in Canada. J Patient Exp 2019;6:24–32.
49. Buchinsky M. Recent advances in quantile regression models: a practical guideline for empirical research. J Hum Resour 1998;33:88–126.
50. Machado JAF, Santos Silva JMC. Quantiles via moments. J Econom 2019; 213:145–173.
51. Chen J, Vargas-Bustamante A, Mortensen K, et al. Using quantile regression to examine health care expenditures during the Great Recession. Health Serv Res 2014;49:705–730.
52. Zhao Y, Atun R, Anindya K, et al. Medical costs and out-of-pocket expenditures associated with multimorbidity in China: quantile regression analysis. BMJ Glob Health 2021;6:e004042.
53. Ortaglia A, Stansbury ML, Wirth MD, et al. Quantile differences in the age-related decline in cardiorespiratory fitness between sexes in adults without type 2 diabetes mellitus in the United States. Mayo Clin Proc Innov Qual Outcomes 2022;6:302–310.
54. Nguyen OT, Turner K, Apathy NC, et al. Primary care physicians’ electronic health record proficiency and efficiency behaviors and time interacting with electronic health records: a quantile regression analysis. J Am Med Inform Assoc 2022;29:461–471.
55. Bokolo AJ. Exploring the adoption of telemedicine and virtual software for care of outpatients during and after COVID-19 pandemic. Ir J Med Sci 2021;190:1–10.
56. Alexander GC, Tajanlangit M, Heyward J, et al. Use and content of primary care office-based vs telemedicine care visits during the COVID-19 pandemic in the US. JAMA Netw Open 2020;3:e2021476.
57. Sieja A, Markley K, Pell J, et al. Optimization sprints: improving clinician satisfaction and teamwork by rapidly reducing electronic health record burden. Mayo Clin Proc 2019;94:793–802.
58. Gluckman TJ, Vavricek JJ. Streamlining evaluation and management payment to reduce clinician burden. Circ Cardiovasc Qual Outcomes 2019;12:e005426.
59. Herzer KR, Pronovost PJ. Ensuring quality in the era of virtual care. JAMA 2021;325:429–430.
60. Hirschtick RE. A piece of my mind. Copy-and-paste. JAMA 2006;295: 2335–2336.
61. Goldstein IH, Hwang T, Gowrisankaran S, et al. Changes in electronic health record use time and documentation over the course of a decade. Ophthalmology 2019;126:783–791.
62. Moon MC, Hills R, Demiris G. Understanding optimisation processes of electronic health records (EHRs) in select leading hospitals: a qualitative study. BMJ Health Care Inform 2018;25:109–125.