This section will review the ACGME self-study process. The objectives of this module are to learn the rationale for the self-study and timeline for program assessment, review the components of the self-study and the stakeholders involved in the process, and describe how to complete the online self-study form. The self-study is a comprehensive description of how the program provides an effective learning and working environment that leads to improved educational outcomes and identifies strengths, weaknesses, and plans for improvement. On this slide is a timeline of the self-study relative to the 10-year ACGME site visit. Year 1, the ACGME notifies the training program to initiate the self-study. Approximately 12 to 18 months after the self-study is completed, the ACGME will conduct a site visit. Throughout the 10-year period between each self-study and site visit, each training program must also complete an annual program evaluation with ACGME review. Programs should initiate the self-study process at least 3 to 4 months prior to the self-study deadline to allow ample time to assess the program, develop strategies for improvement, and review the final plans. The 8-step process includes 1. Assemble a self-study group 2. Discussion of program aims 3. Create a longitudinal assessment of program strengths and areas for improvement 4. Examine opportunities and threats 5. Obtain stakeholder input 6. Interpret data and aggregate findings 7. Discuss findings with stakeholders 8. Write and submit the succinct self-study document. Step 1 is to assemble the self-study group. The members of this group can be program-specific, but important groups to consider include the Program Evaluation Committee, the Program Coordinator, Department and Division Chiefs, Rotation Directors, Chief Fellows if applicable, current and prior trainees, and Core Faculty. In some programs, input from nursing, administration, and leaders from other specialties that frequently interact with your trainees may be valuable. Once the self-study is assembled, their first task is to review the program aims. The aims should define the key expectations for that program and describe how the program differentiates itself from other programs in that same specialty. The aims may describe the types of trainees in the program, how it prepares trainees for particular careers, and how the trainees will ultimately serve the community and health care system. Importantly, the aims should be measurable and achievable, reflect the careers of prior trainees, and align with the community being served. In most instances, the program aims will be vetted by departmental and institutional leadership in order to align program aims with the mission of the institution. Step 3 is to gather information about your program's strengths and identify areas that may need improvement. This information may come from a variety of sources, but should include data from the ACGME Annual Program Update, ACGME Surveys, and in-training examination and board certification results. Information collected from internal program surveys, including the opinions and recommendations from prior graduates, may provide useful insight into potential changes needed in your program. Also consider institutional data such as patient satisfaction surveys, national rankings, and research funding as part of this evaluation. As part of this assessment, address any active citations identified at prior reviews or areas that may not be in compliance with ACGME requirements. Remember that the focus of any improvement should be in line with your program aims. Here is an example of how a program might work through an area in need of improvement. First, your annual program survey notes that graduating fellows do not feel prepared to care for patients with common adult congenital heart disease. Second, your program develops and implements curricular changes that will address this problem. Third, you reassess your trainees' knowledge of this topic and comfort with caring for these patients through in-training examination and board examination results, as well as follow-up surveys. Step four is an assessment of the training environment to identify potential opportunities and threats to the program and help identify factors that may affect the success of the program's strategy for improvement. Opportunities should focus on external factors that are not under the control of the program, but may help it flourish. Such opportunities may be related to the patient population you serve, new technology that may improve patient care, or trainee education and collaborative efforts with other departments and institutions. In a similar manner, threats to the program's growth and improvement should be identified. Duty hour restriction is an example of a change outside the control of a program that may have both positive and negative effects on the educational efforts of a program. In our prior example, the program identified that trainees felt unprepared to care for patients with adult congenital heart disease, and in response, the training program developed a new curriculum to address this deficit. The duty hour restriction provides an opportunity for this change, as reducing work hours may result in less trainee fatigue and provide more time for self-learning. The threat is that there will be less time for patient care, and the potential increased training costs may limit curriculum changes. Step five is to summarize the data and provide the stakeholders an opportunity to critique the findings and offer additional observations and suggestions for improvement. This is also a time to have individuals not included in the self-study review, in the self-study review the findings. Such individuals could include program directors from other specialties, departmental and hospital leadership, and institutional GME members. The data can be presented in whatever format is most suitable for your institution, and can include electronic documents, individual and group meetings, or retreats. The next step is to interpret the data and develop an action plan for program improvement using the results from the self-study. This plan should confirm the aims, list the program's key strengths, prioritize areas for improvement and identify those in which an action plan has been developed, review opportunities and identify methods to utilize these to improve the program, review threats and develop plans to minimize their effects on your program, and develop a plan for the next five years that will elevate the program to the next level. As noted in the previous slides, the action items should be measurable, although it is not expected that the program will have definitive results by the time of the site visit. The final step prior to submitting the self-study document is reviewing the key findings and plan with the stakeholders at your institution, including the division and department chairs, internal medicine program director, and the DIO. The final step is to complete the self-study summary that will document the self-study process and key findings. The program's strengths and plans for improvement are not included in the summary so that the program has time to implement changes prior to the site visit. However, all data and plans collected during the self-study should be documented to serve as a guide for program assessment improvement to be used for the annual program update and to provide data for the ACGME site visit. Now that the self-study is completed, what are the next steps? Ideally, one individual or group of individuals should be assigned to track data for each area of improvement. All changes and measures of improvement should be documented. At the time of the site visit, the program should be prepared to review its strengths and provide an update on improvements made to the program since the self-study had been completed. If you would like more information about the self-study process, please see the below resources on the self-study provided by the ACGME. I hope you have found this module helpful.

ACGME

self-study process

objectives

program assessment

site visit