Hello, my name is Jim Arrigui, and I'd like to welcome you to the Training Directors Program. This program has been put together by the American College of Cardiology with the Cardiovascular Training Leadership Council and Section. This module will cover ACGME structure and accreditation process, and will be part of a larger module that covers issues related to accreditation of our fellowship programs. The objectives of this session are to describe what is accreditation as it pertains to a fellowship program in cardiology, how does the process unfold annually, what accreditation status can my program have, and what are my responsibilities as a program director. Note that program requirements will be discussed in the next module. So what is accreditation? First, it is important to note that it is not certification. Individuals rather than programs receive certification through the certification boards, such as the American Board of Internal Medicine. In contrast, programs and institutions receive accreditation through the ACGME in a process that assesses compliance with peer-derived standards, or the so-called program requirements. This distinction is important. In general, if a program director has a question about an individual, say this individual has deviated from the usual training pathway, or perhaps it's a question about prior credit for training, then this question would go to the certification boards, not the ACGME. If, however, you have a question about structure of a program, or resources, or other things that affect the program overall, then this question would be appropriate for ACGME. It may be helpful to review the overall structure of the ACGME as it pertains to accreditation. The ACGME Board of Directors is the body that is charged with overall operation of ACGME. The board sets major policy, delegates accreditation authority to 29 review committees, and is responsible to the public to guarantee safety in the GME environment. The Review Committee in Internal Medicine is one of the many review committees. It is a committee of our peers. Its members are on the committee after being nominated by the American Board of Internal Medicine, the American Medical Association, the American Osteopathic Association, or the American College of Physicians. Membership includes residents, or fellows, and a public member, and the Review Committee functionally makes all the accreditation decisions. Together, these two entities ensure that the ACGME mission is carried out. This mission is to improve healthcare and population health by assessing and advancing the quality of resident physicians' education through accreditation. This slide is a graphical representation of a typical Review Committee in Internal Medicine membership. The committee is the largest of the ACGME Review Committees and is responsible for accrediting all the programs in Internal Medicine and its subspecialties, well over 2,000 programs. The committee has a large representation from program directors or ex-program directors represented by the stars, as well as DIOs, and there is a good mix of individuals from both general internal medicine and from the subspecialties. The committee typically includes two cardiologists. So what is the process of accreditation? It is most important to remember that the accreditation cycle is an annual cycle in the new or next accreditation system. The committee utilizes an assessment of data that occurs annually, and I will describe this data in subsequent slides. Based on screening this data or analyzing it further by Review Committee staff or members of the Review Committee, the ACGME will identify potential areas for improvement or identify rules violations for each of the programs under its charge. The committee is then charged with trying to figure out what the problem is and making a diagnosis through analysis of this data, or in some cases, a site visit. To date, the site visit option has only been used for serious problems. These site visits can either be focused site visits which are focused on a very specific problem or full site visits if the problem is felt to be very serious. However, in most cases, analysis of the data reveals clues to a problem, and this information will be fed back to the program in the form of an area for improvement which will be described subsequently, and the committee will then, in subsequent annual reviews or potentially through some type of a progress report, want to confirm a fix, and this cycle occurs annually and continuously. The center of the cycle is the concept that we want to promote in our programs with the ACGME both innovation and continuous improvement, and really this cycle is meant to drive continuous improvement. What data are used by the Review Committee in its annual assessments? First, ACGME surveys of fellows and faculty are one of the most significant pieces of data that the ACGME uses. These surveys typically come out in the mid to late winter or early spring. Individuals are surveyed, and the program gets feedback from these surveys sometime around May. Secondly, ABIM pass rates are used as a gross metric of program quality. Thirdly, data that are inputted into the online ADS system, which will be described subsequently, and includes information on your faculty, scholarly activity, attrition rates, major changes, responses to citations if applicable, and of course the committee will look for any data that are omitted. Together, this ADS data represents a significant amount of information that the committee can use to assess overall program quality and compliance with standards. The program director is responsible for all of this data, but particularly the ADS data entry, and specific responsibilities of the program director are defined in Section 2 of the Common Program Requirements. One of the most important points I'd like to make in this presentation is that the program director should check the ACGME website regularly. Any information presented here may change, and the ACGME is constantly refining its processes. The website front page includes major tabs for designated institutional officials, program directors, or the residents and fellows themselves. There is a major section for program directors, for example, where the program director's virtual handbook with a lot of important information for program directors is housed. And of course, there is the page that is specific to the Review Committee for Internal Medicine that includes all the program requirements for both the residencies and the fellowships in Internal Medicine, and importantly, includes the contact information for the staff members of the Review Committee. I'd also like to make the important point that if a program director has a question about the program requirements, or perhaps a letter that it receives from ACGME, or any ACGME process, the staff of the Review Committee are happy to field calls. You do not have to feel afraid to contact the ACGME. They are not going to tag you for being potentially a problem. They want to help you do things right the first time. Let's talk more about the accreditation data system, since this is a key piece of the data that the ACGME uses for accreditation purposes, and it is the one thing that a program director can control. ADS used to be ignored by program directors in the past because it was a document that only needed to be updated every few years. But at this point, it is very important that the program director oversees the data that's being input into ADS on a fairly frequent basis. Again, this system contains a significant amount of data reviewed by the committee, and you want it to be accurate. It is important to consider ADS a live document. That is, it is accessible literally any time, and you could put updates into it any time. If, for example, one faculty member leaves and two are recruited, you could have your program director enter that information when it happens, rather than wait at the end of the academic year and have to do everything at once. The official update to ADS is typically due in July and August. The main purpose of this update is to define your fellow and faculty rosters for that upcoming academic year, but you also want to make sure that other sections of ADS are accurate. This annual update window will be indicated on ADS in the upper right-hand corner of your page when you sign in and look at your program information. However, an important tip is that if you have significant changes to your program, such as major structural changes, sites being added or taken away, major faculty shifts, or if there are issues on your ACGME survey that you will have just received in May, I strongly recommend that you update ADS in June, before the end of the academic year. This will ensure that if you have any information that you want to provide to the review committee to explain potential issues, that you can do so and that the committee will be able to look at it and act accordingly, perhaps having information that may prevent you from getting a citation or an area for improvement if they know that you're acting proactively. ADS has the following components, which are all indicated by tabs on the front page of your program. Overview, which includes entry points for milestone information and self-study uploads. Program information, describes the number of positions you have, the accreditation status, and other information. Faculty roster, including scholarship. Fellow roster, including scholarship. Participating sites. Surveys. Milestones information. At present, case logs are not used by the review committee for internal medicine. Summary tab is where you can print or save a PDF of your whole program information file. And a tab in which you could print various reports. If your program has received citations or areas for improvement in its most recent letter of notification from ACGME, there are several important points to make as it relates to data entry into ADS. First, if you have citations, a response is required in ADS. The citation will be listed and there will be a specific place to enter a response. By virtue of the fact that you have a citation, the RRC will review your response to the citation and make determinations about whether to resolve the citation based on your response, also incorporating any other data it has. But a citation will be reviewed by a member of the committee until it is resolved. If you have an area for improvement, which is the primary mechanism for the RRC to provide feedback on your program, you should address the issue raised in the AFI. Although a formal response in ADS is not required for the area for improvement and there will not be a separate specific space for it, I recommend that you do the following. ADS contains a section called Major Changes and Updates, which is a free text section, and it is really your chance to communicate anything you wish to the RRC. This is where I recommend placing some information about how you are dealing with the area for improvement if you have received one. Major Changes may include changes to rotations, participating sites, block schedules. It may include changes to faculty or division leadership. But apart from the Major Changes, this updates section may, and in my opinion, should be used to communicate to the review committee what your plan to address areas for improvement is. And it can also be used to tell the committee what you were doing if you had major issues identified on the most recent ACGME survey. This is a very important point. If you see an area of a particularly low score on the ACGME survey when you get it in May, it is important that you quickly begin a process to determine whether or not you have an issue in the program. And come June, you can tell the ACGME that you noted this and that you're initiating this process. You may not be very far along in determining whether there's any issue, but the mere act of telling the ACGME that you noticed something was off in the survey and that you are acting may have an impact on the committee's accreditation decision. I would like to make a couple of comments about the faculty roster section of ADS. First of all, it is important to make sure that the information here is accurate and complete. Make sure that you have the proper degree of your faculty member listed, the certification or recertification information, medical license information, and remember things like the certification and medical license may have expiration dates and need to be updated, and scholarly activity, which has its own section attached to that roster. A few tips for defining core faculty. First, know the minimum number of faculty required for your program. Typically, there's an absolute minimum or you need to have at least one faculty member for every 1.5 fellows, but this is defined in the program requirements, and again, over the years may change. Secondly, add additional faculty beyond the minimum based on role in the program. For example, educational leaders, perhaps being members of the clinical competency committee or program evaluation committee, et cetera, could be considered for inclusion, rotation directors, and those faculty with major scholarship contributions. That said, you don't need a huge list in excess of your minimum faculty. I would recommend that there's no need to add more than double the faculty requirement. It just adds more work and will not be for any particular benefit. These faculty members that are defined in this roster will be the ones that get the faculty survey, and they'll be the faculty that the ACGME RRC will be looking at when it's looking for issues related to compliance with the standards. Let's now discuss how all this data are used and ultimately lead to an accreditation decision. This slide depicts a typical academic year. The top half of the slide is information that you see as the program director and things that happen within your program on an annual cycle. For example, throughout the entirety of the academic year, ADS can be updated, as I previously discussed. The update may be informed by an annual program evaluation, or APE, which typically would be done towards the end of the academic year. It may be helpful to link these two processes since you will get valuable information about your program from the annual program evaluation that may be then entered into ADS if relevant. This isn't a requirement, but it may be a best practice. As will be discussed in other formats, another major aspect of data entry from the program director is milestones reporting, which occurs in the middle and at the end of the academic year. Of course, fellow and faculty surveys are typically done in the late winter or early spring, and you will get that board pass rate data annually from the ABIM, which will be entered into ADS. These are all things that happen within the year in your program. The bottom half of the slide depicts what the Residency Review Committee does with this information and when. At the very beginning of the academic year, the committee and its staff are mainly analyzing the data that I previously mentioned. This is primarily a screening process, but for a small number of programs, the screening process may then lead to a more detailed analysis of your program, either by staff or a member of the review committee. Then the review committee will meet. These are just typical dates that may occur. One meeting will be in winter. One meeting may be in the late winter or early spring. The data are reviewed by the committee during these meetings. Most programs, within that first meeting in the winter, receive their accreditation status, since most programs are in a favorable status. You will receive a letter from the ACGME within 60 days of that meeting, but you will receive an email notification typically within about a week of your accreditation status. If for some reason the committee had required further information from your program or perhaps a site visit, or your program is in a different accreditation status that requires more scrutiny, you may not be reviewed until the spring, and you'll get the letter shortly thereafter. The process of the accreditation decision varies, in part, by what your accreditation status is coming in to the process on any given year. If your program is on warning or probation, you will necessarily be reviewed by a member of the review committee and the full committee, with the hope that they review the current data and can remove that warning or probation. If you have previous citations, you will necessarily be reviewed by a member of the review committee. A citation can only be granted by one of your peers on the review committee, and it can be only taken away by a member of that committee. So the committee is charged with review of all programs with citations, with the hope that the program has addressed the citations and that they can be resolved on any particular year. So citations shouldn't linger if you're addressing them. If your program, although it may not have any citations, may have some major potential issues on the annual data screen, maybe a very low board pass rate and a survey that looks bad in four categories, that may precipitate an internal review of your program's data by a member of the review committee or staff. They may even request additional information. But if none of these things occur, then your program passes all screening metrics and you will be placed on a consent agenda at the next RC meeting and receive continued accreditation. A couple of points about the accreditation status schema. If you are a fully accredited program, then the options for accreditation decisions are either that you would continue continued accreditation, or you may, if you have major issues, go on continued accreditation with warning. A site visit is not necessary to go on warning, although it often occurs. If a program ultimately receives probationary accreditation or withdrawal of accreditation, the two most serious categories, this will not occur without a site visit first. The committee cannot grant these statuses without first visiting the program with a site visit. Once a site visit occurs, any accreditation option is possible, and of course you would hope that once the site visitor comes, they confirm no problem and you receive continued accreditation. If you're one of the programs on initial accreditation status, then you will receive a site visit within two years of your initial accreditation, and at that point, a range of possible accreditation actions occur, either continued accreditation, continued accreditation without outcomes if you still haven't recruited fellows yet, you may have a warning status or a withdrawal of accreditation if there are serious problems. I noted on previous slides that the accreditation system is meant to promote continuous improvement of the program, and the graphical representation of this is shown in this slide. Every year, data are submitted to the ACGME, the ACGME gives the program feedback if there are any issues, internally the program is doing its annual program evaluations, and ideally from those evaluations, formulating annual written action plans for improvement, tracking these plans, and as the years go on, that system should promote continuous improvement in the program structure and mission and outcomes. The culmination of this process is in a 10-year cycle capped by a self-study process, which will be described in the next slide, and shortly after the self-study process, a full site visit will be done for compliance purposes. The self-study process at ACGME is still rather new and evolving. I would point you to the instructions on the ACGME website as the most up-to-date source of information, and it includes not just information about the self-study process, but also the 10-year compliance site visit and forms that may be required for the self-study process. But again, this process, particularly for fellowships, is still evolving. If you are due for a self-study, I recommend two major things. First of all, start early. And again, the date for the self-study should be indicated on ADS, although you will receive official notification of when the process will actually be due well before it is due. And secondly, engage your internal medicine program director. Most informed internal medicine program directors will take command of the process within their departments, since the entire department, all the fellowships, and the residency will undergo the self-study process at the same time. And ideally, they should be linked. A full site visit will occur about 18 months after the self-study. I think the current range is between 18 and 24 months. The site visit will assess compliance with the program requirements and will be used for an accreditation decision. At the moment, the self-study, which will be documented in a summary document, is not currently used for accreditation decision, but rather as a QIA process. The only way that the self-study could affect accreditation decisions at present is if you don't do it. So I would urge you to do it as instructed by the ACJME and your program directors, your internal medicine program director. Finally, what is your role as a program director? You probably have a pretty good idea based on hearing some of the information in the previous slides. But first, with regard to the reporting structure and the support that you may have within your institution, I urge you to stay engaged with both your internal medicine program director and, as needed, the DIO or director of graduate medical education at your institution. These are valuable resources to a typical cardiology program director for information, especially if you are just starting as a program director. Secondly, ADS, keep it accurate. While a lot of the entry can be done by your program coordinator, you are responsible for the accuracy of this information. And as I stated before, I would recommend treating it as a live document with periodic updates rather than just looking at it once or twice a year. Thirdly, be attentive and responsive to ACJME surveys. You want to stay ahead of the curve. If you see that there's a trend downward in one of the major areas on the survey or a very precipitous drop, you, of course, will want to address this in your program. And it may be nothing of importance, but you have to determine what the problem is if there is one. And as I mentioned in previous slides, you should communicate that with the ACJME and be upfront about it. Fourth, ensure an effective evaluation process. And by that, I mean the processes of fellow evaluation, faculty evaluation, program evaluation. The self-study process could be considered a form of a program evaluation and strategic assessment. An effective evaluation system is the foundation of any solid program. So it could help prevent problems in the program overall. And finally, I would urge you to become a member of the ACC program director community. Through engagement with the program director's group, with the website, with programs for education like this, the program director community in cardiology is robust, full of talent, and those that are more experienced are always willing to help more junior colleagues. On that note, I'd like to thank you for listening and encourage you to view subsequent modules in this series. Thank you.

ACGME accreditation process

fellowship programs

cardiology

certification

accreditation

Review Committee in Internal Medicine