| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
|
|
|
|||||||
|
|||||||||
From the Neurological Institute, Columbia University, New York.
Address correspondence and reprint requests to Dr. M.S.V. Elkind, Neurological Institute, Columbia University, 710 W. 168 St., New York, NY 10032; e-mail: mse13{at}columbia.edu
Swallow hard. If you are a neurology residency program director, plan to be one, or spend considerable amounts of time around one, you may feel that you are in for a rough ride over the next couple of years. New work hours restrictions imposed by the Accreditation Council on Graduate Medical Education (ACGME) have created significant strains on our system of providing neurologic education, leading to a fair amount of debate in the graduate medical education community. New ACGME requirements regarding teaching and evaluating core competencies provide another burden and challenge. The core competencies, moreover, are relevant not just to residency education, but to practicing neurologists, as they are to be incorporated into continuing medical education, board certification, and credentialing. But while the core competencies initially struck me as abstruse and burdensome, the difficulties they engender are, on reflection, readily surmountable. In my case, a helpful approach to the competencies began with the recognition that we were already doing most of the things the ACGME asked for, though we may not have labeled them as such.
 |
What are the core competencies? |
|---|
 Top. What are the core... Principles for painless...Specific methods of evaluating...Core competencies: Not just...AcknowledgmentReferences |
|---|
The first two of these, Patient Care and Medical Knowledge, fall within the traditional sphere of resident education, and most neurology residency programs already do a pretty good job of evaluating them: mock boards, Residency In-Service Training Exam, written evaluations, quizzing residents on rounds (also known as "pimping"), and other methods.
The latter four competencies appeared a lot more nebulous to me, however. How does one teach or evaluate Professionalism? What on earth do they mean by Systems-Based Practice? But after my initial frustration subsided, and I had survived my first couple of years of program directorship, I could see that our program was already teaching and evaluating these items implicitly, although most of our faculty members were not aware of it. A simple approach to addressing the core competencies, I realized, would be to make more explicit what we were already doing.
|
Principles for painless incorporation of core competencies into residency training |
|---|
|
Top.What are the core...Principles for painless...Specific methods of evaluating...Core competencies: Not just...AcknowledgmentReferences |
|---|
1. Minimize resident burden. The evaluation process should add minimally to the workload of individual residents, as residents are generally already extremely busy, and work hour limits have eroded any additional time available for completing logs, surveys, and other materials required to demonstrate competence. The newly introduced Neurology Residency Review Committee’s resident case log system could be an example of a minimally burdensome addition to the demonstration of competence, if, in fact, only 1 minute is required per patient entered into the system
2. Minimize administrative burden. The evaluation process should add minimally to the burden of the administrative staff (residency program directors and residency coordinators), as no support for additional educational and evaluative activities has been provided by residency funding sources. (It was around this time that "unfunded mandates" became one of my favorite expressions.)
3. Real-life situations. The evaluation process should utilize real-life clinical opportunities to the greatest extent possible, rather than introduce artificial elements. I believed that our program’s clinical load was large enough that it would be most efficient to utilize real patients in the process rather than, say, actors. This would limit the amount of additional time and work for residents, faculty, and administrative staff. In some instances, standardized materials or artificial situations may be of benefit (e.g., mock oral boards), but these should be used on a limited basis. For example, we perform mock oral boards annually.
4. The teachable moment. Use existing meetings, conferences, and committees when possible. Many of the competencies lend themselves to teachable moments that occur commonly in the course of routine resident gatherings and meetings. At our monthly resident meetings, issues related to practice and hospital systems frequently arise, such as medical record completion, hospital initiatives to improve documentation, and optimal ways to structure rotations to maximize efficiency. Residents and faculty may not recognize these as examples of resident education in the core competencies, but they can easily serve this function. By involving residents in committees focused on these issues, moreover, we also have occasion to evaluate their competence in these areas. For example, we recently formed a subcommittee of residents and faculty members to create a protocol that defines when junior residents must seek assistance from more senior residents or attendings. Active participation in this committee was an opportunity for residents to demonstrate competence in Practice-Based Learning and Improvement, Professionalism, and Systems-Based Practice. Morbidity and mortality conferences also provide obvious settings to address Practice-Based Learning and Improvement and Professionalism.
5. Maximize educational value. The evaluation process should seek to simultaneously maximize educational value as it evaluates. Mock oral boards, for example, when done properly with immediate feedback, often function as educational sessions as much as evaluative ones.
|
Specific methods of evaluating the core competencies |
|---|
|
Top.What are the core...Principles for painless...Specific methods of evaluating...Core competencies: Not just...AcknowledgmentReferences |
|---|
Detailed descriptions of these and other competency evaluation tools can be found on the ACGME Web site (www.acgme.org/outcome/assess/toolbox). Program directors and faculty members involved in education of residents will also gain a better appreciation of the methods for evaluation of core competencies that will be expected of them when the site visit comes around by reviewing the new on-line General Competencies Addendum to the Program Information Form. This addendum provides examples of several different methods of evaluation and asks for brief descriptions of up to eight different ways the program is evaluating the competencies. This document is a good example of why it is never too early to begin preparing for the next site visit!
Currently, it is expected that programs will begin to address the core competencies in their curricula and evaluation tools. There is no absolute requirement that programs do one thing or another, and there is no consensus on what works. Going forward, therefore, it will be important for the neurologic community, and in particular its organized groups of educators, such as chairpersons and program directors, to share their successes and failures at meetings and in publications. The ACGME now sponsors meetings addressing evaluation of competencies, and the American Academy of Neurology has sponsored similar courses, as well as less formal discussions, at its annual meeting. The Consortium of Neurology Program Directors is another resource for sharing experiences with these evaluation tools. By combining forces, it is hoped, programs may avoid duplicative efforts at testing methods.
Following are brief descriptions and personal comments regarding these three key methods.
1. Personal observation. Faculty document their observation of residents performing various tasks such as examining patients, doing procedures, interpreting radiologic studies, etc. These observations can be incorporated into the usual end-of-rotation global evaluations that most programs already have attending physicians complete.
2. 360° evaluations. These are evaluations not only by faculty, but also by others in a better position to observe certain aspects of a resident’s performance, including nurses, co-residents, patients, self, and others. Theoretically, by incorporating a broader spectrum of opinions, 360° evaluations better capture competence in Interpersonal Skills and Communication, Professionalism, and Systems-Based Practice. Evaluations collected from patients and busy primary nurses would create an undue burden on our administrative staff, however, since heroic efforts would likely be needed to retrieve the evaluations consistently. Attempts to validate such evaluations or to demonstrate their utility would be a major project, moreover, and one for which we do not have funding. While 360° evaluations have been found useful in the business community, experience in residency education remains limited to a few pilot projects,2–4 which have found some benefit but at tremendous cost.5 Methodologic issues also include the finding that sicker patients tend to give their physicians lower scores6 and the fact that patients may have language barriers and limited access to computers to enter results. Although systematic research into the optimal way to conduct these evaluations is ongoing, several questions remain unanswered: Who should evaluate? How often? What should be done with the information? Evaluators and instruments will also likely need to be different across specialties. What works in surgical or emergency medicine programs may not be relevant to neurology practice, either, with its greater emphasis on chronic care and cognitively impaired patients. For these reasons, no consensus has been reached on an approach to the 360° evaluation in neurology.
Nonetheless, in our program, incorporation of a limited number of additional evaluators with knowledge of different aspects of residents’ performance into the process has provided unique information and was accomplished with minimal effort. We have found it useful, for example, to include reports from our nurse–practitioners, who work closely with the residents and take sign-out from post-call residents every day. The nurse–practitioner evaluations include information about quality of sign-out (relevant to competence in Interpersonal Skills and Communication) and interactions with therapists and support staff (relevant to competence in Systems-Based Practice). Resident self-assessments have provided the opportunity for residents to reflect on their strengths and weaknesses and to focus their educational goals and objectives (relevant to competence in Professionalism). Based on the administrative cost of retrieving more extensive evaluations from other team members and patients, I believe it is reasonable to expect that proscriptive requirements for the use of more complete 360° evaluations (including patients, patients’ families, floor nurses, radiology technicians, and others) would follow completion of rigorous, systematic studies demonstrating the benefits to residency outcomes and competence. Until such studies are done, a reasonable compromise is for programs to incorporate into the evaluation system those individuals with whom the resident interacts that are most likely to provide obviously useful information.
3. Portfolios. Portfolios are resident compilations of experiences encountered during residency. These would include diagnoses encountered and procedures performed but may also include, as we have begun to do, other experiences including lectures given and conferences led, medical errors and responses to these, literature searches, research projects, papers written, books and papers read, major meetings attended, and even personal milestones achieved during residency. Participation in departmental or hospital committees would also fit in here. These experiences can be easily kept in a binder or, potentially, an on-line recording system.
It should be recognized, however, that case logs and other systems for listing experiences do not in themselves guarantee competence: Seeing 25 cases of Parkinson disease does not in itself ensure that a resident will care for those patients appropriately. Determination of competence in caring for these patients will ultimately depend on a combination of logs with the other methods, like observation and well-rounded evaluations. But logs can serve to provide evidence of an appropriate breadth of experience for residents, a minimum exposure to all the important areas of neurology. Those residents whose case logs demonstrate lacunae in experience (only a few cases of Parkinson disease during the first 2 years of residency, for instance) may need to be directed toward future rotations in which they can increase their exposure to certain disease conditions or procedures. Similarly, programs can use their residents’ logs in aggregate to evaluate the program’s overall breadth of experience; if no one in the program is seeing enough multiple sclerosis (MS), then the program may need to send residents to rotations at other institutions where MS is seen more often or introduce other ways of broadening their exposure.
|
Core competencies: Not just for residents anymore! |
|---|
|
Top.What are the core...Principles for painless...Specific methods of evaluating...Core competencies: Not just...AcknowledgmentReferences |
|---|
And, remember, that which does not kill us can only make us stronger!
|
Acknowledgment |
|---|
|
Top.What are the core...Principles for painless...Specific methods of evaluating...Core competencies: Not just...AcknowledgmentReferences |
|---|
Received July 18, 2004. Accepted in final form September 20, 2004.
|
References |
|---|
|
Top.What are the core...Principles for painless...Specific methods of evaluating...Core competencies: Not just...AcknowledgmentReferences |
|---|
This article has been cited by other articles:
|
|
 |
|
  E. Delphin and M. Davidson Teaching and Evaluating Group Competency in Systems-Based Practice in Anesthesiology Anesth. Analg., June 1, 2008; 106(6): 1837 - 1843. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | HELP | FEEDBACK | SUBSCRIPTIONS | ARCHIVE | SEARCH | TABLE OF CONTENTS |
