feature
Physician Training
New Tools Measure Competency
BY ANDREW RABINOWITZ, M.D.

In recent years, forward-thinking members of the medical community have turned their attention to improving the training of the next generation of doctors. The first wave of baby boomers turned 60 this year. This huge cohort of aging Americans, with their increased needs for physician services, will put large additional demands on the nation's already strained medical system.

Will our doctors be able to meet this challenge? In this article, I'll discuss the concerns that we all should share about the way this country has been developing new physicians and discuss some specific new ideas for improving the training of ophthalmologists.

Flaws in Current Training

Economic factors have influenced the landscape of medical training over the past quarter century. Concerns exist as to whether the current academic system can produce physicians and surgeons with adequate "competence" at the completion of their training.

The classic model of residency and fellowship training appears to be showing signs of strain. Government funding for teaching hospitals is waning. Intrusions from the legal system have placed additional stress upon the academic community. Additional factors that are influencing the medical landscape are expectations of both the managed care market and the patients themselves.

The only constant influx of economic support earmarked toward physician training has come from pharmaceutical and medical device companies. Industry has shown unwavering support of academic medicine from medical school through fellowship training. Although questions about the financial motivations of these companies have been raised, it is clear that at the core industry is truly dedicated to improving quality of life for patients of all ages and all socio-economic groups.

One brief positive example of this commitment, specific to ophthalmology, is the annual Fellows Forum for second-year vitreoretinal fellows. This event brings together attendees from all over the country for a weekend to meet their peers, hear from companies that provide vital products and services and receive professional advice from leading retinal specialists. This year's Fellows Forum had a major sponsor in Bausch & Lomb, along with 13 additional sponsors representing a cross-section of the ophthalmic industry. This type of cooperative effort that produces tangible benefits stands as one of the bright spots in the overall current landscape of physician development and training.

Concerns are Longstanding

As far back as a century ago, the quality of physician training was of great national concern. In 1908, as a response to this concern, Derrick Vail, M.D., embarked upon an effort to improve the quality of ophthalmologic care in the United States. Dr. Vail was especially concerned with raising the standards for physician education. The Residency Review Committee (RRC) was formed as a consequence of Dr. Vail's efforts. Alfredo A. Sadun, M.D., described the genesis of the RRC in an article published in November of 2000.1 At its inception, the RRC was responsible for ensuring high standards for residency training. Additionally, the RRC remains concerned with protecting the interests of the public to whom the residents will eventually deliver care. The RRC is composed of 10 members: three from each of the three parent organizations — The American Academy of Ophthalmology (AAO), The American Board of Ophthalmology (ABO) and The American Medical Association (AMA), as well as a current resident in ophthalmology. Nearly 60 residency programs are evaluated by the RRC every year.

Subsequent to the formation of the RRC, the American Board of Medical Specialties (ABMS) was formed in 1933. The ABMS continues to maintain an active role in the training and certification of American physicians.

More recently, the needs for standardization among all residency programs were addressed by the formation of the Accreditation Council for Graduate Medical Education (ACGME) The ACGME, which was created in 1980, is a private, professional organization that is responsible for the accreditation of 7,800 residency education programs throughout the nation.

What Constitutes Competency?

Defining physician competence is a challenging task. Once defined, competency is even more difficult to measure. Together, the ABMS and the ACGME have developed a series of mandatory "competencies" that all training physicians must achieve to successfully complete their education. Andrew G. Lee, M.D., published an informative article on the impact of these competencies.2 The goal of the ACGME is to help steer the medical education entities towards "competence-based medical training." The ACGME has created a list of six specific competencies necessary for the development of future physicians. These competencies include

► Patient care
► Medical knowledge
► Practice-based learning and improvement
► Interpersonal and communication skills
► Professionalism
► System-based practice

Currently, some momentum exists in the medical community toward generating a seventh competency for surgical skills.

Perhaps the most proactive and influential organization dedicated to the promotion of medical education, research and patient care in ophthalmology is the Association of University Professors of Ophthalmology (AUPO). The role of the AUPO, as well as its interaction with other educational governing bodies, was eloquently described by Thomas J. Liesegang, M.D., and H. Dunbar Hoskins, M.D., in their landmark paper on the status of ophthalmic education; past, present and future. Their work was published in the American Journal of Ophthalmology in July 2003.3 This informative and enlightening article provides a wealth of information regarding ophthalmic education.

As discussed in their article, the AUPO was formed in 1967 by the joint collaboration of academic ophthalmologic department chairs. The AUPO was designed to serve as a vehicle for academic units within ophthalmology to share issues, concerns and approaches to education, research program development and department management for the betterment of all ophthalmology.3 Today, the AUPO serves as a forum for discussion of problems and development of mutual interests and concerns among heads of departments of ophthalmology in medical schools in the United States and Canada, and in other institutions sponsoring ophthalmology residency training programs accredited by the ACGME.3

Thus, the efforts of the AAO, AUPO, ACGME, ABMS and RRC are focused on developing "competent" physicians and surgeons. Despite the continued efforts of these organizations, economic obstacles have made the implementation of their plans quite challenging.

Residency Programs Struggle

Maintaining high-quality residency education is challenging on many fronts. The amount of medical knowledge doubles every 4 years. Residency programs are forced to add this new information and transfer an ever-growing skillset to an already robust curriculum. The transfer of information and skills to residents must transpire during a brief 36-month period. Additionally, program directors must provide more formal documentation for internal and extramural organizations such as the RRC, ABMS and ACGME.

Academic institutions have been forced to survive and try to thrive despite decreasing economic support from federal and state government. During the early 1970s, medical economists were scrutinizing the relatively unlimited financial support to training hospitals. By the mid 1980s, federal and state funds were cut drastically. The decline in funding to teaching hospitals prevented these institutions from hiring the best and brightest instructors. Academic institutions that only a decade earlier were viewed of as ivory towers were struggling to attract full-time teaching physicians.

In addition to direct funding, academic institutions also received federal funding in the form of research grants. These research grants were used to expand departments and attract thought-leaders to major teaching hospitals. Funding for these research grants has declined precipitously over the past quarter century. This decline has further strained the funds available to academic institutions. The lack of federally funded research grants has dramatically reduced the magnitude of medical and surgical research performed at teaching institutions. This is an area in which the pharmaceutical industry has proven to be an invaluable source of continued financial support. Without the sponsorship of pharmaceutical companies, and medical device makers, scientific research would grind to a standstill. This is another arena in which the pharmaceutical industry has helped keep medical education afloat.

In summary, the first major challenge to teaching hospitals was coping with a profound decrease in both direct and indirect government funding. The end result of this decline in funding was a brain drain away from academic medicine and toward private practice settings.

Evidence of the brain-drain can be seen in the difficulty which institutions are facing in hiring and retaining residency directors and full-time teaching staff. Twenty years ago, the average tenure for a residency director was nearly 7 years. Currently, the average tenure has dropped to slightly more than 2.5 years. In 1998, approximately 12% of all program directors in ophthalmology were reassigned; in 1999, this number was approximately 25.5%. In 2000, the rate of reassignment approached 33%. This alarming lack of continuity challenges the long-term success of training future physicians.

Migration from Teaching Hospitals

Another challenge to academic institutions stems from the migration of patients away from teaching institutions toward private practices. This migration was augmented by the growth of group-style private practices. The one-physician private practice model has given way to the multiple-physician group practice model. These large groups have advantages over small practices when it comes to bidding on contracts with managed care organizations and third-party payers. Multiple-physician groups have evolved to become models of both efficiency and productivity. Large private practices can offer higher salaries to the best and brightest physicians because they attract an expanding patient base. Their ability to competitively bid on managed care contracts was born out of their ability to achieve economies of scale.

Within surgical subspecialties such as ophthalmology and orthopedics, group practices gradually became completely independent of the hospital-based healthcare system. Many ophthalmology practices built their own for-profit ambulatory surgery centers (ASCs). The number of surgical cases performed in hospital settings has dropped dramatically over the past 25 years.

As surgical cases migrated away from teaching hospitals, so did the financial remuneration these cases brought to these institutions. Teaching hospitals were unable to pay their physicians competitive salaries. The erosion of the patient census at teaching hospitals had profound financial ramifications. A major consequence of the eroding patient base was the decline in number of teaching or resident cases. The loss of patients to the private sector was a major market shift that further challenged the success and survival of academic medicine. Medical students, residents and fellows suffered on two fronts. First, they no longer had the best physicians as teachers. Secondly, the trainees did not have an adequate volume of surgical patients to care for during their academic careers. Residents and fellows are now finishing their training having performed fewer cases than their colleagues who graduated from similar programs in previous years.

Another factor which has reduced the number of resident surgical cases stems from the legal consequences of sub-optimal surgical outcomes. The cost of ending-up on the losing side of a legal battle has impacted how attending physicians participate in the care of patients in teaching hospitals. In past decades, attending physicians would serve as the supervising physician and the resident or fellow would be the treating physician, especially with regard to surgical care. Currently, attending physicians are less likely to allow residents or fellows to act as the primary surgeon.

Residents historically worked under the credo of "See one, do one, teach one." This saying becomes less viable as large teaching institutions become risk-aversive in a legal regard. Additionally, physicians at teaching hospitals are forming corporate entities. As such, they are often competing with private practices for large managed care contracts. To maintain their viability as participants on the panels of these managed care plans, attending physicians at teaching hospitals are more inclined to perform surgical cases on their own as opposed to putting their patients in the hands of residents and fellows.

The aforementioned stressors have challenged the ability of training programs to produce properly educated, trained and surgically proficient doctors.

Deficiencies Becoming Apparent

The consequences of these challenges to academic medicine are now beginning to play out as newly minted physicians attempt to integrate themselves into the real world.

Private practices are currently trying to hire new doctors. When a private practice attempts to hire a newly trained physician, it is less likely to find a physician with the clinical, cognitive and/or surgical experience necessary to function in an unsupervised, autonomous role than in decades past.

When a practice attempts to recruit a newly trained doctor, it often will contact teachers and preceptors who are familiar with the candidate. The teacher or preceptor may be limited in the degree of candor he or she can exercise when discussing a graduate. This limit stems from the fact that the legal system can penalize a teacher for expressing negative sentiment. Most evaluations are rendered as neutral or positive. An evaluation that expresses negative sentiment can put the evaluator at risk for legal pursuance. The hiring practice may not be able to obtain an accurate impression of a potential candidate. The prospect of hiring a candidate who may not have had adequate surgical experience, along with the inability to obtain an honest evaluation, places the hiring party at risk for hiring an inappropriate candidate for the open position.

Dealing with the Problem

The governing bodies responsible for training our future physicians are becoming increasingly aware of these issues and are attempting to prioritize the core information and skills that need to be transferred to our residents. Key ophthalmologic organizations, such as AUPO and the AAO, are rethinking and redesigning the ophthalmology residency education curriculum. The ABO, in conjunction with the ACGME, is attempting to identify the core competencies that every ophthalmologist must exhibit.

Creating standards and tools for assessing residents and fellows is a daunting task. This task is made even more ominous by the fact that the standard ophthalmology residency currently covers 36 months. The amount of new information both within medicine itself as well as the business of becoming a physician may force the training period to be expanded beyond the historic 36-month residency period.

The ABO has created a task force on resident competencies. Richard P. Mills, M.D., and Mark J. Mannis, M.D., published the "Report of the American Board of Ophthalmology Task Force on the Competencies" in July 2004.4 This report discusses strategies, tools and approaches to achieving a competence-based teaching paradigm.

The first proposed tool developed to assess the core residency competencies identified by ACGME is known as the Ophthalmic Clinical Evaluation Exercises (OCEX.) OCEX is a tool designed to assess the ophthalmology resident's competence in patient care. A superbly constructed study by Karl C. Golnik, M.D., describes the rationale used to construct OCEX. Dr. Golnik's work was initially presented at the Annual Academy of Ophthalmology Meeting in October 2005. The companion paper was published in Ophthalmology in the October 2005 issue.5

The OCEX has been shown to have face and content validity. It is a one-page form that the attending physician completes as he or she observes a resident taking the history of a patient and performing a physical exam on that patient. The resident presents the case to the attending and then provides an assessment and plan. The attending completes the OCEX during the observation and then immediately provides formative feedback to the resident.5 In a subsequent publication, the OCEX was shown to also have inter-rater reliability.5 Thus, the OCEX meets the criteria mandated by the ACGME that assessment tools are both valid and reliable.

New Tools: OASIS and GRASIS

Tools for evaluating the surgical competence of a resident or fellow are also being developed. Objective Assessment of Skills in Intraocular Surgery (OASIS) is a tool created by members of the ophthalmology department of the Harvard Medical School and designed to establish an objective ophthalmic surgical evaluation protocol to assess residents' surgical competency and improve residents' surgical outcomes.

As discussed previously, many forces have served to decrease the number of cases performed by residents during the course of their training. Thus, surgical proficiency must be acquired and verified in less time, with the risk that some surgeons may not be sufficiently skilled at the completion of their training.

Sandra L. Cremers, M.D., F.A.C.S., instructor of ophthalmology, Harvard Medical School, and a prime mover in the development of OASIS, describes OASIS in an article published in Ophthalmology.6 The composite OASIS evaluation form consists of three parts — preoperative information, perioperative information and postoperative information. The OASIS evaluation form and computer database include some unique features. First, at each postoperative visit, the database allows for the calculation of surgically induced astigmatism. Second, the database can calculate automatically the amount of overcorrection in the final refractive outcome. Third, the database can easily provide graphic analysis of posterior capsular-tear cases, or rates of vitreous-loss cases — according to the residents or according to the surgical preceptors. Fourth, the program can provide graphic representation of vitreous-loss rates over the course of the academic year for a particular resident or for the residents as a whole.6 Thus, OASIS is one of the first tools that can be used to generate outcomes data.

Another tool that provides information complementary to OASIS is the Global Rating Assessment of Skills in Intraocular Surgery (GRASIS). Dr. Cremers eloquently described GRASIS in her follow-up article published in Ophthalmology in October 2005.7 GRASIS can be used to assess a resident's surgical care of patients as well as a resident's surgical knowledge, preparedness and interpersonal skills.

In contrast to OASIS, which has fixed parameters, such as total phacoemulsification time or the need for attending intervention, GRASIS provides a much broader view of how the resident-surgeon functions intraoperatively with regard to the mastery of techniques as well as an assessment of general issues of professionalism.

Additionally, GRASIS can be used to provide both formative and summative feedback. GRASIS allows an evaluation of a resident's overall ethical behavior and professionalism in the new surgical sphere. GRASIS can evaluate a resident's readiness to operate without any assistance.7 This may be a vital piece of information when an employer is considering hiring a new surgeon. Finally, GRASIS allows surgical preceptors and residents to continuously improve the quality of care that patients receive by improving the transfer of surgical knowledge and skills more effectively and efficiently.

The Surgical Challenge

Thus, it appears that the academic side of ophthalmology is making great strides toward achieving higher standards of training, starting from medical school and lasting through fellowship. These standards now exist for both knowledge and interpersonal skills. Achieving similar standards for surgical skills, however, may prove to be more challenging.

The challenge of developing highly skilled surgeons with appropriate surgical experience during the training years is becoming more difficult due to the migration of patients away from the hands of residents and fellows and into the hands of high-volume surgeons. To bridge this gap, I believe that physicians who practice in fields of surgical medicine will likely need to participate in "apprenticeships." These apprenticeships can be achieved by creating partnerships between academic medicine and the higher-volume, group-style private practices within the geographic region of a given teaching facility.

Historically, fellowship programs have discouraged their fellows from practicing medicine in the same city or region where they completed their training. I believe that this strategy is counterproductive to the development of skilled surgeons because the potential employer has no opportunity to evaluate the surgical skills of a recent graduate of a residency or fellowship program. When potential employers contact the training program, they are not likely to hear negative commentary on the former resident or fellow.

The Apprenticeship Solution

One potential solution would be to allot a 3- to 12-month block of time for a resident to perform an apprenticeship with a potential future employer. This would allow both the employer and potential employee to decide if they were a good long-term match. The potential employer could assess the candidate's surgical and clinical judgment. The newly trained physician could determine whether the prevailing practice culture is one that may provide long-term mutual benefits. This type of apprenticeship may become essential in the near future, when meeting pay-for-performance outcomes criteria could well determine the level of practice profitability.

This partnership between academic institutions and private practices may require a lengthening of the current 36-month training period. Extension of the ophthalmology residency to 48 months has been discussed for decades. Initially, lengthening the residency was thought to be one way of preventing an oversupply of ophthalmologists in the private sector. At this juncture, the primary motivation for lengthening the training period would be to provide adequate time to train ophthalmologists.

The only constant in medicine, as in life, is change. Clearly, the healthcare training paradigms must evolve to meet many new demands. Cooperation between academic institutions and the private sector will be necessary to stimulate this evolution. Additionally, the continuing participation of the pharmaceutical and surgical industries will also provide an important contribution toward the training of our doctors of tomorrow. This triumvirate will need to work in harmony. Together, these entities will help to usher in a new age of medical training. The ultimate goal will be to provide our patients with the opportunity to receive care from a caring, motivated, professional, ethical and skilled army of physicians and surgeons.

Andrew Rabinowitz, M.D., is a glaucoma specialist at Barnet Dulaney Perkins Eye Centers, a multi-location practice based in Phoenix, Ariz. He can be reached via e-mail at andrewrabinowitz@aol.com

References:

1. Sadun AA. The Challenge of teaching ophthalmology: a residency review committee perspective. Ophthalmology. 2000;107:1971-1972.

2. Lee AG. The New Competencies and their impact on resident training in ophthalmology. Survey of Ophthalmology. 2004;48:652-662.

3. Liesegang TJ. Hoskins HDJr, Albert DM, et al. Ophthalmic education: where have we come from, and where are we going? American Journal of Ophthalmology. 2003;136:114-121.

4. Mills, RP, Mannis MJ. Report of the american board of ophthalmology task force on the competencies. Ophthalmology. 2004;111:1267-1268.

5. Golnik KC, Goldenhar L. The Ophthalmic clinical evaluation exercise: reliability determination. Ophthalmology. 2005;112:1649-1654.

6. Cremers SL, Ciolino JB, Ferrufino-Ponce ZK, Henderson, BK. Objective assessment of skill In intraocular surgery (OASIS). Ophthalmology. 2005; 112:1236-1241.

7. Cremers SL, Lora AN, Ferrufino-Ponce, ZK.. Global rating assessment of skills in intraocular surgery (GRASIS.) Ophthalmology. 2005;112:1655-1660.