GME

GME Evaluation Task Force Recommendation

Back to GME Evaluation
and Assessment Portal

July 1, 2008

PROFESSIONALISM, INTERPERSONAL AND COMMUNICATION SKILLS

Overview - Professionalism, Interpersonal and Communication Skills

360 evaluations include observations by a variety of individuals from the multiple contexts where professionalism and communication skills are demonstrated. Ratings by self, peers, colleagues and faculty are generally collected using different tools than ratings by patients. The 360 method captures information on most of the competencies listed by the ACGME under “Professionalism” and “Interpersonal & Communication Skills and is highly recommended for every program’ assessment system.

Core Measure for UCSF GME

For Health Care Team and Self-Evaluation, we recommend the Professional Associate Evaluation Form, developed by the CREOG (Ob/Gyn) Competency Task Force

For Patient Surveys, we recommend the American Board of Internal Medicine (ABIM) Patient Survey

Health Care Team Evaluations: PROFESSIONALISM

Reliability and Validity:

The optimal number and frequency 360 assessments is uncertain. A feasible minimum to achieve inter-rater reliability = .80 may be 5 non-clinical and 6 clinical raters each on 2 occasions (1).

Preliminary evidence of construct validity shows modest growth in 360 scores comparing senior vs. junior residents with a magnitude similar to the growth in other competencies including critical self-reflection skills (2).

Content validity exists to the extent that the survey items actually assess the professionalism and communication skills they are intended to measure. The 9-item evaluation includes communication (patients/families, nursing/allied staff), respect (patients, nursing/allied staff), compassion, reliability, honesty/integrity, responsibility, and advocacy.

Administration
Timing: More frequent observations tied to specific learning experiences provide more valid data than the minimum, i.e., global impressions provided twice annually.
Who Performs: Many individuals can legitimately contribute to 360 assessments. The list usually includes faculty, residents (supervisors, peers and juniors), nursing and other clinical staff, and consultants. Selection should be based upon the individual’s opportunities to directly observe the resident interacting with patients and the health care team.
N.B. Faculty need to complete the 360 evaluation separately from their global ratings at the end of the rotation so the data can be summarized and reported accurately.
Format: 9 items sample aspects of professionalism and communication skills. Each item is scored on 9-point scales with 1-3 = unsatisfactory, 4-6 = meets expectations, and 7-9=excellent.
Scoring Criteria and Training: Each item defines a specific trait in clear language. Standard-setting would be helpful for consistently distinguishing the 3 levels of unsatisfactory, the 3 levels of satisfactory and the 3 levels of excellent. Documentation: Assessment can be documented on E-value and learners can access the results confidentially.

Uses of the Data

Summarizing the data: Score averages, ranges, and comparative data for the PGY year are provided as part of the data report residents review before their semi-annual meetings. The other data sources for the 360 assessment include the relevant items (e.g., respect) from the residents’ clinical educator evaluations and patient surveys. These are also summarized as means and ranges compared with averages for the PGY year.

Formative Uses: Assessment results support development of professionalism and communication skills by individual residents, identify trends in performance across PGY groups and spur possible improvements to the curriculum in these two competencies

Summative Decisions and Remediation: Scores in the unsatisfactory (scores 1-3) would trigger remediation and a low satisfactory score (4) would trigger suggestions for improvement

Workflow Procedures

A systematic approach is recommended to maximize the use of the assessments and facilitate data management. An example follows in the Appendix.

References

1. Murphy DJ, Bruce DA, Mercer SW, Eva KW. The reliability of workplace-based assessment in postgraduate medical education and training: a national evaluation in general practice in the United Kingdom. Adv in Health Sci Educ 2008 DOI 10.1007/s10459-008-9104-8.

2. Learman LA, Autry AM, O'Sullivan P. Reliability and validity of reflection exercises for obstetrics and gynecology residents. Am J Obstet Gynecol 2008;198(4):461.e1-8; discussion 461.e8-10.

Professionalism - Health Care Team

Health Care Team Professionalism, Communication and Interpersonal Skills	Core Measure for UCSF GME Health Care Team Evaluation Form Health Care Team Sample Workflow
	E*Value Forms (pending)

Go to Professionalism - Patient Care Surveys
If you have questions and/or requests about the recommended tools, or how to incorporate these forms into your clinical training program evaluation system (E*Value), please contact: kapurg@medsch.ucsf.edu

Back to GME Evaluation and Assessment Portal

Curriculum Home

Graduate Medical Education
500 Parnassus Avenue, MU 250 East
San Francisco, CA 94143-0474
(415) 476-4562