Enhancing undergraduate medical education using handheld computing: a pilot program during the UCSF Clinical Core curriculum

Meg Autry,MD; Manuel Pardo, Jr., MD
Fall 2003

Background information
The growth of information technology has had a dramatic impact on both medical education and the practice of medicine. This is evident in the program requirements for both graduate and undergraduate medical education. For example, the Accreditation Council for Graduate Medical Education (ACGME) has endorsed six general competencies. The Patient Care competency states that residents should use information technology to support patient care decisions and patient education. The Practice-based Learning and Improvement competency states that residents should use information technology to manage information, access on-line medical information, and support their own education. In undergraduate medical education, the Liaison Committee on Medical Education (LCME) requires that medical schools must have access to well-maintained library and information facilities, sufficient in size, breadth of holdings, and information technology to support its education and other missions.

In the last five years, handheld computers, also called Personal Digital Assistants (PDAs), have increased in popularity. In 2001, approximately 26% of physicians used PDAs, with an estimate of 50% by 2005. The use of PDAs in medical education has also increased. A medical subject heading (MESH) term of “handheld computer” was created in 2003. A literature search using this term has revealed a variety of uses in medical education. The categories of use include:
1. Patient and procedure tracking. Several institutions have reported the use of personal digital assistants in patient/procedure tracking and accomplishment of educational objectives. There is also data that shows excellent student compliance with hand held collection of experience as well as good correlation with paper or more traditional collecting systems.
2. Evaluation of training of medical students. Reports include use of PDAs to evaluate a longterm general practice experience, and a family medicine clerkship. PDA database entry has been used to document that medical students are receiving the appropriate balance and diversity of patients.
3. Providing point of care medical reference software. This has included a variety of medical specialty areas, including family medicine, anesthesia, pediatrics, pain management, critical care, pharmacology and cardiology. PDAs may be useful in other aspects of medical education. They may facilitate several LCME accrediting requirements, such as point-of-care medical informatics, teaching, and monitoring the equivalency of educational experiences across multiple clinical sites. At the graduate level, the ACGME has required a resident case log system for many specialties, and offers a PDA version for the majority of them.

A few years ago, the UCSF School of Medicine implemented a computer requirement for students and the Office of Educational Technology has recommended minimum standards for these computers. The UCSF digital curriculum, iRocket, was developed to enhance the face-to-face teaching in the Essential Core curriculum. iRocket also provides basic information on the Clinical Core clerkships.

Currently, each Clinical Core clerkship collects patient or procedure data independently, in a variety of formats. In 2003, the Education Technology Oversight Committee surveyed current medical students on their PDA use. Sixty-nine percent of students surveyed currently own a PDA, with the majority buying it during the Essential Core. Furthermore, 85% of the students believed that the PDA would become an integral tool in the medical profession. In addition to student opinion on the potential for PDAs in medicine, faculty also believe there are possible benefits. At the 2003 School of Medicine Curriculum Retreat, handheld computing was identified as a potential area for improving the curriculum. Also, at the 2003 School of Medicine Educational Technology Strategic Vision Retreat, handheld computing was described as an important future vision theme.

Goals of Project
Our proposal will focus on the Clinical Core because we believe the greatest benefits of handheld computing technology will be seen in this part of the curriculum. Many PDA uses in the Clinical Core will also apply to graduate medical education and clinical practice. A general goal of our proposal includes developing a method of measuring the equivalency of clinical rotations between different sites, and among different students. We believe that this will help clerkship directors to advance the quality and consistency of the Clinical Core educational experience. By the end of the funding period, we will achieve the following specific goals:
1. We will develop minimal standards for a handheld computing device (to include memory, expandability, and other hardware-related issues).
2. For each clinical rotation, we will develop a list of PDA medical software that has been evaluated by students and key faculty members.
3. We will implement a PDA patient/procedure database that will fit the specific needs of a given clerkship. The database will link to medical school objectives stated on the Clinical Core Web site on iRocket.
4. For each rotation, we will provide participating students with clerkship specific information in a PDA-compatible format. This information will include contact information for the rotation as well as weekly schedules for conferences, lectures, etc.

Plan for measurement and documentation of project efficacy and outcome
Each one of our goals has a specific desired outcome, listed below.
Goal one: develop minimal standards for a handheld computing device.
• Students will complete surveys after each clerkship on different aspects of the PDA they used.
• We will prepare a list of minimal standards for PDAs based on student evaluations throughout the year.
Goal two: develop a list of PDA medical software that has been evaluated by students and key faculty members.
• After each clerkship, students will complete surveys on the medical software they used.
• We will survey key faculty identified by the clerkship director regarding recommended PDA medical software.
• Based on the surveys, we will create a reference list specific to the clerkship or specialty. For each medical reference, we will summarize the student or faculty ratings as “essential”, “recommended”, or “not useful”. In addition, we will make inquiries into special prices or volume discounts for students.
Goal three: implement a PDA patient/procedure database that will fit the specific needs of a given clerkship.
• We will compare the tracking results by the handheld device to that collected by students using the more standard paper method, for those clerkships collecting paper data. We will be able to provide the student and clerkship director with the statistics regarding number of patients and procedures.
• From the student perspective, the success of this goal will depend on student perception of ease of use, as well as potential utility for their learning.
• From the clerkship director perspective, the success will depend on the ability of the database to provide accurate and relevant information regarding the students’ educational experience. The ability to judge equivalence of the educational experience will also depend on this.
• Therefore, the documentation of the efficacy of this goal will consist of survey results from students and clerkship directors. The first survey should occur approximately 2-3 months from the start of the project, to allow time to adjust the patient/procedure database accordingly.
Goal four: provide participating students with clerkship-specific information in a PDA-compatible format.
• At the end of each rotation, students will complete a questionnaire on the utility of the PDA clerkship information.
• We will modify this information and determine whether these subjective student evaluations improve.

The real success of our proposal will be judged by the spread of handheld technology to other aspects of undergraduate and graduate medical education. Our pilot program may lead to widespread acceptance of PDAs in the Clinical Core, and the medical school may decide to implement a PDA requirement, similar to the desktop computer requirement.

Plan for continuation of project at the end of funding cycle
At the end of the funding cycle, we will share our findings and recommendations with the School of Medicine leadership group, including Vice Dean for Education David Irby, Associate Dean for Curriculum Helen Loeser, and Clinical Studies Steering Committee Chair Molly Cooke. Future development of handheld computing technology at UCSF will depend on their assessment of our project. However, because the vast majority of students already purchase PDAs and PDA medical software, our recommendations should be useful to all Clinical Core students. Future development of the patient/procedure log at UCSF may include spread to undergraduate clinical electives and graduate medical education. At that point, a more costly school wide PDA server system may be required. Finally, once students are facile with handheld technology, there will be increased opportunity for PDA enabled education research.

In summary, we expect to meet our one-year goal of recommending a minimal hardware requirement for a PDA, a PDA-based patient/procedure log, PDA medical software, and clerkship-specific information for the PDA. The long-term development of handheld computing will depend on the School of Medicine’s commitment to this type of educational technology.

For more information, please contact:
Meg Autry, MD
Manuel Pardo, Jr. MD