Inspired Teaching

Rapid Evaluation of Educational Innovation & Reform

📅 Monday, January 24th, 2022 from 1-2:30pm EST

💻 Delivered Virtually

Effective implementation and sustained adoption of innovations in health professions education depends on systematic program evaluation efforts. In this session, we will introduce program leaders, educators, faculty, residents and administrators to the intersections of evaluation theory and practice. We will introduce a unique adaptation-focused program evaluation strategy, termed Rapid Evaluation, and illustrate its use in the evaluation of Competency-Based Medical Education (CBME). We will focus on evaluating the fidelity and integrity of implementation, as well as identifying early outcomes and unintended consequences to engage in a process of adaptation.

Learning Objectives

At the end of this virtual event, participants will be able to:

  • Report on the value of evaluating educational innovations such as competence-based medical education (CBME).

  • Define fidelity and integrity of implementation.

  • Explain the strengths and limitations of rapid evaluation as one approach to evaluation of educational programs.

  • Describe important steps and considerations in implementing rapid evaluation in practice.

  • Develop a plan for evaluation within their own program and institutional context.

Agenda

  • Introduction & Agenda (5 min.)

  • Program Evaluation Primer (10 min.)

  • Brief Exercise (5 min.)

  • Rapid Evaluation Methodology (10 min.)

  • Describe Implementations as Intended (5 min.)

  • Breakout & Debrief (10 min.)

  • Measure Enacted Implementation (10 min.)

  • Breakout +& Debrief (10 min.)

  • Compare Planned vs. Enacted to Inform Adaptation (10 min.)

  • Questions & Discussion (15 min.)

Step-by-Step, We Will Illustrate & Explain How to:

  • Establish a team identify, organize, and engage with primary stakeholders.

  • Determine evaluation priorities and questions.

  • Select methods for gathering useful qualitative and quantitative data.

  • Mobilize findings to share with stakeholders and broader audiences using technical reports and peer-reviewed scholarship.

Speaker

Andrew K. Hall

Dr. Andrew K. Hall is an Associate Professor in the Department of Emergency Medicine (EM) at the University of Ottawa, where he is a Simulation Educator and an Assessment Advisor. He works with the Royal College of Physicians and Surgeons of Canada as a Clinician Educator and chair of the Competency-By-Design Program Evaluation Operations team. He is additionally a member of the International Competency-Based Medical Education Collaborators, and co-chair of the Canadian Emergency Medicine Simulation Educators Research Collaborative (EM-SERC). His current research areas include simulation, simulation-based assessment, and competency-based medication education, with a focus on evaluation and outcomes.

Facilitator

Sarrah Lal

Sarrah Lal (@SarrahML) is an assistant professor in the Division of Education & Innovation (DEI) within the Department of Medicine. She is a entrepreneurship and innovation expert and directs various educational efforts within the Michael G. DeGroote Initiative for Innovation in Healthcare. Currently she is also the lead of the Leadership & Management team in the McMaster Faculty of Health Sciences Program for Faculty Development (@MacPFD).

Facilitator

X. Catherine Tong

Dr. Tong is an Assistant Clinical Professor affiliated with the Department of Family Medicine at McMaster University. She currently practices emergency medicine at the Grand River and St. Mary’s General Hospitals in Kitchener-Waterloo, and family medicine at the Grand Valley Institute for Women, a federal correctional facility in Kitchener. She is the faculty development coordinator at Waterloo Regional Campus (@MacMedWRC), and the team lead for the Inspired Teaching Team of the Program for Faculty Development at the Faculty for Health Sciences.