Educator resource - evaluation of education

Evaluating adult healthcare education within health services is essential for ensuring that educational programs meet the needs of learners and effectively improve healthcare quality and consumer outcomes. A well-rounded evaluation approach involves assessing both learning outcomes and the overall impact of the education provided.

Key aspects of evaluation include:

Measuring learner outcomes and skill application - Assessing whether learners can implement new skills and practices effectively helps understand the educational program’s relevance and quality.

Gathering feedback from learners and stakeholders - Collecting feedback from participants and key stakeholders, such as managers or department heads, provides insights into the program’s strengths and any barriers learners may face in implementing new practices.

Evaluating patient and organisational impact - Ultimately, healthcare education should contribute to better consumer outcomes and enhanced service delivery.

Program evaluation resources

Articles

Allen LM, Hay M, Palermo C. Evaluation in health professions education—Is measuring outcomes enough? Med Educ. 2022; 56 (1): 127-136.

Frye, A. W., & Hemmer, P. A. (2012). Program evaluation models and related theories: AMEE Guide No. 67. Medical Teacher, 34 (5), e288–e299.

Hosseini, S., Allen, L., Khalid, F., Li, D., Stellrecht, E., Howard, M. and Chan, T.M. (2023) ‘Evaluation of Continuing Professional Development for Physicians – Time for Change: A Scoping Review’, Perspectives on Medical Education, 12 (1), p. 198–207.

Program Logic Model

Developing and using a program logic: A guide. NSW Health.

WK Kellogg Foundation. Logic Model Development Guide. Battle Creek, Michigan: WK Kellogg Foundation; 2004.

Taylor-Powell, E., & Henert, E. (2008). Developing a logic model: Teaching and training guide. Madison: University of Wisconsin-Extension.

Video – Exploring Program Logic by NSW Health: Exploring Program Logic

Training Needs Analysis

Hennesy Hicks Training Needs Analysis Toolkit

Markaki, A., Malhotra, S., Billings, R. et al.Training needs assessment: tool utilization and global impact. BMC Med Educ 21, 310 (2021).

Learner evaluation and outcome resources

Resources

Best Practice Clinical Learning Environment (BPCLE) framework – Department of Health (Victoria) recommended framework to help health services and education providers to meaningfully measure and monitor the quality of education and training

Kirkpatrick’s Evaluation Model
- Kirkpatrick's four level training evaluation
- Heydari, M.R., Taghva, F., Amini, M. et al. Using Kirkpatrick’s model to measure the effect of a new teaching and learning methods workshop for health care staff. BMC Res Notes 12, 388 (2019).
- Video: How to Use the Kirkpatrick’s 4 Levels of Evaluation Model for Effective L&D Assessment.

Brookfields critical incident questionnaire article

Templates

Example evaluation surveys:
- Template questionnaire for short training
- Template questionnaire for workshop

Clinical teaching/educator evaluation resources

Articles

van Lierop, M., de Jonge, L., Metsemakers, J., & Dolmans, D. (2018). Peer group reflection on student ratings stimulates clinical teachers to generate plans to improve their teaching. Medical teacher, 40 (3), 302–309.

Maastricht Clinical Teaching Questionnaire (MCTQ)
- Boerboom, T. B. B., Mainhard, T., Dolmans, D. H. J. M., Scherpbier, A. J. J. A., Van Beukelen, P., & Jaarsma, A. D. (Debbie) C. (2012). Evaluating clinical teachers with the Maastricht clinical teaching questionnaire: How much “teacher” is in student ratings? Medical Teacher, 34 (4), 320–326.
- Stalmeijer, R. E., Dolmans, D. H. J. M., Wolfhagen, I. H. A. P., Muijtjens, A. M. M., & Scherpbier, A. J. J. A. (2010). The Maastricht Clinical Teaching Questionnaire (MCTQ) as a Valid and Reliable Instrument for the Evaluation of Clinical Teachers. Academic Medicine, 85 (11), 1732–1738.