Historically, there has been a lack of rigorous evidence-based studies of Professional Development, particularly for science educators. This has been due to PD providers merely assessing participants’ opinions of the professional learning experience, rather than incorporating well-thought out evaluations of PD and collecting empirical evidence of the effectiveness of PD. Recently, the advent of sophisticated and more robust evaluation procedures is leading to important insights into the impacts on teacher practices and student learning outcomes and is thus giving rise to more productive professional learning opportunities and trends, changing the way PD is structured and carried out.
Evaluation of learning in professional development is often underused, though it can be valuable for professional developers, participants, and other stakeholders (Loucks-Horsley, Love, Stiles, Mundry, & Hewson, 2003). As an important contributor toward continuing improvement, a well-designed evaluation provides a compelling picture of a PD and identifies changes that should be made going forward (Loucks-Horsely et al., 2003). Successes, shortcomings, and changes should be examind, using a wide range of instruments and sources of information (Loucks-Horsely, et al., 2003).
Carefully planning for the evaluation from the start makes it possible to generate evidence about the impact of the PD that is more helpful than simply asking the participants “how well they enjoyed” the PD (Hirsch & Killion, 2009). Five critical levels of professional development evaluation have been distinguished, ranging from simple to more complex, with each level building on the one before it (Guskey, 2002):
Evidence of each of these stages can be gathered using comparison groups and pre/post measures. (See Guskey, 2002, for references for more in-depth descriptions).
Measurement should concentrate on critical core features of teachers’ learning experiences, not on the specific content of the PD (Desimone, 2009). These features can be captured in a variety of ways, in a “mosaic” of methods (Heller, Daehler, & Shinohara, 2003), incorporating observation, interviews, and surveys, as well as formative and summative evaluations, among other means of gathering evidence.
It is imperative to discern between outputs and outcomes (Learning Forward, 2014). Outcomes are indicators of success in achieving the goals of the PD, such as how educator practice will change and how student learning will improve. Outputs, while important, can be a learning plan that is produced during the PD or data about participants, such as their impression of a presenter, etc. – professional development elements unrelated to outcomes (Learning Forward, 2014). Evaluation should focus not on “activities” that are unrelated to sustained learning, but on learning experiences connected to outcomes (Learning Forward, 2015).
As mentioned in discussions of “what works” in PD, key features that need to be assessed include in-depth science content, active learning opportunities, coherence and alignment of curriculum and standards, duration and length of contact time, and a presence of processes for collective participation (Heller et al., 2003; Desimone, 2009; Ingvarson, Meiers, & Beavis, 2003). Planners and evaluators should build in feedback loops each day to respond to what was clear and what was not clear in any presentations. Providers must also assess continually how participants are feeling, understanding the material, and participating in the various learning experiences so that they can provide adjustments as needed.
While it continues to be important to evaluate the delivery of PD, it is even more important to understand what participants actually learn, when and how they begin to apply their learning, and when and how it benefits students. PD is a process not an event (Mizell, H., 2003; Heller at al., 2003), and changes in the classroom due to the implementation of what was learned in the PD may take time to evolve.
For more in-depth guides to evaluating learning in professional development, see Haslam (2010), Killion (2003), Pendray & Crockett (2016), Loucks-Horsely et al, Guskey (2002), and Learningforward.org.
Desimone, L. M. (2009). Improving impact studies of teachers’ professional development: Toward better conceptualizations and measures. Educational Researcher, 38(3) 181–199. Retrieved from https://www.semanticscholar.org/paper/Improving-Impact-Studies-of-Teachers%E2%80%99-Professional-Desimone/42f136ad2d6650cc522337411e5de1b3710faec0?p2df.
George, S. (2018, July 11). Innovative Back to School Professional Development. [Blog post]. Retrieved from https://www.edweek.org/leadership/opinion-innovative-back-to-school-professional-development/2018/07.
Guskey, T., & Huberman, A. (1995). Professional Development in Education: New paradigms and practices. New York: Teachers College Press.
Guskey, T. (2002). Professional Development and Teacher Change. Teachers and Teaching, 8(3), 381 – 391. Retrieved from http://dx.doi.org/10.1080/135406002100000512.
Guskey, T. (2002). Does It Make a Difference? Educational Leadership, 59(6), 45-51.
Guskey, T. (2003). What Makes Professional Development Effective? Phi Delta Kappan, 84(10), 748-750. Retrieved from http://tguskey.com/wp-content/uploads/Professional-Learning-6-What-Makes-Professional-Development-Effective.pdf
Guskey, T., & Yoon, K. (2009). What Works in Professional Development? The Phi Delta Kappan, 90(7), 495-500. Retrieved from https://journals.sagepub.com/doi/abs/10.1177/003172170909000709?journalCode=pdka.
Guskey, T. (2014). Planning Professional Learning. Educational Leadership 71(8), 10-16.
Haslam, M. B. (2010). Teacher Professional Development Evaluation Guide. Retrieved from https://learningforward.org/wp-content/uploads/2017/08/teacher-professional-development-evaluation-guide.pdf.
Heller, J., Daehler, K., & Shinohara, M. (2003). Connecting All the Pieces: Using an evaluation mosaic to answer an impossible question. Journal of Staff Development, 24(4), 36-41.
Heller, J., Daehler, K., Wong, N., Shinohara, M., & Miratrix, L. (2012). Differential Effects of Three Professional Development Models on Teacher Knowledge and Student Achievement in Elementary Science. Journal of Research in Science Teaching, 49(3), 333-362. Retrieved from https://scholar.harvard.edu/files/lmiratrix/files/differential_effects_of_three_professional_development_models_on_teacher_knowledge_and_student_achievement_in_elementary_science_2012.pdf.
Hirsch, S., & Killion, J. (2009). When Educators Learn, Students Learn: Eight Principles of Professional Learning. Phi Delta Kappan, 90(7), 464-469. Retrieved from http://www.jstor.org/stable/20446154.
Ingvarson, L., Meiers, M., & Beavis, A. (2003). Evaluating the quality and impact of professional development programs. Building teacher quality: research conference 2003, Melbourne: Australian Council for Educational Research. Retrieved from https://research.acer.edu.au/professional_dev/3.
Ingvarson, L., Meiers, M., & Beavis, A. (2005). Factors affecting the impact of professional development programs on teachers' knowledge, practice, student outcomes & efficacy. Education Policy Analysis Archives, 13(10). Retrieved from https://research.acer.edu.au/cgi/viewcontent.cgi?referer=https://www.google.com/&httpsredir=1&article=1000&context=professional_dev.
Killion, J. (2003) Steps to Your Own Evaluation. Journal of Staff Development, 24(4), 22-23.
Learning Forward. (2015, July 23). Senate's Version of ESEA Affirms Professional Learning Principles [Blog post]. Retrieved from https://learningforward.org/2015/08/01/senates-version-esea-affirms-professional-learning-principles/.
Learning Forward. (2014). Evaluating Professional Learning: Measuring Educator and Student Outcomes. Retrieved from https://learningforward.org/journal/april-2018-vol-39-no-2/discuss-collaborate-facilitate/.
Learning Forward. (2018). Launch an Analysis of Professional Learning. The Learning Professional, 39(2), 62-66. Retrieved https://learningforward.org/journal/april-2018-vol-39-no- 2/discuss-collaborate-facilitate/
Loucks-Horsley, S., Love, N., Stiles, K. E., Mundry, S. E., & Hewson, P. (2003). Designing professional development for teachers of science and mathematics (2nd ed.). Thousand Oaks, CA: Corwin Press.
Mizell, H. (2003). Facilitator: 10, Refreshments: 8, Evaluation: 0. Journal of Staff Development, 24(4), 10–13.
NSTA. (2006). NSTA Position Statement on Professional Development in Science Education. Retrieved from http://www.nsta.org/about/positions/profdev.aspx.
Pendray, A., & Crockett, J. (2016). Make Evaluation Count. Journal of Staff Development, 37(6), 42-45. Retrieved from https://learningforward.org/journal/december-2016-issue/make-evaluation-count/.
Price, C. A., & Chiu, A. (2018). An experimental study of a museum-based, science PD programme’s impact on teachers and their students, International Journal of Science Education, 40(9), 941 – 960. Retrieved from https://doi.org/10.1080/09500693.2018.1457816.