A qualitative study of validation practices in classroom-based assessment in health professions education: what do faculty members actually do?
DOI:
https://doi.org/10.36834/7e1tgj57Abstract
Background: Faculty in health professions education (HPE) play a key role in assessing student learning through classroom-based assessments. While researchers have studied admission and licensure assessments, we know little about how classroom assessments are validated. Through validation practices, faculty help ensure that decisions about student learning are well-supported. This study explores how HPE faculty validate classroom-based assessments.
Methods: The authors conducted a qualitative interpretive descriptive study. Twenty faculty members in HPE from four Canadian universities were recruited for individual interviews using quota sampling. Two team members analyzed the qualitative data through inductive thematic analysis and held discussions with the team to reach a consensus.
Results: Although they do not refer to recommended practices by name, participants used many means to ensure the quality of their classroom-based assessments. When preparing their assessments, they consider elements such as authenticity, content representativity and adequate preparation of students. During assessment, they pay attention to elements such as plagiarism and support. Following the assessment, they make decisions on scoring strategy and process and use assessment data to adjust assessments.
Conclusion: Faculty members in HPE recognize the importance of validation and used multiple strategies to document validity evidence. To better support faculty in their validation practices, it is valuable to understand current approaches and provide adequate resources to build on existing validation practices.
References
1. Young M, St-Onge C, Xiao J, Vachon Lachiver E, Torabi N. Characterizing the literature on validity and assessment in medical education: a bibliometric study. Perspect Med Educ. 2018;7(3):182‑91. https://doi.org/10.1007/s40037-018-0433-x
2. Touchie C, Kinnear B, Schumacher D, et al. On the validity of summative entrustment decisions. Med Teach. 2021;43(7):780‑7. https://doi.org/10.1080/0142159x.2021.1925642
3. van der Vleuten CPM, Schuwirth LWT. Assessing professional competence: from methods to programmes. Med Educ. 2005;39(3):309‑17. https://doi.org/10.1111/j.1365-2929.2005.02094.x
4. Baird JA, Andrich D, Hopfenbeck TN, Stobart G. Assessment and learning: fields apart? Assess Educ Princ Policy Pract. 2017;24(3):317‑50. https://doi.org/10.1080/0969594X.2017.1319337
5. Cizek GJ, Rosenberg SL, Koons HH. Sources of validity evidence for educational and psychological tests. Educ Psychol Meas. 2008;68(3):397‑412. https://doi.org/10.1177/0013164407310130
6. DeLuca C. Interpretive validity theory: mapping a methodology for validating educational assessments. Educ Res. 2011;53(3):303‑20. https://doi.org/10.1080/00131881.2011.598659
7. Downing SM. Validity: on the meaningful interpretation of assessment data. Med Educ. 2003;37(9):830‑7. https://doi.org/10.1046/j.1365-2923.2003.01594.x
8. Messick S. Standards of validity and the validity of standards in performance assessment. Educ Meas Issues Pract. 1995;14(4):5‑8. https://doi.org/10.1111/j.1745-3992.1995.tb00881.x
9. Chan EKH. Standards and guidelines for validation practices: development and evaluation of measurement instruments. In: Zumbo BD, Chan EKH, editors. Validity and validation in social, behavioral, and health sciences. Cham: Springer International Publishing; 2014. p. 9‑24. https://doi.org/10.1007/978-3-319-07794-9_2
10. Gordon S, Smith E. Who are faculty assessment leaders? Assess Eval High Educ. 2022;47(6):928‑41. https://doi.org/10.1080/02602938.2021.1983518
11. García-Ramírez Y, Bijelić V. Assessing task prioritization for professors through affinity and satisfaction scores. Cogent Educ. 2024;11(1):2321355. https://doi.org/10.1080/2331186X.2024.2321355
12. Link AN, Swann CA, Bozeman B. A time allocation study of university faculty. Econ Educ Rev. 2008;27(4):363‑74. https://doi.org/10.1016/j.econedurev.2007.04.002
13. Canadian Institutes of Health Research. Knowledge translation. Available from: https://cihr-irsc.gc.ca/e/29418.html
14. Straus SE, Tetroe JM, Graham ID. Knowledge translation is the use of knowledge in health care decision making. J Clin Epidemiol. 2011;64(1):6‑10. https://doi.org/10.1016/j.jclinepi.2009.08.016
15. American Educational Research Association., American Psychological Association., National Council on Measurement in Education., Joint Committee on Standards for Educational and Psychological Testing (U.S.). Standards for educational and psychological testing. Washington, DC: American Educational Research Association; 2014.
16. Thorne SE. Interpretive description: qualitative research for applied practice. New York, NY: Routledge; 2016.
17. Marceau M, Gallagher F, Young M, St‐Onge C. Validity as a social imperative for assessment in health professions education: a concept analysis. Med Educ. 2018;52(6):641‑53. https://doi.org/10.1111/medu.13574
18. Braun V, Clarke V. Thematic analysis: a practical guide. Los Angeles, CA: SAGE Publications; 2022.
19. Messick S. Validity. In: Linn RL, editor. Educational Measurement. 3rd ed. New York, NY: Macmillan; 1989. p. 13‑103.
20. Messick S. Validity of psychological assessment: validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. Am Psychol. 1995;50(9):741‑49. https://psycnet.apa.org/doi/10.1037/0003-066X.50.9.741
21. Kane MT. Validation. In: Brennan RL, editor. Educational Measurement. 4th ed. Westport, CT: American Council on Education/Praeger; 2006. p. 17‑64.
Published
Issue
Section
License
Copyright (c) 2026 Mélanie Marceau, Kathleen Ouellet, Aliki Thomas, Jean-Sébastien Renaud, Christina St-Onge
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Submission of an original manuscript to the Canadian Medical Education Journal will be taken to mean that it represents original work not previously published, that it is not being considered elsewhere for publication. If accepted for publication, it will be published online and it will not be published elsewhere in the same form, for commercial purposes, in any language, without the consent of the publisher.
Authors who publish in the Canadian Medical Education Journal agree to release their articles under the Creative Commons Attribution-Noncommercial-No Derivative Works 4.0 Canada Licence. This licence allows anyone to copy and distribute the article for non-commercial purposes provided that appropriate attribution is given. For details of the rights an author grants users of their work, please see the licence summary and the full licence.
How to Cite
