To the content
Спецвыпуск . 2022

AMEE GUIDE # 119. THE FOUNDATIONS OF MEASUREMENT AND ASSESSMENT IN MEDICAL EDUCATION

Abstract

As a medical educator, you may be directly or indirectly involved in the quality of assessments. Measurement has a substantial role in developing the quality of assessment questions and student learning. The information provided by psychometric data can improve pedagogical issues in medical education.

By measuring, we are able to assess the learning experiences of students. Standard setting plays an important role in assessing the performance quality of students as doctors in the future. Presentation of performance data for standard setters may contribute towards developing a credible and defensible pass mark. Validity and reliability of test scores are the most important factors for developing quality assessment questions. The analysis of assessment individual questions provide useful feedback for assessment leads in order to improve the quality of each question, and hence make students' marks fair in terms of the diversity and ethnicity. Item Characteristic Curves (ICC), Differential Item Function (DIF) analysis and option analysis will send signals to assessment leads to improve the quality of individual question.

Conflict of interests. The authors declare no conflict of interests.
Tavakol M., Dennick R. The Foundations of Measurement and Assessment in Medical Education. Medical Teacher. 2017; 39 (10): 1010-5.

· Feher Waltz C., Stricland O., Lenz E. Measurement in nursing and health research. New York: Springer, 2010.

· Goodwin L. Changing conceptions of measurement validity: an update on the new standards. J Nurs Educ. 2002; 41: 100-6.

· Haladyna T., Hess R. An evaluation of conjunctive and compensatory standard-setting strategies for test decisions. Educ Assess. 1999; 6: 129-53.

· Haladyna T.M., Downing S. Functional distractors: implications for test-item writing and test design. 1988 [Electronic resource]. URL: http://files.Eric.Ed.Gov/fulltext/ed293851.pdf (date of access August 10, 2015)

· Hambleton R., Itoniak M., Copella J. Essential steps in setting Performance standards on educational tests and strategies for assessing the reliability of results. In: G. Cizek (ed.). Setting Performance Standards. London: Routledge, 2012.

· Hambleton R., Pitoniak M. Setting performance standards. In: R.L. Brennan (ed.). Educational Measurement. USA: American Council on Education, 2006.

· Henryssen S. Gathering, analyzing, and using data on test items. In: R. Thorndike (ed.). Educational measurement. Washington, DC: American Council on Education, 1971.

· Hoyt C.J. Test ability estimated by analysis of variance. Psychometrika. 1941; 6: 153-60.

· Hurtz G., Auerbach M. A meta-analysis of the effects of modification to the Angoff method on cut-off scores and judgment consensus. Educ Psychol Meas. 2003; 63: 584-601.

· Kane M. Validating high-stakes testing programs. Educ Meas Issues Pract. 2002; 21: 31-41.

· Kelley T., Ebel R., Linacre J. Item discrimination indices. Rasch Meas Trans. 2002; 16: 883-4.

· Kolen M. Scaling and norming. In: R. Brennan (ed.) Educational Measurement. Westport, CT: American Council on Education, 2006.

· Lane S., Stone C. Performance assessment. In: R. Brennan (ed.) Educational Measurement. USA: ACE, 2006.

· Margolis M., Clauser B. The impact of examinee performance information on judges’ cut scores in modified Angoff standard-setting exercises. Educ Meas Issues Pract. 2014; 33: 15-21.

· Mcdonald M. Guide to assessing learning outcomes. New York: Jones and Bartlett Learning, 2014.

· Mckinley D., Norcini J. How to set standards on performance-based examinations: AMEE Guide No. 85. Med Teach. 2014; 36: 97-110.

· Miller M., Linn R., Gronlund N. Measurement and assessment in teaching, Boston, Pearson, 2013.

· Norcini J., Dawson-Saunders B. Issues in recertification in North America. In: D. Newble, B. Jolly, R. Wakeford (eds). The Certification and Recertification of Doctors. Cambridge: Cambridge University Press, 1994.

· Schmeiser C., Welch C. Test development. In: R.L. Brennan (ed.). USA: American Council on Education, 2006.

· Shepard L. Classroom assessment. In: R. Brennan (ed.). Educational Measurement. Westport, CT: American Council on Education, 2006.

· Sireci S.S., Robin F. Using cluster analysis to facilitate standard setting. Appl Meas Educ. 1999; 12: 301-5.

· Taube K. The incorporation of empirical item difficulty data into the Angoff standard-setting procedure. Eval Health Prof. 1997; 20: 479-98.

· Tavakol M., Dennick R. Post-examination analysis of objective tests: AMEE Guide No. 54. [Electronic resource]. Dundee AMEE, 2001a. URL: www.amee.org

· Tavakol M., Dennick R. Post examination analysis of objective tests. Med Teach. 2011b; 33: 447-58.

· Tavakol M., Dennick R. Post-examination interpretation of objective test data: monitoring and improving the quality of high-stakes examinations: AMEE Guide 66. Med Teach. 2012; 34: 161-75.

· Tavakol M., Dennick R. Psychometric evaluation of a knowledge based examination using Rasch analysis: an illustrative guide: AMEE Guide No. 72. Med Teach. 2013; 35: 74-84.

· Tavakol M., Dennick R. Post-examination analysis: a means of improving the exam cycle. Acad Med. 2016a; 91: 1324.

· Volkan K., Simon S., Baker H., Todres I. Psychometric structure of a comprehensive objective structured clinical examination: a factor analytic approach. Adv Health Sci Educ. 2004; 9: 83-92.

· Zieky M., Perie M. A primer on setting cut scores on tests of educational achievement [Elecrtonic resource]. ETS, 2006. URL: https://www.ets.Org/Media/Research/pdf/Cut_Scores_Primer.pdf [date of access June 10, 2016]

All articles in our journal are distributed under the Creative Commons Attribution 4.0 International License (CC BY 4.0 license)

CHIEF EDITOR
CHIEF EDITOR
Balkizov Zalim Zamirovich
Secretary General of the Russian Society of Medical Education Specialists, Director of the Institute of Training of Medical Education Specialists of the Russian Medical Academy of Continuing Professional Education, 125993, Moscow, Russian Federation, Professor of the Department of Vocational Education and Educational Technologies of the N.I. Pirogov RNIMU of the MOH of Russia, CEO of GEOTAR-Med, Advisor President of the National Medical Chamber, Moscow, Russian Federation

Journals of «GEOTAR-Media»