Psychometric characteristics of integrated multi-specialty examinations: Ebel ratings and unidimensionality
Homer, MS; Darling, J; Pell, G · White Rose Research Online
Abstrak / Ringkasan
Over recent years, UK medical schools have moved to more integrated summative examinations. This paper analyses data from the written assessment of undergraduate medical students to investigate two key psychometric aspects of this type of high-stakes assessment. Firstly, the strength of the relationship between examiner predictions of item performance (as required under the Ebel standard setting method employed) and actual item performance (‘facility’) in the examination is explored. It is found that there is a systematic pattern of difference between these two measures, with examiners tending to underestimate the difficulty of items classified as relatively easy, and overestimating that of items classified harder. The implications of these differences for standard setting are considered. Secondly, the integration of the assessment raises the question as to whether the student total score in the exam can provide a single meaningful measure of student performance across a broad range of medical specialties. Therefore, Rasch measurement theory is employed to evaluate psychometric characteristics of the examination, including its dimensionality. Once adjustment is made for item interdependency, the examination is shown to be unidimensional with fit to the Rasch model implying that a single underlying trait, clinical knowledge, is being measured.
Topik Terkait
Referensi Serupa
Using item response theory to explore the psychometric properties of extended matching questions examination in undergraduate medical education
Bhakta, B.; Tennant, A.; Horton, M.; Lawton, G.; Andrich, D.
Using item response theory to explore the psychometric properties of extended matching questions examination in undergraduate medical education
Bhakta, B.; Tennant, A.; Horton, M.; Lawton, G.; Andrich, D.
Medical Student use of Facebook to Support Preparation for Anatomy Assessments
Pickering, JD; Bickerdike, SR
Pengembangan Instrumen Integrated Assessment antara Keterampilan Proses Sains dan Penguasaan Konsep IPA untuk Mengukur Aspek Kognitif Proses Siswa SMP Kelas VII Semester I
Santoso, Agus
THE DEVELOPMENT OF A SET OF INSTRUMENT FOR STUDENT PERFORMANCE ASSESSMENT IN THE PHYSICS LABORATORY WORK SUBJECT OF JUNIOR HIGH SCHOOL
Supahar
Standarisasi Instrumen Integratted Assessment Hasil Belajar Bahasa Konvergen dan Divergen Menggunakan Item Respons Theory 1-PL dengan Program QUEST
Pujiati Suyata, Prof. Dr.; Nur, Hidayanto
Standarisasi Instrumen Integratted Assessment Hasil Belajar Bahasa Konvergen dan Divergen Menggunakan Item Respons Theory 1-PL dengan Program QUEST
Pujiati Suyata, Prof. Dr.; Nur, Hidayanto
The impact of teamwork on peer assessment: a qualitative analysis of a group exercise at a UK medical school
Pocock, T.M.; Sanders, T.; Bundy, C.
ANALISA PENGUKURAN KINERJA YAYASAN ABC SMA XYZ MENGGUNAKAN METODE INTEGRATED PERFORMANCE MEASUREMENT SYSTEM (IPMS)
ZULKAMAR RIZKI, -
STUDENTS’ SPEAKING PERFORMANCE AT AZ ZUHRA INTEGRATED ISLAMIC SENIOR HIGH SCHOOL PEKANBARU
TIVANI SRI HARTATI, -
AN ANALYSIS OF THE DIFFICULTY OF NATIONAL EXAMINATION TEST ITEMS FOR THE THIRD GRADE STUDENTS OF STATE SENIOR HIGH SCHOOL
DEA ANISA YUSBET
Pengembangan Instrumen Integrated Assessment untuk Mengukur Critical Thinking Skills dan Science Process Skills Kimia Peserta Didik SMA Kelas XI pada Materi Asam dan Basa
Sari, Anggi Ristiyana Puspita
Literatur Lain dari White Rose Research Online
Using item response theory to explore the psychometric properties of extended matching questions examination in undergraduate medical education
Bhakta, B.; Tennant, A.; Horton, M.; Lawton, G.; Andrich, D. · 2005
Using item response theory to explore the psychometric properties of extended matching questions examination in undergraduate medical education
Bhakta, B.; Tennant, A.; Horton, M.; Lawton, G.; Andrich, D. · 2005
Medical Student use of Facebook to Support Preparation for Anatomy Assessments
Pickering, JD; Bickerdike, SR · 2017
The impact of teamwork on peer assessment: a qualitative analysis of a group exercise at a UK medical school
Pocock, T.M.; Sanders, T.; Bundy, C. · 2010
Prehospital Randomised Assessment of a Mechanical Compression Device in out of Hospital Cardiac Arrest (PARAMEDIC): a pragmatic cluster randomised trial and economic evaluation
Gates, S; Lall, R; Quinn, T; Deakin, CD; Cooke, MW; Horton, J; Lamb, SE; Slowther, A-M; Woollard, M; Carson, A; Smyth, A; Wilson, K; Parcell, G; Rosser, A; Whitfield, R; Williams, A; Jones, R; Pocock, H; Brock, N; Black, JJM; Wright, J; Han, K; Shaw, G; B · 2017
Bridging the gap: the roles of social capital and ethnicity in medical student achievement.
Vaughan, S.; Sanders, T.; Crossley, N.; O'Neill, P.; Wass, V. · 2014
Is short-term remediation after OSCE failure sustained? A retrospective analysis of the longitudinal attainment of underperforming students in OSCE assessments
Pell, G; Fuller, R; Homer, M; Roberts, T · 2012
Blended learning approaches enhance student academic performance
Morris, NP · 2010
Literatur Terbaru yang Relevan
Using item response theory to explore the psychometric properties of extended matching questions examination in undergraduate medical education
Bhakta, B.; Tennant, A.; Horton, M.; Lawton, G.; Andrich, D. · 2005
Using item response theory to explore the psychometric properties of extended matching questions examination in undergraduate medical education
Bhakta, B.; Tennant, A.; Horton, M.; Lawton, G.; Andrich, D. · 2005
Medical Student use of Facebook to Support Preparation for Anatomy Assessments
Pickering, JD; Bickerdike, SR · 2017
Pengembangan Instrumen Integrated Assessment antara Keterampilan Proses Sains dan Penguasaan Konsep IPA untuk Mengukur Aspek Kognitif Proses Siswa SMP Kelas VII Semester I
Santoso, Agus · 2015
THE DEVELOPMENT OF A SET OF INSTRUMENT FOR STUDENT PERFORMANCE ASSESSMENT IN THE PHYSICS LABORATORY WORK SUBJECT OF JUNIOR HIGH SCHOOL
Supahar · 2014
Standarisasi Instrumen Integratted Assessment Hasil Belajar Bahasa Konvergen dan Divergen Menggunakan Item Respons Theory 1-PL dengan Program QUEST
Pujiati Suyata, Prof. Dr.; Nur, Hidayanto · 2012
Standarisasi Instrumen Integratted Assessment Hasil Belajar Bahasa Konvergen dan Divergen Menggunakan Item Respons Theory 1-PL dengan Program QUEST
Pujiati Suyata, Prof. Dr.; Nur, Hidayanto · 2012
The impact of teamwork on peer assessment: a qualitative analysis of a group exercise at a UK medical school
Pocock, T.M.; Sanders, T.; Bundy, C. · 2010