TY - JOUR
T1 - Investigating the characteristics of language test specifications and item writer guidelines, and their effect on item development
T2 - a mixed-method case study
AU - Al Lawati, Zahra Ali
N1 - Publisher Copyright:
© 2023, The Author(s).
PY - 2023/12
Y1 - 2023/12
N2 - This study discusses the characteristics of test specifications (specs) and item writer guidelines (IWGs), their role in item development of English as a Second Language (ESL) reading tests, and the use of the CEFR for specs development. This mixed-method study analyzed specs, IWGs, tests, and the Pearson Test of English General test statistics. Moreover, interviews and focus groups were conducted with the specs’ developers, IWGs, and item writers. The findings show no unique way of conceptualizing specs and IWGs. Moreover, translating the CEFR reading descriptors into specs is a challenging task. However, results from the judgmental study and item statistics suggest that the investigated specs and IWGs facilitated the development of good-quality items at a certain difficulty level. This study reveals the potential role of specs and IWGs in establishing test validity. This research contributes to understanding the under-researched area of specs and IWGs and shows the type of information required for effective item writing and ways of enhancing the validity and reliability of tests. Practical and theoretical suggestions and future research have also been identified.
AB - This study discusses the characteristics of test specifications (specs) and item writer guidelines (IWGs), their role in item development of English as a Second Language (ESL) reading tests, and the use of the CEFR for specs development. This mixed-method study analyzed specs, IWGs, tests, and the Pearson Test of English General test statistics. Moreover, interviews and focus groups were conducted with the specs’ developers, IWGs, and item writers. The findings show no unique way of conceptualizing specs and IWGs. Moreover, translating the CEFR reading descriptors into specs is a challenging task. However, results from the judgmental study and item statistics suggest that the investigated specs and IWGs facilitated the development of good-quality items at a certain difficulty level. This study reveals the potential role of specs and IWGs in establishing test validity. This research contributes to understanding the under-researched area of specs and IWGs and shows the type of information required for effective item writing and ways of enhancing the validity and reliability of tests. Practical and theoretical suggestions and future research have also been identified.
KW - AUA
KW - CEFR
KW - ESL reading tests
KW - Item writer guidelines
KW - Pearson Test of English General (PTE General)
KW - Test specifications
UR - http://www.scopus.com/inward/record.url?scp=85153213399&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85153213399&partnerID=8YFLogxK
U2 - 10.1186/s40468-023-00233-5
DO - 10.1186/s40468-023-00233-5
M3 - Article
AN - SCOPUS:85153213399
SN - 2229-0443
VL - 13
JO - Language Testing in Asia
JF - Language Testing in Asia
IS - 1
M1 - 21
ER -