The subject-specific operationalization of cognitive activation in primary EFL education. Developing student and teacher questionnaires

by Joel Guttke (University of Duisburg-Essen, Germany)

_______________________________________________________________________

The overarching goal of this PhD project is to develop subject-specific student and teacher questionnaires for assessing cognitive activation in 4th year EFL instruction. This is particularly relevant in light of the Masterplan Grundschule (MSBNRW, 2020) that aims at increasing instructional quality in primary EFL education. The factorial structure of the three generic dimensions of instructional quality has been confirmed repeatedly while findings on their predictive validity on student achievement remain inconsistent (Praetorius et al., 2018). Hence, it has been proposed to reconceptualize cognitive activation in a subject-specific manner (Lipowsky & Bleck, 2019; Wilden, 2021). Research in instructed second language acquisition (ISLA; Loewen, 2015) offers rich findings on factors increasing the quality of language teaching. Consequently, cognitive activation in primary EFL education can be operationalized regarding language input, opportunities for interaction, language output and language awareness. 
 
The research design comprises four stages. First, a review of ISLA research yielded hypotheses on subject-specific aspects of cognitive activation in primary EFL education which serve as the theoretical underpinnings of item construction in both questionnaires. Second, the resulting items will be rated by experts from the field of EFL education. Item comprehensibility will be evaluated as part of cognitive interviews with students (n=10) and primary EFL teachers (n=10). Third, both questionnaires will be revised repeatedly during several pilot studies with students (n=50) and primary EFL teachers (n=25). All analyses will be grounded in classical test theory (Moosbrugger & Kelava, 2020). Fourth, as part of the main study, both questionnaires will be used (a) to compare students’  and  teachers’ perspectives on cognitive activation in EFL instruction with observer ratings and (b) to analyze the impact of cognitive activation on student achievement. 
 
This talk particularly focuses on the results of the first pilot study (April–July 2022). Subject-specific aspects of cognitive activation will be presented by drawing on findings from ISLA research. Sample items will then be introduced to illustrate how cognitive activation was operationalized. Finally, the results of both reliability and item analysis from the first pilot study will be summarized. 
______________________________________________________________________

 
References:

Lipowsky, F. & Bleck, V. (2019). Was wissen wir über guten Unterricht? – Ein Update. In U. Steffens & R. Messner (Hg.), Unterrichtsqualität: Konzepte und Bilanzen gelingenden Lehrens und Lernens (pp. 219–250). Waxmann. 

Loewen, S. (2015). Introduction to Instructed Second Language Acquisition. Routledge. 

Moosbrugger, H. & Kelava, A. (2020). Qualitätsanforderungen an Tests und Fragebogen („Gütekriterien“). In H. Moosbrugger & A. Kelav (Hg.), Testtheorie und Fragebogenkonstruktion (pp. 13–38). Springer.

MSBNRW [Ministerium für Schule und Bildung des Landes Nordrhein-Westfalen] (2020). Masterplan Grundschule: Qualität stärken – Lehrkräfte unterstützen. www.schulministerium.nrw/system/files/media/document/file/Masterplan%20Grundschule.pdf 

Praetorius, A.-K., Klieme, E., Herbert, B. & Pinger, P. (2018). Generic dimensions of teaching quality: the German framework of Three Basic Dimensions. ZDM 50, 407–426. 

Wilden, E. (2021). Fachspezifische Aspekte von Unterrichtsqualität im Schulfach Englisch. Unterrichtswissenschaft 49(2), 211–219.