Diego Cortes, Jeff Dominitz, Maximiliano Romero
arXiv 1 Apr 2025 · Econometrics · publishedJournal of the Royal Statistical Society Series A (Statistics in Society) (2025)
arXiv:2504.01209 · PDF · DOI · OpenAlex
International large-scale assessment (ILSA) studies collect information across education systems with the objective of learning about the population-wide distribution of student achievement in the assessment. In this article, we study one of the most fundamental threats that these studies face when justifying the conclusions reached about these distributions: the identification problem that arises from student non-participation during data collection. Recognizing that ILSA studies have traditionally employed a narrow range of strategies to address non-participation, we examine this problem using tools developed within the framework of partial identification of probability distributions. We tailor this framework to the problem of non-participation when data are collected using a multi-stage stratified random sample design, as in most ILSA studies. We demonstrate this approach with application to the International Computer and Information Literacy Study in 2018. We show how to use the framework to assess mean achievement under reasonable and credible sets of assumptions about the non-participating population. We also provide examples of how these results may be reported by agencies that administer ILSA studies. By doing so, we bring to the field of ILSA an alternative strategy for identification, estimation, and reporting of population parameters of interest.