Skip to content

Information for students, faculty and staff regarding COVID-19. (Updated: 18 January 2021)

printicon

New statistical methods to ascertain high quality over time in standardized achievement tests

Research project The overall aim is to develop new efficient statistical methods within the area of psychometrics, in order to ascertain and guarantee high quality over time in large standardized achievement tests.

By creating a new way to model items we can give more understandable information to test takers. We can obtain fair tests in more situations by extending the framework of equating, i.e. methods used in order to adjust test scores from different test versions so they can be used interchangeably. In line with this we need tools to assess the equating transformations, which will be developed. Finally, to ascertain high quality over time it is planned to develop and adjust quality control methods so they are feasible to use with standardized achievement tests which are typically administrated several times and where the test takers results are valid many years.

Head of project

Project overview

Project period:

2015-01-01 2019-12-31

Funding

The Swedish Research Council, 2015-2018: SEK 4,212,000

Participating departments and units at Umeå University

Umeå School of Business, Economics and Statistics

Research subject

Statistics
  • Project members

    External project members

    Alina von Davier, Strategic advisor
    Jorge González, Associate professor

Project description

The overall aim is to develop new efficient statistical methods within the area of psychometrics, in order to ascertain and guarantee high quality over time in large standardized achievement tests.

1. To develop a new method to model items, especially the items characteristics (e.g. difficulty and discrimination) more efficiently which take care of the limits and challenges of the current classic test theory and item response theory.

2. To extend the observed-score equating framework, i.e. methods used in order to adjust test scores from different test versions so they can be used interchangeably.

3. To propose a statistical assessment of the equating transformation instead of using equating specific criteria. There is still no best way to assess the equating transformation. This is extremely important because which equating method is chosen (based on different selection criteria) has direct impact on the test scores the test takers achieve and thus can have impact if the test is used for admissions to universities.

4. To develop and adjust quality control (QC) methods so they are feasible to use with standardized achievement tests which are typically administrated several times and where the test takers results are valid many years. The focus is on standardized tests which are administered several times and where traditional methods, compiled for relatively few administrations or small amounts of data, are not suitable.

By solving the outlined issues we want to make a strong theoretical and practical contribution to the psychometric field. The research is of great relevance in order to obtain unbiased and fair tests for all test takers. The results are extremely valuable for test constructors, policymakers, teachers, researchers and every test taker who use standardized achievement tests.

The research project is planned to be conducted over five years in three steps. First, analytical statistical models and methods will developed with strong connections to psychometric theory. Second, the obtained analytical models and methods will be evaluated using simulated data and adjusted accordingly. The third step aims at using real standardized test data and obtains a conceptual framework within which test quality can be assured.