Method development for analysing and evaluating computerized achievement tests

Research project The overall aim is to develop the theory for analysing and evaluating computerized achievement tests in order to create tests which are fair for all test takers

The overall aim is to develop the theory for analysing and evaluating computerized achievement tests in order to create fare tests. The project aims to develop new methods for special test types, e.g. sequential tests nd licensing tests. Further, to develop a method to equate test versions which have common items, using information from the test takers. The project aim to examine the possible error which occur when complex tests which measure more than one ability are evaluated with methods developed for tests which only measure one ability. Finally, the project aim to generate a new method to estimate the probability to answer an item correct and to examine the items characteristics.

Head of project

Marie Wiberg Professor

+46 90 786 95 24

Project overview

Project period:

2007-01-01 – 2009-12-31

Funding

Finansår , 2007, 2008, 2009

huvudman: Marie Wiberg, finansiär: vetenskapsrådet, y2007: 400, y2008: 400, y2009: 400,

Participating departments and units at Umeå University

Umeå School of Business, Economics and Statistics

Research subject

Statistics

Project description

The overall aim is to develop the theory for analysing and evaluating computerized achievement tests in order to create fare tests. Methods of DIF, i.e. if two groups (e.g. men and women) of test takers with the same ability gives different answer to an item, will be examined and developed for licensing tests. The loss of information will be examined when tests which measure several dimensions of a test-taker's ability are analysed with simple models, which are based on the assumption of one dimension. A method for equating two test versions which have some items in common will be created. Statistical inference theory and functional data analysis will be integrated with psychometrics in order to estimate the probability to answer an item correctly when the test-takers' ability are viewed as nuisance parameters. Analytical models will be created and tested through simulations where the parameters are known. Then, the methods will be applied to standardized tests, e.g. the Swedish SAT, national tests and the driving-license test.