Tailored tests

ACARA has developed a tailored test design, which could be applied to NAPLAN online. The 'tailored test' is a multistage, branching test design that delivers different sets of questions ('testlets') to students depending on their achievement on previous questions. Online delivery of assessments allows the use of a tailored test design to better match questions to an individual student's ability in a way that is not feasible for paper-based testing.

The tailored test design for reading and numeracy was developed and trialled in schools in 2013. Reading and numeracy tests had a number of testlets. Each testlet was developed as a mini-test. Each student completed three testlets.

The diagram and discussion below illustrate a proposal for the potential paths that students may take through NAPLAN online reading and numeracy tests:

All students at each year level start with the same set of questions (testlet A). The computer system scores the student's answers automatically and the student then progresses to a second testlet. The second testlet may be easier (B) or more difficult (D) than testlet A, depending on the student's achievement.

At the end of the second testlet, the student is directed to a third testlet, again depending on achievement. The final testlets are of varying difficulty: hard (F), medium (E), easy (C).

Proposed tailored test designs for grammar and punctuation, and spelling will be trialled in 2014.

Benefits of this tailored test design include the following:

• Tailored tests provide a more precise measurement of student performance. This design allows for greater differentiation of students by using a wider range of question difficulty, without adding to the length of the test for each individual student.
• Trials of the tailored test design suggest that students are more engaged with tests that adapt to their achievement. Students who struggle with the initial set of questions are given questions that are more suited to their ability, so that students are less likely to become discouraged as they progress through the tests. High-achieving students are given questions that better challenge their abilities.
• The tailored test design has the potential to reduce anxiety in students who may find the current paper-based format of NAPLAN too challenging.
• While each student will still answer the same number of questions as for the paper tests, the number of items in the whole test is larger. As a result, a wider range of aspects of the curriculum can be tested across the NAPLAN cohort.

ACARA's online assessment research continues with a second trial of the tailored test design in 2014. The trial will finalise the targeting and measurement aspects of the tailored test design. The research will also investigate how testlets will be trialled for the new design and how the testlets will be used to construct NAPLAN online tests. Fully branching tailored tests in reading and numeracy will be trialled as well as new options for grammar and punctuation, and spelling.
For more information about upcoming trials, please contact: onlineresearch@acara.edu.au.