NAPLAN Online Trials

The Australian Teacher Magazine

STUDENTS have swapped traditional pencil and paper assessment sheets for computer keyboards to take part in trials of online NAPLAN testing. A whopping 15,000 students from schools in every state and territory volunteered to put the digital delivery method through its paces. The trials are part of an ongoing research project by ACARA into the viability of large scale online assessment. “The main focus is on NAPLAN but it’s important to remember at the same time there’s consideration to building a capability to have other sorts of assessments delivered online as well,” ACARA’s Peter Adams explains. Australia’s federal, state and territory education ministers agreed to online trials in April 2012 and in December last year decided the focus should be on NAPLAN.

Although they targeted 2016 as a possible start date, Adams — general manager for assessment and reporting — says it’s not a ‘fait accompli’. “They said last year that there were a number of things on which they would like more information in order to be able to finally commit to 2016. “… that’s only a proposed date, we’re expecting that ministers will say more at the end of this year.” The online NAPLAN trials for Year 3, 5, 7 and 9 reading, writing and numeracy assessments have been put together by ACARA following consultation with teachers, principals and students.

“There are a number of streams of work and a number of levels of consultation,” Adams says.

“We’re looking at it from an educational and technical dimension. Our work is centring on, first and foremost: What benefits do we see to students and teachers and schools?”

The trials are also looking at whether students give different answers when they complete tests online, the capability of both staff and students to use the technology, and how to make NAPLAN more accessible for students with learning needs and disabilities.

“What we’ve been doing as part of our research is looking at what do we think we can do to enhance the current assessment practice by going into an online environment.

“What do we need to know about the way that students complete assessments and what differences are there between doing an assessment online and doing an assessment on paper?

“We’ve … done live studies with students and schools where we have a look at — if they complete the same work on pencil and paper and then they do it online, are there any differences in the way they go about it?

“Do they come up with different material? Is there a difference in the level of performance they demonstrate depending on the mode?

“We’ve also looked at what sorts of things kids can do at, say, Year 3 — can they use a keyboard for example?

“We also need to look at how can we link data that comes out of an online testing system with data that’s been developed [from] a pencil and paper system.

“We think on the work and research we’ve done we’ll be able to increase the level of accessibility for students with special requirements — and that’s a really good thing.

“We’re also looking at how we might use the technology to improve the types of items that we can use; or, to use the slightly more complex or sophisticated items — because there are things you can do electronically that you can’t do on pencil and paper.”

Adams says one particularly exciting aspect of the research relates to tailored test design. In a test broken into three sections, students complete a set of common questions and their performance on those affects the difficulty of the next set, and so on.

“Now, the value of this tailored test design for NAPLAN is that we can better target kids’ ability in the testing so that high performing students are going to get the opportunity to demonstrate their ability by completing more difficult, complex and challenging items.

“And for the less able students we can give them more questions that are suitable to their level of ability at the time of the test, and therefore they engage more with it and most importantly we can learn more about what they can do and what they know.

“So, tailored test design is very exciting. There’s a lot of interest in it from teachers, principals, researchers, administrators and people in education departments. It’s a key focus within the research program that we’re doing.

“We do see kids disengage early with the test, and that’s not good for their confidence, it doesn’t give them the opportunity to demonstrate what they know and we learn less about them, so we’re very strongly of the view that [tailored test design] is going to be a really good thing.”

Adams says students trialling the new systems can talk about their experiences and give feedback on what they did and didn’t enjoy through ‘cognitive laboratories’.

ACARA is confident that moving the tests online will lead to richer data on student performance and some of that data will be returned to teachers much quicker.

“I’m not saying that we’ll get everything that we currently give back much quicker, but we’re looking at what we can give back to schools very soon after the tests,” Adams says.

The research team is looking at automated scoring programs being used internationally and exploring the possibility of using the technology in combination with professional markers.

Another important stream of the work is technical readiness — in relation to schools and teachers.

“… one of the things we’re really aware of is some schools are much better equipped with technology and infrastructure than others … so we’re looking at how we can do it technically.

“We’ve got four levels of possible technology and are going around the country at the moment talking to the government and non-government sectors.

“We could have a web-based, that’s our preference, but we know in some circumstances you can’t say that, you could put it on a local server, or you could use some other technique.

“The other thing we’re putting a strong focus on is learning what we can about how teachers feel about this, how comfortable they are in managing students doing an online test, what assistance they might need, what are their main concerns and how can we best meet them.

“Because, in a NAPLAN test for example online, the teacher is going to be the main person administering the test, so they’ve got to be comfortable about what they’re doing, know how to do it and have relevant support available.

“We’d like to think that we’re being realistic and informing ourselves of all of the challenges as well as the benefits.”

In addition to the NAPLAN trials, ACARA is running an online program that will see 14,000 Year 6 and 10 students across the country take the Civics and Citizenship assessment in October.

 http://www.ozteacher.com.au/technology/naplan-online-trials/24356