The Australian Teacher Magazine
STUDENTS have swapped traditional pencil and paper
assessment sheets for computer keyboards to take part in trials of
online NAPLAN testing. A whopping 15,000 students from schools in every
state and territory volunteered to put the digital delivery method
through its paces. The trials are part of an ongoing research project by
ACARA into the viability of large scale online assessment. “The main focus is on NAPLAN but it’s important to
remember at the same time there’s consideration to building a capability
to have other sorts of assessments delivered online as well,” ACARA’s
Peter Adams explains. Australia’s federal, state and territory education
ministers agreed to online trials in April 2012 and in December last
year decided the focus should be on NAPLAN.
Although they targeted 2016 as a possible start date,
Adams — general manager for assessment and reporting — says it’s not a
‘fait accompli’. “They said last year that there were a number of things
on which they would like more information in order to be able to
finally commit to 2016. “… that’s only a proposed date, we’re expecting that ministers will say more at the end of this year.” The online NAPLAN trials for Year 3, 5, 7 and 9
reading, writing and numeracy assessments have been put together by
ACARA following consultation with teachers, principals and students.
“There are a number of streams of work and a number of levels of consultation,” Adams says.
“We’re looking at it from an educational and
technical dimension. Our work is centring on, first and foremost: What
benefits do we see to students and teachers and schools?”
The trials are also looking at whether students give
different answers when they complete tests online, the capability of
both staff and students to use the technology, and how to make NAPLAN
more accessible for students with learning needs and disabilities.
“What we’ve been doing as part of our research is
looking at what do we think we can do to enhance the current assessment
practice by going into an online environment.
“What do we need to know about the way that students
complete assessments and what differences are there between doing an
assessment online and doing an assessment on paper?
“We’ve … done live studies with students and schools
where we have a look at — if they complete the same work on pencil and
paper and then they do it online, are there any differences in the way
they go about it?
“Do they come up with different material? Is there a
difference in the level of performance they demonstrate depending on the
mode?
“We’ve also looked at what sorts of things kids can do at, say, Year 3 — can they use a keyboard for example?
“We also need to look at how can we link data that
comes out of an online testing system with data that’s been developed
[from] a pencil and paper system.
“We think on the work and research we’ve done we’ll
be able to increase the level of accessibility for students with special
requirements — and that’s a really good thing.
“We’re also looking at how we might use the
technology to improve the types of items that we can use; or, to use the
slightly more complex or sophisticated items — because there are things
you can do electronically that you can’t do on pencil and paper.”
Adams says one particularly exciting aspect of the
research relates to tailored test design. In a test broken into three
sections, students complete a set of common questions and their
performance on those affects the difficulty of the next set, and so on.
“Now, the value of this tailored test design for
NAPLAN is that we can better target kids’ ability in the testing so that
high performing students are going to get the opportunity to
demonstrate their ability by completing more difficult, complex and
challenging items.
“And for the less able students we can give them more
questions that are suitable to their level of ability at the time of
the test, and therefore they engage more with it and most importantly we
can learn more about what they can do and what they know.
“So, tailored test design is very exciting. There’s a
lot of interest in it from teachers, principals, researchers,
administrators and people in education departments. It’s a key focus
within the research program that we’re doing.
“We do see kids disengage early with the test, and
that’s not good for their confidence, it doesn’t give them the
opportunity to demonstrate what they know and we learn less about them,
so we’re very strongly of the view that [tailored test design] is going
to be a really good thing.”
Adams says students trialling the new systems can
talk about their experiences and give feedback on what they did and
didn’t enjoy through ‘cognitive laboratories’.
ACARA is confident that moving the tests online will
lead to richer data on student performance and some of that data will be
returned to teachers much quicker.
“I’m not saying that we’ll get everything that we
currently give back much quicker, but we’re looking at what we can give
back to schools very soon after the tests,” Adams says.
The research team is looking at automated scoring
programs being used internationally and exploring the possibility of
using the technology in combination with professional markers.
Another important stream of the work is technical readiness — in relation to schools and teachers.
“… one of the things we’re really aware of is some
schools are much better equipped with technology and infrastructure than
others … so we’re looking at how we can do it technically.
“We’ve got four levels of possible technology and are
going around the country at the moment talking to the government and
non-government sectors.
“We could have a web-based, that’s our preference,
but we know in some circumstances you can’t say that, you could put it
on a local server, or you could use some other technique.
“The other thing we’re putting a strong focus on is
learning what we can about how teachers feel about this, how comfortable
they are in managing students doing an online test, what assistance
they might need, what are their main concerns and how can we best meet
them.
“Because, in a NAPLAN test for example online, the
teacher is going to be the main person administering the test, so
they’ve got to be comfortable about what they’re doing, know how to do
it and have relevant support available.
“We’d like to think that we’re being realistic and informing ourselves of all of the challenges as well as the benefits.”
In addition to the NAPLAN trials, ACARA is running an online program
that will see 14,000 Year 6 and 10 students across the country take the
Civics and Citizenship assessment in October.
http://www.ozteacher.com.au/technology/naplan-online-trials/24356
