Note: This is the second of a two-part post on the challenges faced by principals implementing online testing tied to the Common Core and new college- and career-ready standards. So much has happened in recent weeks that I divided the entry into two parts because one post would not do justice to the topic.
In part 1, I described that with the spring testing season now winding down, principals in a number of states feel as though they are under siege. For some schools, whatever could go wrong has gone wrong.
From my contact with principals in a number of states and my ongoing work with principals in schools in five states, I have learned that online assessments present principals with a number of new and old challenges.
I divided this post into two parts. Part 1 addressed no-technical challenges principals face in implementing the new assessments. This entry will address the technical issues.
Following are eight technical challenges school leaders face in implementing online testing related to the Common Core and new college- and career-ready standards:
The transition from paper-and-pencil to online assessments takes several years before it becomes normal.
We learned from previous experiences with online instruction that it usually took time to work out the technical problems. So, when we began high-stakes, online testing, we expected problems. We prepared for problems, and we learned from the problems. In other words, online testing was rarely perfect. Our school made the transition from paper and pencil tests to total online testing. The four-year transition to online tests was full of problems from which we learned a number of useful lessons. Having gone through that painful change, I cannot imagine making that switch in a single year. I spent a decade under online testing. If everything else remained the same, the move from paper to online testing would be a huge challenge, but this is only one of many testing changes.
Cheating Scandals Place a Premium on Test Security
Cheating scandals around the country have brought more and more scrutiny on how school leaders manage the testing process. When we had paper and pencil tests, I was paranoid about test security. I had workers install a special lock on a storage room with no outside windows that we used to store the tests. Two people had keys to the room–the district locksmith and our testing coordinator. I never saw or touched a test in all the years we gave paper and pencil tests.
I was relieved when we went to online tests, because the security was much better and easier to manage. We had guidelines for how teachers were to supervise the assessments and each proctor signed an agreement.
Better tests cost more and take more time to administer.
In the past, many states chose the cheapest and least effective tests money could buy, but held schools, teachers, and principals to the highest levels of accountability. “Improve your test scores or we will find someone who can,” a district leader once said to me.
One state spent less that $12 per student on annual assessments and later withdrew from one of the two testing consortia because the cost of the tests was too high. In fact, the average per student spending of the states in the consortia was approximately $29. Incidentally, that same state also has one of the highest gaps between state proficiency scores and proficiency on NAEP assessments, but had high-stakes accountability measures for schools and teachers.
For many schools without adequate technology, logistics are a nightmare.
One principal told me that it took months to plan the movement of students to operating computers during the testing period. The more tests and students a school tests, the more challenging the planning process. Most schools have to shut down their libraries and labs and devote their entire inventory of computer hardware to testing for an entire month.
Lack of Hardware and Software
“Fewer than 30 percent of K-12 school technology leaders believed their district was ready for online assessments, according to an annual survey by the Consortium for School Networking.” Yes, we were told that old computers and operating systems would run the new tests, the reality is that they do not run well. Principals tell me that old operating systems with old network cards frequently lose network connections.
In addition, one principal in a state with years of online testing experience learned that, while tablets and Chromebooks worked with the old multiple-choice tests, he had to purchase computer mice because his students became frustrated because they could not navigate the tests and use the test tools without them.
The move to online testing has exposed the lack of technology in many schools. In Oklahoma, only 20% of schools reported having enough technology to participate in online assessments. Overall, “63 percent of public schools don’t have access to broadband speeds needed for digital learning. The problem is particularly acute in rural and low-income districts: Only 14 percent in those areas meet high-speed Internet targets.”
In our school, we conducted speed tests on every device and we performed stress tests on our network throughout the school every year before the testing season. We would set up a number of computers in a room using a continuous ping test, which would overload the network. We found that, from year to year, network access points would go bad, need to be reset, or become overloaded due to changes in course offerings and room assignments.
Need to Build Student Tech Skills
“Many schools have stepped up their work on computer skills because their states’ paper-and-pencil tests are moving online. “Their primary concern is ensuring that students have access to these technology tools prior to sitting down for online tests.”
Context matters! We learned that the lack of familiarity with online tests and the skills needed to navigate the tests was hurting our students. In fact, some find that student performance actually drops with initial administration of online tests. If our students were going to succeed, they needed regular exposure to online assessments throughout the school year. As most states shift their required tests to computers, teachers are discovering that their students are missing key technical skills needed to show what they know.
We believed that our students tested better in familiar environments—their classrooms. While this is not convenient, it did result in significant improvement in student attitude and performance.
One expert advises that schools integrate the testing “into the regular classroom schedule by stopping the test when needed and picking it back up again at a convenient time in the future (within test parameters) until each student is finished. The advantages that he outlined for this approach include avoiding shutting down the school so everyone can take a test, giving more tests and using less testing time, and taking into account student fatigue. He advised educators to stop treating online tests like paper tests because “locking a kid in a room for 2 ½ hours with a bubble sheet is not the gold standard of validity.”
The rash of vendor failures nationwide has undermined schools and districts that have made huge financial commitments to upgrade their hardware and infrastructure. For example, Virginia, which is not a Common Core state, but has over a decade of experience with online testing using Pearson as the vendor, has surprisingly experienced serious interruptions and failures proving that even the most experienced vendors are undergoing overload with the mass migration to online testing.
Some states provide a hedge against vendor server failures. For example, Virginia “allows the local storage of test answers as students move through the test so no completed answers are lost, thus reducing the impact of network and local connectivity issues on students already testing.
The bottom line is that, with the exception of the massive vendor server failures, we expected problems. However, experience has taught me two important lessons. First, online assessment is worth the effort. Second, each year we found fewer problems.