Tuesday, April 6, 2021

On Educational Testing Services And A Rather Impressive and Lucrative History of Failure

Each year in the Spring students across Texas take a standardized test known as the State of Texas Assessments of Academic Readiness, or STAAR test. The tests you take (and have to pass) vary by grade level, and start as early as 3rd Grade, with Math and Reading. The following grades take the following state standardized tests:

4th: Math, Reading, Writing

5th: Math, Reading, Science

6th: Math, Reading

7th: Math, Reading, Writing

8th: Math, Reading, Science, Social Studies

9th-12th: Algebra I, English I, English II, Biology, US History

Essentially these cover the core classes: Math, English, Science, Social Studies. It’s possible to get a waiver in high school depending on the student’s status but in order to go from 5th to 6th Grade and 8th to 9th Grade, students have to pass the Math and Reading exams. You get three tries every year.

According to TexasAssessment.gov, these tests are “designed to measure what students are learning in each grade and whether or not they are ready for the next grade.” The State also notes that the tests will be given in both paper and online format. And that’s where things went crazy this morning across Texas.

The Texas Education Agency runs the show. In the past TEA (which is headed by, wait for it, former software developer Mike Morath) had outsourced state standardized testing to Pearson Education for 35 years. Why dump Pearson after such a long relationship with the State of Texas? A 2013 state audit of Pearson “found holes in state oversight, as well as lax terms that allowed the company to hire former state employees without restrictions or disclosure.”

So beginning with the 2015-16 school year TEA awarded a four-year/$280 million contract to the New Jersey-based Educational Testing Services (ETS) to handle the STAAR test. Before the first tests were even administered, ETS “delivered tests to wrong addresses and prefilled answer sheets with incorrect student information. Confidential student data, including Social Security numbers, were sent to the wrong school districts.” By the time the first round of testing took place in Spring 2016, things were decidedly not better.

A March 2016 computer glitch caused students to lose their answers on over 14,000 tests statewide, affecting mostly Special Education and English Language Learner students who were taking their tests online. TEA had to waive the results from the annual school accountability ratings for that year. Later in the Spring, ETS temporarily misplaced a number of tests from Eanes ISD, a high-performing district on Austin’s west side. In June 2016, ETS missed the deadline to give “some school districts” the scores from the summer re-test, which schools use to determine which students need summer school. Morath, and TEA, left the decision to promote (or not) students from 5th and 8th grades up to the school districts, since the data wasn’t there to make the call.

All of these issues led TEA to fine ETS – in their first year overseeing the STAAR tests – $21 million ($6 million in damages + $15 million of their own money to fix the problems encountered that Spring) and said that if ETS didn’t get their crap together by December 2017, they could come back around for some for right-nice fining.

I’m sure you will be shocked to learn that it did, in fact, happen again. Students taking online versions of the STAAR in April and May 2018 found themselves logged out of the testing server, or kicked out of the testing platform altogether. 71,000 test results were thrown out, over 100,000 more students were affected by the glitch. ETS blamed two things: human error and issues with the server stemming from hundreds of thousands of students trying to log in at the same time (which is required at the behest of TEA).

In June 2018 TEA issued another $100,000 fine on ETS and announced they would put the contract up for re-bidding once ETS’ contract ended in August 2019 which TEA apparently awarded to...ETS. When the COVID-19 pandemic closed schools across the country for in-person learning in March 2020, Texas Governor Greg Abbott waived STAAR testing requirements for the 2019-2020 school year, seeing as how it was clear many schools would be closed during the testing window. But what about the 2020-2021 school year?

As with most anything before the Texas Legislature, the decision turned quickly partisan: Democrats called on Abbott and Morath/TEA to cancel STAAR testing, saying that any standardized test that comes after a year of a global pandemic isn’t going to provide valid data relating to student, and school, performance. Republicans said that the test was needed to see how far behind (or not) students/schools are after the COVID year. I’m not here to weigh in on the validity of these arguments.

In February 2021 TEA issued new guidance on the 2021 STAAR tests as many school districts have allowed students to learn remotely as well as in-person. Namely:

-Yes, students have to take them, but! the requirement to pass the STAAR in 5th and 8th grade in order to move to 6th and 9th grade have been waived.

-High school seniors still needing to pass any of their 9th-11th grade tests (the latest test, high school timeline-wise, is 11th grade, and it’s US History. You don’t have to take any “new” STAAR tests during your senior year) to graduate have to do so.

-Also students take the STAAR test on a school-issued laptop with the testing software pre-loaded, but they take the test in a classroom on campus.

-But STAAR test results will not factor into the dreaded district accountability ratings.

Which brings us to this morning, April 6, 2021.

I was all set to administer the English I STAAR test. Out of 15 students on my list, ten showed up. Two of those ten didn’t have their school-issued laptop to access the test and had to be relocated to the Library. No worries. As I’m reading the ever-so-formal instructions and get to the part where they log in with their respective usernames and passwords, every single one of the students taking their test said their login was rejected. Stepping out into the hallway, pretty much every other test administrator was doing the same thing, wondering what had happened.

ETS’ online testing system had crashed due to the number of students trying to log in at the exact same time…just like it did in 2018.

It did not take terribly long for TEA to issue a statement: “What happened today is completely unacceptable,” the press release said. “ETS, the testing vendor, experienced problems with their database system, which are in the process of being corrected. The 2021 online administration of STAAR will be ETS’s last for the State of Texas. Beginning next year, Cambium Assessment will be taking over these critical testing functions to ensure that users have a seamless online testing experience moving forward. All involved in public education in Texas should expect better than what they have experienced today; we are working to ensure that our students do not experience future testing issues.”

That gives ETS about 43 hours to get their servers (which have had issues for going on three years now) straightened out: the English II STAAR test is scheduled for Thursday morning, April 8. I'm sure it'll be fine.

Whatever happens from today until August 31, when ETS’ contract will be allowed to expire, we know that online testing is here to stay: Beginning September 1 Cambium Assessment (who was awarded a three-year $262 million contract to “manage the administration, scoring and reporting on an online testing platform”) and old testing friend Pearson (three years, $126 million to develop the assessment tests) will run the show.

ETS screwed up early, and they screwed up often. But at least they got a ton of money to do it.