On Educational Testing Services And A Rather Impressive and Lucrative History of Failure

Each year in the Spring students across Texas take a standardized test known as the State of Texas Assessments of Academic Readiness, or STAAR test. The tests you take (and have to pass) vary by grade level, and start as early as 3^rd Grade, with Math and Reading. The following grades take the following state standardized tests:

4^th: Math, Reading, Writing

5^th: Math, Reading, Science

6^th: Math, Reading

7^th: Math, Reading, Writing

8^th: Math, Reading, Science, Social Studies

9^th-12^th: Algebra I, English I, English II, Biology, US History

Essentially these cover the core classes: Math, English, Science, Social Studies. It’s possible to get a waiver in high school depending on the student’s status but in order to go from 5^th to 6^th Grade and 8^th to 9^th Grade, students have to pass the Math and Reading exams. You get three tries every year.

According to TexasAssessment.gov, these tests are “designed to measure what students are learning in each grade and whether or not they are ready for the next grade.” The State also notes that the tests will be given in both paper and online format. And that’s where things went crazy this morning across Texas.

The Texas Education Agency runs the show. In the past TEA (which is headed by, wait for it, former software developer Mike Morath) had outsourced state standardized testing to Pearson Education for 35 years. Why dump Pearson after such a long relationship with the State of Texas? A 2013 state audit of Pearson “found holes in state oversight, as well as lax terms that allowed the company to hire former state employees without restrictions or disclosure.”

So beginning with the 2015-16 school year TEA awarded a four-year/$280 million contract to the New Jersey-based Educational Testing Services (ETS) to handle the STAAR test. Before the first tests were even administered, ETS “delivered tests to wrong addresses and prefilled answer sheets with incorrect student information. Confidential student data, including Social Security numbers, were sent to the wrong school districts.” By the time the first round of testing took place in Spring 2016, things were decidedly not better.

A March 2016 computer glitch caused students to lose their answers on over 14,000 tests statewide, affecting mostly Special Education and English Language Learner students who were taking their tests online. TEA had to waive the results from the annual school accountability ratings for that year. Later in the Spring, ETS temporarily misplaced a number of tests from Eanes ISD, a high-performing district on Austin’s west side. In June 2016, ETS missed the deadline to give “some school districts” the scores from the summer re-test, which schools use to determine which students need summer school. Morath, and TEA, left the decision to promote (or not) students from 5^th and 8^th grades up to the school districts, since the data wasn’t there to make the call.

All of these issues led TEA to fine ETS – in their first year overseeing the STAAR tests – $21 million ($6 million in damages + $15 million of their own money to fix the problems encountered that Spring) and said that if ETS didn’t get their crap together by December 2017, they could come back around for some for right-nice fining.

I’m sure you will be shocked to learn that it did, in fact, happen again. Students taking online versions of the STAAR in April and May 2018 found themselves logged out of the testing server, or kicked out of the testing platform altogether. 71,000 test results were thrown out, over 100,000 more students were affected by the glitch. ETS blamed two things: human error and issues with the server stemming from hundreds of thousands of students trying to log in at the same time (which is required at the behest of TEA).

In June 2018 TEA issued another $100,000 fine on ETS and announced they would put the contract up for re-bidding once ETS’ contract ended in August 2019 which TEA apparently awarded to...ETS. When the COVID-19 pandemic closed schools across the country for in-person learning in March 2020, Texas Governor Greg Abbott waived STAAR testing requirements for the 2019-2020 school year, seeing as how it was clear many schools would be closed during the testing window. But what about the 2020-2021 school year?

As with most anything before the Texas Legislature, the decision turned quickly partisan: Democrats called on Abbott and Morath/TEA to cancel STAAR testing, saying that any standardized test that comes after a year of a global pandemic isn’t going to provide valid data relating to student, and school, performance. Republicans said that the test was needed to see how far behind (or not) students/schools are after the COVID year. I’m not here to weigh in on the validity of these arguments.

In February 2021 TEA issued new guidance on the 2021 STAAR tests as many school districts have allowed students to learn remotely as well as in-person. Namely:

-Yes, students have to take them, but! the requirement to pass the STAAR in 5^th and 8^th grade in order to move to 6^th and 9^th grade have been waived.

-High school seniors still needing to pass any of their 9^th-11^th grade tests (the latest test, high school timeline-wise, is 11^th grade, and it’s US History. You don’t have to take any “new” STAAR tests during your senior year) to graduate have to do so.

-Also students take the STAAR test on a school-issued laptop with the testing software pre-loaded, but they take the test in a classroom on campus.

-But STAAR test results will not factor into the dreaded district accountability ratings.

Which brings us to this morning, April 6, 2021.

I was all set to administer the English I STAAR test. Out of 15 students on my list, ten showed up. Two of those ten didn’t have their school-issued laptop to access the test and had to be relocated to the Library. No worries. As I’m reading the ever-so-formal instructions and get to the part where they log in with their respective usernames and passwords, every single one of the students taking their test said their login was rejected. Stepping out into the hallway, pretty much every other test administrator was doing the same thing, wondering what had happened.

ETS’ online testing system had crashed due to the number of students trying to log in at the exact same time…just like it did in 2018.

It did not take terribly long for TEA to issue a statement: “What happened today is completely unacceptable,” the press release said. “ETS, the testing vendor, experienced problems with their database system, which are in the process of being corrected. The 2021 online administration of STAAR will be ETS’s last for the State of Texas. Beginning next year, Cambium Assessment will be taking over these critical testing functions to ensure that users have a seamless online testing experience moving forward. All involved in public education in Texas should expect better than what they have experienced today; we are working to ensure that our students do not experience future testing issues.”

That gives ETS about 43 hours to get their servers (which have had issues for going on three years now) straightened out: the English II STAAR test is scheduled for Thursday morning, April 8. I'm sure it'll be fine.

Whatever happens from today until August 31, when ETS’ contract will be allowed to expire, we know that online testing is here to stay: Beginning September 1 Cambium Assessment (who was awarded a three-year $262 million contract to “manage the administration, scoring and reporting on an online testing platform”) and old testing friend Pearson (three years, $126 million to develop the assessment tests) will run the show.

ETS screwed up early, and they screwed up often. But at least they got a ton of money to do it.

The Valentine's Day (Electricity) Massacre

Let's start by noting that on February 8, the guys at Space City Weather were raising the possibility for potential winter weather in Houston early in the week of February 15. By February 10, they were sounding the alarms. And that was just for Houston. So the idea that the state was caught completely off-guard is disingenuous at best. Why have millions in Texas been without power for multiple days now, trapped in the grip of the coldest weather in decades, 10 years after a similar power crisis? [cracks knuckles]

There are three electric grids in the lower 48 states: the Eastern Interconnection, the Western Interconnection, and then there's Texas. Texas' electricity is managed by ERCOT - the Electric Reliability Council of Texas. ERCOT is technically a non-profit organization that controls 40,000 miles of electricity transmission lines to over 22 million Texans in 75% of the state. Some of the Panhandle isn't serviced by ERCOT, neither is El Paso, or the far East Texas border. 

ERCOT does not generate electricity or manage the state's electrical grid. ERCOT simply schedules power and "ensures the reliability of the electrical network." They didn't do much of either this week. It started on Sunday night, February 14, around 11pm. As temperatures across the state plummeted, energy generating units started switching off thanks to the fact that most did not winterize their units. Essentially, the systems used to generate power literally froze up. With demand for electricity high (people were cold), the supply plummeted. To counter this gap, ERCOT "asked transmission providers to turn off large industrial users that had previously agreed to be shut down. But the situation deteriorated quickly," according to WFAA. At the order of "state power regulators," CenterPoint Energy - which delivers electricity and natural gas to homes - were told to begin rolling blackouts for the Houston area. By 1:30am on Monday, February 15, ERCOT had implemented these rolling outages, and then couldn't get them actually rolling. If you lost power, you were basically screwed, because ERCOT couldn't manage them. By 8am on Monday, February 15 Houston mayor Sylvester Turner was telling Houston residents that there were system-wide power outages across the state. The majority of the power that was knocked offline wasn't wind power (while most West Texas wind turbines did freeze up, that doesn't seem to be a problem with the three wind turbines in Antarctica, or the ones in the Alps), the issue was with oil and natural gas, both of which are "regulated" by the Railroad Commission of Texas. An emergency order from the Railroad Commission on the night of Friday, February 12 called for the management of shortages of natural gas, whose pipelines had frozen thanks, in part, to a lack of winterization protection.

To clarify: Power plants generate electricity from oil, coal, natural gas, nuclear, wind, solar, and any other forms of energy. ERCOT "manages the flow" of that electricity within its network, while energy companies like CenterPoint maintain the method of delivery (i.e., power lines and poles). They're separate entities. Then you have Reliant, which markets and sells that electricity to you, the consumer, though Reliant is not responsible for the generation of power, nor the delivery of power. This is a result of 1999's Senate Bill 7, which we'll get to in a bit. 

How much power does it take to generate electricity for a house? At one point over half of the state's energy generators were offline, an estimated 45 gigawatts (or 45,000 megawatts). One megawatt of electricity can power 500 homes. So if 45,000 megawatts are offline, that removes enough electricity to power 22.5 million homes. 

Enter the Federal Energy Regulatory Commission. FERC was established in 1977 to replace the Federal Power Commission, which was established in the 1920s to regulate hydropower dams. FDR expanded FERC's power in the 1930s under the Federal Power Act to dismantle utility monopolies during the Great Depression and to set "just and reasonable" wholesale electricity prices, regulate pricing and order refunds for overcharges by utility companies across state lines. Today FERC regulates the United States' natural gas industry, hydroelectric projects, oil pipelines, and wholesale rates. But Texas isn't part of that, and that goes back to FDR. The Federal Power Commission was clear that it brought oversight to "interstate" commerce of electricity. If Texas had its own grid, they could avoid federal oversight. Richard D. Cudahy wrote in 1995, "freedom from federal regulation was a cherished goal - more so because Texas had no regulation until the 1970s." ERCOT - formed in 1970 - doesn't have to abide by federal regulations because it's not interstate commerce, so it takes its orders from the Texas State Legislature and the Governor so as not to follow federal guidelines and regulations. 

Texas has the only deregulated power market in the entire United States, meaning that it's your basic laissez-faire economic market, trading on supply-and-demand. If you're a power company, the only thing that matters to ERCOT is how cheaply you can provide it, and how much ERCOT can charge to sell it. And if you - the energy company - are running a narrow profit margin, there's not much money left over to invest in your infrastructure like, oh I don't know, winterizing your pipelines. This actually works both ways: when it's freezing cold, Texas can't import energy from one of the other two electric grids in the United States. When it's hot in the summer and things are moving along just fine for energy production, Texas can't really export its surplus energy to help out, say, California. And because Texas' power market is deregulated, supply is based on free market pricing, exactly the thing that the FDR's Federal Power Commission was created to stop.

On December 23, 1989 a cold wave consumed the central part of the United States that dropped temperatures to -42 in at Scottsbluff, Nebraska, -21 in Springfield, Illinois and sent almost all of Texas into the single-digits. Busted pipes caused $25 million of damage in Dallas alone. North and South Carolina got a foot of snow. ERCOT implemented rolling power outages for the first time during this  storm. The Public Utility Commission of Texas issued a report in the months following the 1989 storm that recommended "all utilities should ensure that they incorporate the lessons learned during December of 1989 into the design of new facilities" and "ensure that procedures are implemented to correct defective freeze protection equipment prior to the onset of cold weather." These recommendations were not mandatory, and were allowed to lapse as time passed.

Ten years after the 1989 winter storm, the Texas State Legislature passed Senate Bill 7. It broke up most of the state's public utilities. Where previously a city would rely on a municipal utility or an investor-owned one (a co-op if you were in a more rural area) to manage the power plant, power lines, and the customer service side, deregulation meant each of these aspects were given to the lowest bidder. In theory, it's competition that brings down prices (the way I describe this to my students is, "when McDonald's introduced their Dollar Menu, you had a whole bunch of other fast food chains come up with a value menu to retain/attract customers.") In Houston, it was Houston Lighting & Power. After SB 7 it was the Wild West. Then-Texas Governor George W. Bush:

Competition in the electric industry will benefit Texans by reducing rates and offering consumers more choices.

The deregulation was phased in over a number of years, with a price floor set so the existing (former) monopolies couldn't just smash the crap out the startup retail energy providers (REPs), but the full deregulation of the energy industry in Texas was complete by 2007. As a reminder, Texas has had three governors since Ann Richards: George W. Bush, Rick Perry, and Greg Abbott. 

In early February 2011 a similar weather event to 1989 occurred, sending snow and ice all the way to Brownsville, crippling traffic in the Metroplex right before the Super Bowl was scheduled to be played there. The meteorologist in charge of the National Weather Service office in Fort Worth said, "Our temperatures rarely stay this cold for this long." Pat McDonald, an NWS forecaster for Austin and San Antonio, said, "We get about one event like this every ten years."

Six months after that February 2011 storm, FERC and the North American Electric Reliability Corporation issued a report that noted many of the power stations that failed in 1989 failed in 2011 and said, in part:

Generators and natural gas producers suffered severe losses of capacity despite having received accurate forecasts of the storm. Entities in both categories report having winterization procedures in place. However, the poor performance of many of these generating units and wells suggests that these procedures were either inadequate or were not adequately followed. 

Following the 2011 storm, the Public Utility Commission of Texas was given the power by the state legislature to require energy generators to report on their "abnormal weather preparations" and to file an emergency plan. Was that actually done? No one seems to know. 

Despite proudly announcing back in November that they were in control of enough energy to last the winter, ERCOT noted of the Valentine's Day Electricity Massacre: "This event was well beyond the design-parameters for a typical or even extreme Texas winter that you would plan for. They began as rotating outages but they're (now) controlled outages and they are lasting longer than what would normally happen because of the magnitude." 

This is despite ERCOT saying on Thursday, February 11 that their system was ready for what the storm was about to bring. On Tuesday, February 16 ERCOT said that the problem (inconveniently, if you're a Texas politician appearing on a popular cable news network) stemmed mostly from the freezing of natural gas pipelines, not from freezing wind turbines. ERCOT estimated that 80% of their grid capacity could be generated by natural gas, coal, and some nuclear sources. Only 7% of what ERCOT forecasted for winter was to come from wind or another renewable energy source. 

Ed Hirs, an energy fellow in the University of Houston's Economics Department, said: "The ERCOT grid has collapsed in exactly the same manner as the old Soviet Union. It limped along on underinvestment and neglect until it finally broke under predictable circumstances."