Prior-Prior Learning Gains and Other Post-Covid Accountabaloney.

CORRECTION: The original version of this post incorrectly stated that students would be tested on standards they were not taught in the 2021-22 school year making learning gains calculations nonsensical in the 2022 School Grade calculation. This was INCORRECT and the post has been corrected. In 2020-21, the FLDOE will proceed with the adoption of new ELA Instructional Materials but the FLDOE recommends not implementing the use of that curriculum for grades 3-12 until the 2022-23 when new assessments are available. The calculation of annual learning gains will be nonsensical for only two of the next three years.

---

On 3/17/2020, in the midst of the pandemic induced transition to distance learning, Governor DeSantis announced the cancellation of state assessments and A-F School Grade calculations for the 2019-2020 school year. The announcement allowed educators, in the midst of the greatest transformation in Florida education history, a brief sigh of relief while they focused their efforts on the herculean task of continuing to provide learning experiences for Florida’s nearly 3 million public school students. Teachers across Florida rose to the challenge. Even so, the impact of the pandemic has hit some students more than others and, as clearly explained by Education Historian Jack Schneider, the pandemic “laid bare the vast inequities in U.S. public education,” demonstrating that students who struggle tend to come from families that struggle. Now, districts are focused on the process of re-opening their campuses while identifying and providing additional services to those students whose learning was most disrupted by Covid.

How will the absence of 2020 test scores impact next year’s school grade calculations? Last week, the FLDOE gave a glimpse of the future…

Without 2020 test scores, the calculation of annual learning gains in 2021 will be impossible. Learning gains make up a significant (40 -57%) portion of the school grade calculation (up to 40% for high schools, 44% for middle schools and 57% for elementary schools). Learning gains may, also, more accurately identify academic success in struggling students. Assuming that the 2020-2021 school year will be event-free enough to even allow spring assessments, how can 2021 School Grades be calculated in the absence of annual learning gains?

On May 20, 2020, the FLDOE released a K-12 Accountability update, which outlined anticipated “adjustments” to the 2020-2021 Accountability calculations, most notably recommending that next year’s learning gains be calculated by comparing 2021 test results with pre-Covid 2019 test scores (the so-called “prior-prior” year). You can watch the webinar here.

Will comparing pre-Covid 2019 test results to 2021 test results result in a meaningful school grade calculation, or will schools and students with the greatest needs during Covid be judged particularly harshly? For example, will districts see their community’s lack of Broadband access reflected in their school grades? Will schools serving the families of essential workers, whose parents were less able to be involved in their child’s distance learning, be judged fairly? Let’s look at a few of the possible impacts:

• In elementary schools, 57% of the school grade is dependent on calculated learning gains. In 2021, that calculation will only reflect the 2 year learning gains of the 5th grade class, since neither 3rd or 4th graders will have a score from 2019.
• Students in the lowest quartile, the most struggling students, count twice in the calculation. In 2021, the performance of the lowest performing 5th graders in an elementary school will impact more than 35% of an elementary school’s entire school grade calculation. In the smaller schools in my district, the scores of fewer than a dozen 5th graders will be responsible for more than 35% of their school’s grade. Is this an adequate measure of school quality?
• Middle and High Schools will see their incoming student’s learning gains calculated based on prior-prior scores from their feeder schools. A high school’s calculated learning gains will reflect the quality and effectiveness of the Covid induced distance learning provided by the previous middle school. A middle school’s calculated learning gains will reflect the quality and effectiveness of the Covid induced distance learning provided by the previous elementary school.
• The calculation of Math learning gains in high school have always been nonsensical but, for many in 2021, Math learning gains will be calculated by comparing Geometry EOC scores with 8th grade FSA scores or Algebra 1 EOC scores with 7th grade FSA scores.
• The middle school acceleration points calculation will also be adjusted to incentivize the placement of next year’s 8th grade students into Algebra 1 based on their 6th (rather than 7th) grade FSA Math scores.
• Legislation will be needed to proceed with this “prior-prior” year to current year learning gains calculation because state law requires the school grade calculation to include annual learning gains.
• There was no discussion of compassion or grace when it comes to adjustments to the high stakes attached to these nonsensical school grade calculations and what that might mean for communities hardest hit by the pandemic.

According to the FLDOE, the decision to shift to the two year calculation of learning gains was made after they realized that if learning gains (and middle acceleration points) had been excluded from the 2019 school grade calculation “the distribution of results would vary greatly, with the change more likely being a reflection of a significant change in the calculation rather than an actual change in school performance.” The simulation did NOT include measuring the impact of learning gains over a massive and unexpected disruption to distance learning during a global health crisis and whether that would reflect an actual change in school performance. What are expected learning gains during a global pandemic and should learning gains even be the primary focus when students return? (We recommend you read this excellent piece “Maslow Before Bloom“.)

In a 5/29/20 conference call with the Florida School Boards Association, Jacob Oliva, Executive Vice Chancellor, Division of K12 Public Schools, FLDOE explained the prior-prior year calculation to school board members saying “this decision would be the least disruptive; it would be the most consistent and, by far, we’ve heard from the field, this is the recommendation that they would want us to move forward with.”

Really? Who are these people who overwhelmingly support the two-year learning gains calculation? In the absence of accurate learning gains, are School Grade calculations even useful for accountability purposes? The stated goal of the Florida School Grades system is to “provide an easily understandable way to measure the performance of a school. Parents and the general public can use the school grade and its components to understand how well each school is serving its students.” Will prior-prior year learning gain calculations help parents decide where the best school is or will they reflect, for example, the broadband access in that community? Should parents choose an elementary score based on the performance of a handful of 5th graders? Will letter grades reflect academic achievement or the impact of living in Covid hotspot? Should parents choose a school based on how well the previous school instituted distance learning?

The prior-prior year scheme may provide consistent school grades but the calculation will be nonsense (and by that we mean: even more nonsensical than usual).

There have been concerns regarding the importance of including annual learning gains in the school grade calculation in the past. In 2015, when annual learning gains were unable to be calculated due to the transition from the FCAT 2.0 to the new Florida Standards Assessments (FSA) school grades were reported on an “informational baseline” only. At that time, the FLDOE acknowledged that two consecutive years of assessment data were necessary to properly calculate learning gains. Programs dependent on assessment performance, like the “300 Lowest-Performing Elementary Schools list,” carried on using calculations from the previous school year. In the midst of a global pandemic, the FLDOE could learn from its past and, without the data to calculate annual learning gains, issue informational school grades only.

Moving forward, 2021 will not be the only year with nonsensical learning gains calculations. In the same K-12 Accountability Update, the FLDOE announced their intent to ignore the pandemic and continue (full steam ahead) along the “Most Aggressive Transition Timeline in Florida’s History,” implementing the new Benchmarks for Excellent Student Thinking (B.E.S.T.) standards.

Next year, the FLDOE will proceed with the Adoption of ELA Instructional Materials but recommend not using that curriculum for grades 3-12 until the 2022-23 when new assessments are available.

In the 2022-2023 school year: we will be right back in the situation we we’re in in 2015 – the new assessment will make learning gains impossible to calculate.

The FLDOE understands the importance of including learning gains in the school grade calculation, but for two of the next three years the calculation of authentic annual learning gains will be impossible.

In 2014, Miami-Dade Superintendent, Alberto Carvahlo, called for a pause in the calculation of school grades during the transition from the FCAT 2.0 to the FSA, saying “indefensible or baseless grades are worse than no grade at all.“

“How can results that are potentially inconclusive and not trusted to drive consequences be trusted to produce school grades with all the implications they are known to carry from something as subjective as school prestige to school recognition funding?

It should be evident that a rush to protect legacy or policy status quo could prove to be not only disingenuous but also potentially disastrous. Simply getting it done cannot be a cheap, expedited substitute for getting it done right. There is a high moral imperative at play here, which if not addressed and reasonably responded to, will define not only who we are, but also what we value, how we tackle challenges when we know better, and how we value our children and those who teach them.”

In the midst of a global pandemic, causing massive disruptions in schools, communities and economies… during the transition to new standards, curriculum and assessments… Florida could pause the test and punish accountability system. State Assessments could continue to be administered but the scores could be used to inform instruction and focus resources during this transition period. We did it in 2015 in much less difficult situation.

To all those “in the field,” who Chancellor Oliva says are recommending that the FLDOE move forward with “prior-prior” learning gains because they are the “least disruptive,” please reconsider what you are supporting. As Florida moves forward with a rapid standards rollout during a global health and economic crisis, we could choose “compassion and grace,” pause the high stakes attached to state assessments, and issue informational-only School Grades, giving educators the ability to focus on their student’s and community’s unique needs during this time of rapid transition. We have done it before, without catastrophe.

Remember:

“Indefensible or baseless grades are worse than no grade at all.”