Algebra 2 EOC Results: Has Florida Jumped The Shark?

In 1977, in an episode of the sitcom “Happy Days”, Fonzie displayed his bravery by jumping over a confined shark while on waterskis. Ever since then, the phrase “jumping the shark” has come to mean the moment in a TV series that defines the beginning of the show’s decline; a desperate attempt to retain viewership where it becomes obvious to the audience that the show has strayed irretrievable from its original formula. We believe the administration of the 2015 Algebra 2 FSA EOC (Florida State Assessment End of Course exam) will be remembered as the moment when Florida’s Education Accountability System “jumped the shark.”

Over the last week or so, the preliminary FSA EOC student results have been released to parents. Because the cut score process is still in progress, the reports show a percentile ranking for the child and the score breakdown of questions answered correctly in each “reporting category”. The full “horror” of the Algebra 2 EOC is being revealed.

Red flags were raised regarding the Algebra 2 EOC even before it was administered. Prior to the Algebra 2 EOC, every previous state-created EOC had been treated as a baseline administration during its inaugural year. Students would take the new EOCs in their inaugural year but performance on those test would not affect their course grade. The Algebra 2 EOC, however, from day one was planned to be worth 30% of the student’s course grade. Multiple emailed questions regarding this break from protocol went unanswered by the Florida Department of Education last spring.

Students recognized significant problems with the Algebra 2 EOC while it was being administered last spring. Here is a first person account from a 15 year old, gifted, straight A, honor student from Monroe County.

“I took the Algebra 2 EOC last year and it was absolutely ridiculous. I got my scores back and it said that I did better than 83% of the students that took it, but I’m not sure how they could even score such a horrible test. I walked out of the testing room and knew for sure that I failed it, and that they were not going to count it because of how flawed it was. The format that the test was on had many errors, and was filled with information that the teachers were not asked to teach. For example, the test would say “graph this piece wise function” and it would give you the wrong tools to graph it with, so even if you were totally capable of completing the question successfully, you couldn’t.”

We are certain that almost every Algebra 2 student will tell a similar story. The test was a disaster, bringing usually confident high school students to tears.

Before we discuss the results, understand this: Florida (along with the rest of “Common Core” America) has a “standards based education”. Teachers are asked to teach the standards and if students can demonstrate, on their state assessment, that they have learned enough of those standards, they will be deemed “proficient” and “pass” the exam. Florida State Assessments (like the previous versions, the FCAT) are “Criterion Referenced Tests” or CRTs. A CRT should compare a student’s performance to established expectations; if every student meets those expectations, all students would be deemed proficient. Likewise, if no students met the established criterion, all would fail. On CRTs, it is not only possible, but desirable, for every student to pass the test or earn a perfect score. (Remember that, it is important.)

A common example of a CRT is the written test for your driver’s license: if you answer 8 out of 10 questions correctly, you pass the test… even if thousands before you answered 9 or more correctly. (Learn more about Criterion Referenced Tests and their usage here)

The first problem parents should notice when reading their child’s Algebra 2 EOC report is that the score is reported as a percentile ranking, which is an inappropriate way to present results from a criterion based exam. Remember, if all students achieved the established expectations, ALL students would pass. In a criterion based exam, percentile ranking should be irrelevant.

Why would the Department of Education release the percentile ranking of students for a Criterion Referenced Test? Probably, in part, because they don’t expect parents to understand the difference and they are under time pressure to release some information regarding student performance, given that the exam was taken more than 6 months ago. The main reason, though, is that the established expectations for the new FSA have not yet been established and won’t be established until the January 4^th State Board of Education meeting (more on that later).

So, despite having NO established performance standards, the Algebra 2 results were released to parents, highlighting a student’s percentile ranking in the state and their performance in individual “Reporting Categories.” Here is what we have learned from reports parents have shared with us:

ALGEBRA 2 2015 FSA EOC results

#correct (out of 56)	Total %correct	Student percentile ranking
16	25%	63 %tile
18	27%	67%tile
20	36%	71%tile
29	52%	88%tile
43	77%	99%tile

 

Keep in mind that those who took Algebra 2 are essentially all college bound, often honors and/or gifted math students, and this exam is, by statute, worth 30% of their course grade (the 30% was waived after test administration last spring but, by report, will be back in effect next spring). The State’s most gifted math students (99th percentile) still missed one out of 4 questions and students will have passed (if the Commissioner of Education’s current cut score recommendation is approved) missing ¾ of the questions. Something definitely seems amiss.

Remember: “On criterion-referenced tests, it is not only possible, but desirable, for every student to pass the test or earn a perfect score”. So why has Florida created a CRT where its most gifted math student are unable to answer 25% of the test questions correctly?

Of course, the cut scores/expected performance levels have not yet been set for the Algebra 2 exam, so these student reports, actually, say nothing as to whether a student passed or not.

On September 15-17, 2015, at a Rule Development Workshop, the general public got a first look at the impact of potential cut scores. It wasn’t pretty. You can see the complete presentation here.

To summarize, the Educator Panel recommended cut scores than would fail 69% of students, including 85% of African American students. Since then, the Commissioner of Education has made her recommendations which would (only) fail 64% of students. The ultimate cut scores will be determined by the Florida Board of Education (BOE) on January 4, 2016. Several members of the board have indicated they wanted cut scores that were even higher than the Commissioner’s, failing even more students. Sadly, the process of establishing cut scores seems to reflect political ideology more than sound education judgement. (Learn about the cut score process here)

Prior to the release of the Algebra 2 test scores, on October 28^th, I spoke in front of the BOE, and described why I felt “the most egregious example of Accountabaloney is the state’s unwavering assertion that the Algebra 2 EOC is valid.” You can watch my speech here. In summary:

• In 2015, Algebra 2 had a new set of standards, which now included Pre-calculus and Statistics (learn about the standards here).
• The commissioner has admitted that no one knows whether it is even possible to teach these standards in a single school year (details here).
• The Algebra 2 EOC could not have been field tested because Utah does not test Algebra 2 or Trigonometry standards.
• The Alpine Validity Study completely ignored Algebra 2 in the interest of time (Report here).
• Honor students reported being unable to answer any questions on day two of this EOC, so it should have been no surprise that the exam’s results were total outliers with almost half of students failing including 85% of African American students. (Report of results here)
• At the Keep Florida Learning Committee, the department suggested the problem was “the teachers didn’t teach the standards”. (more info here)
• Crushing the confidence and destroying the GPA of our best and brightest students does NOT make them college and career ready. Blaming it on their teachers does NOT help recruit and retain high quality STEM educators.

I concluded with “the Algebra 2 exam and its standards deserve a complete review.” Seriously, parents shouldn’t have to ask. In the face of falling national math scores (read here), the entire math sequence deserves a review.

Currently, parents and teachers report concern regarding both the pace and scope of the new Algebra 1 and Algebra 2 standards. When Florida switched to the Common Core-like Florida Standards in 2014, much of what used to be taught in Algebra 1 was moved to Pre-Algebra. Algebra 1 now includes statistics and at least a third of the previous Algebra 2 content. Algebra 2 now includes pre-calculus and statistics. The Florida Standards are now out of alignment with the new CCSS-based textbooks. If a teacher teaches from the new, state approved Algebra 1 or 2 textbooks, they will not cover the required standards. This is a problem. Why would the State approve textbooks that do not cover the required course content?

In addition, the amount of content seems to be too much to complete in a single year, especially since the EOCs are given in late April/early May. One Algebra 2 teacher said:

“The problem is that there are TOO MANY standards in the test item specs. It’s simply TOO MUCH to teach, prior to the Algebra 2 EOC which starts in early April to the first week in May. We have to teach at the speed of light to get everything in, and it’s nearly impossible to do so.”

Parents are concerned that the pace does not allow their kids to fully understand concepts and many kids are dropping out of subsequent advanced math courses. Another math teacher wrote:

“Algebra 1 now looks like Algebra 2 and Algebra 2 now looks like Precalculus/Statistics. Regular kids who are not strong in math are drowning. It’s just so developmentally inappropriate. These new standards and new EOC’s are geared towards honors students who excel in math.”

Clearly, when Florida created its new “rigorous” Math standards, moving a significant amount of Algebra 2 content into the Algebra1 course, it created problems beyond the EOCs. By statute, passing the Algebra 1 EOC is a graduation requirement so Florida now requires a mastery of the higher level Algebra 2 content to graduate high school. Was this the legislature’s intention? Also, in the current A-F grading system, middle schools are being graded based on their number of advanced math students (Algebra 1 and Geometry) encouraging increasing student placement into Algebra 1 (Learn about how School Grades are calculated here). Now these very young math students are being asked to master Algebra 2 standards; is this developmentally appropriate? Also, the middle school math teaching credential is for grades 5-9 and DOES NOT include Algebra 2 content. Now that Algebra 2 standards are placed in Algebra 1 courses, most middle school math teachers are not credentialed to teach the content. This is a serious problem.

The current pace, scope and assessment of Math standards has been significantly disrupted by #accountabaloney. We believe re-evaluation of the entire advanced math sequence and math FSA EOCs is warranted.

Here are the concerns that must be addressed:

1. The overwhelming evidence suggests there is CLEARLY something wrong with this Algebra 2 exam, yet the DOE is releasing test scores, cut score determinations are underway and the test remains on the Spring 2016 schedule. This defies reason. Is the DOE addressing this? Why would they release the scores from such a disastrous test administration? Why even require a state mandated EOC for a non-required class?

2. Has either the Algebra 1 or 2 EOC been definitively shown to be fair, reliable and valid for special populations including special education (ESE), English Language Learners (ELL), low income or racial subgroups? Where are those reports? 85% of African Americans failed the Algebra 2 EOC… 85% of the best and brightest African American students failed this test… Something is wrong. Also, since passing the Algebra 1 EOC is a graduation requirement, the Algebra 1 EOC definitely needs to be evaluated for fairness for ESE, ELL, etc, students. Where is the evidence this has been done on these new EOCs? (Please don’t accept the answer “the Alpine Validity Study”; it did NOT address these issues.)

3. Is there any evidence that an accelerated pace through advanced math concepts is a good idea? Why have Algebra 2 standards in an Algebra class or pre-calculus standards on Algebra 2?  Should Algebra 2 mastery be required for high school graduation. Are middle school teachers properly credentialed to teach the current Florida Algebra 1 standards?

4. We have significant concerns regarding the entire accountability system, extending beyond the math EOCs, at this point. Clearly the “establishing cut score” process has as much to do with political ideology as it does educationally sound practices. We worry that, with the new tests and the new more “rigorous” cut scores, the initial intent of many of the State’s legislated mandates are no longer aligned to the current assessments. For example, mandatory third grade retention is meant to identify and provide interventions for students reading below grade level; is there any evidence that the current cut scores assess grade level proficiency? Will 3^rd grade students reading at or above grade level be marked for retention? Likewise, passing Algebra 1 was a challenging requirement for high school graduation but now it appears the EOC is testing a much higher, more challenging Algebra 2 curriculum. Is that what the original legislation intended? We suspect not.

CALL TO ACTION:

We encourage parents and teachers to share this blog with Governor Rick Scott, Commissioner Pam Stewart, the Florida Board of Education and your legislators. Tell them the Algebra 2 EOC is the most egregious example of the problems with the current accountability system. Ask them how parents are expected to have confidence in a system that creates final exams that are so abusive in nature and then seemingly ignores the obvious issues. If the DOE is not evaluating the math standards and assessments, demand they do so. If they are investigating these issues, remind them it would be wise to inform teachers and parents before we lose all confidence and respect.  We need a DOE that can recognize flaws in the current system and work to fix them. Florida needs to re-evaluate both the advanced math sequence and math FSA EOCs.

With the creation and administration of the Algebra 2 EOC, we believe Florida’s Education Accountability has “jumped the shark.” Without dramatic policy reversals, we fear parent confidence in the system will be irretrievable.