What Is the Goal of the Algebra 1 EOC?

It’s that time of the year: all across Florida, the children are crying, stressed and overwhelmed. Why? It is Algebra 1 EOC time.

Since 2010, Florida’s middle and high school students have been administered Algebra 1 End of Course (EOC) exams. We have argued that the Algebra 1 EOC has the highest stakes of all: not only does it count as 30% of a student’s course grade, it is a high school graduation requirement, a requirement for admission to many magnet schools, calculated into school and district grades, etc, but, this year, schools will be fined 1/6 of a student’s per pupil funding (about $1,000) if the child does not pass their EOC. Those are high stakes!

How do students react to the stress? This is what parents have to say:

• “A 7th grader (earning high school credit) took the Algebra 1 EOC/FSA and didn’t finish the test. She is stressed and terrified of receiving a failing grade (she has always been an outstanding student). She said there was material on the test her teacher never covered and she is concerned because it makes up 30% of her grade and could lower her future high school GPA.”
• “My 8th grader is set to take her honors Algebra I EOCs tomorrow and Wednesday. She’s so stressed and scared she was bursting into tears all weekend.”
• “My 8th grader is taking Alg 1 tomorrow and Wednesday. The kids are all afraid to fail the test — Crying, stressed and overwhelmed. The prep work prior to the test was brutal. Sent home with a 40 pg packet, then a 12 page packet trying to help prepare them. This on top of other FL standards tests and EOC. The teacher told them they were all going to fail.” (emphasis mine)

After students take the test, they tell us this:

• “The questions on the EOC were confusing. I left the test feeling tired, stressed out, and not confident that I did well on it at all.”
• “This is a test of “gotcha”-type questions. There are a mishmash of questions and they are often more of a function of how to use a computer than how to do math.”
• “My son took his Algebra 1 EOC yesterday and is feeling the same way. He too has had an A all year and said that he knew nothing on the test. This is his first year in public school and he was so upset. He said that he felt like he was set up for failure and by his teacher. How sad that an 8th grader who works so hard at school now feels this way.”

So, we have a ridiculously high stakes test, students are stressed and overwhelmed, and they describe the questions as confusing and a test of computer skills more than math knowledge. Some children are blaming their teachers, feeling they have been set up for failure. Interestingly, the Florida Department of Education (FLDOE) has responded similarly. Commissioner Pam Stewart said (during the Keep Florida Learning Committee) that if anything on the test doesn’t match what’s being taught, it’s the fault of the teachers who just don’t understand what they’re being asked to do. Is that true? Is the test is fine and the teachers are the problem? What if the problem is not the teachers, but, rather, the test or the standards? During the Keep Florida Learning Committee meetings, Alachua County Superintendent, Dr. Owen Roberts, repeatedly questioned the validity of the instructional standards. While they are busy placing blame on the teachers, what is the FLDOE doing to assure the validity of the EOC, its associated standards and its many high stakes uses? Is the Algebra 1 EOC designed to confirm student proficiency or teacher failure?

2015 was the inaugural administration of the new Florida Standard’s based Algebra 1 EOC, created by American Institutes of Research (AIR).  As we have described previously (read here), the AIR tests were never field tested and they lack completed validity studies. During the summer of 2015, Alpine Testing Solutions’ “Independent” Validity Study did little more than confirm Algebra 1 test questions were aligned to the standards, with no mention of fairness, reliability or validity for at risk sub populations of students; hardly (in our opinion) an appropriate evaluation for a test with such high stakes.

The 2015 FSA Technical Report reveals some interesting finding regarding the Algebra 1 EOC.

On page 4 of Volume 2 of the 2015 FSA Tech report, the number of field tested items on each assessment. Students should know that 10 of the 68 are field test items and will not be counted for or against them.

 

This is also from the 2015 FSA technical report and shows the number of technology enhanced items that had to be replaced when writing the paper/pencil version of the test.

Nearly a third of the Algebra 1 test questions are technology enhanced, requiring specific computer skills to answer them correctly. This is important because my district (Monroe) informed the school board that students performed worse on the technology enhanced items than the standard items. This brings up the question “is this test assessing math skills or computer skills?” Are students who do not have computers at home at a disadvantage on this graduation requirement? Neither the technical report nor the Validity Study addressed this. We believe this should be addressed.

There is no report comparing student performance on the FSA Algebra 1 EOC to any established, normed Algebra 1 assessment. This would be vital to assuring the FSA EOC tests what it purports to (an essential ingredient of validity).

This is the most alarming thing we discovered:  The Technical Study confirmed the EOC is ridiculously hard, yet it continues to be administered with the hope that students will, someday, acquire the skill needed to pass it. In the meantime, student self-confidence, emotional well being and GPAs (not to mention school finances) will just have to suffer.  Here is what we found:

The Algebra 1 (and 2) EOC was shown to be more difficult for the test population, with an atypical standard error curve.

From Volume 4 FSA Technical Report:

 

 

Figure 3 show, for grades 3-7, the SEM curves show the expected “U” shape.

 

 

 

 

Figure 4 shows the right side of the “U” has flattened out for Algebra 1 and Algebra 2 (and grade 8 above), indicating these exams are more “challenging” for the tested population.

 

From Page 18, Volume 4:

“For most tests, the standard error curves follow the typical expected trends with more test information regarding scores observed near the middle of the score scale. However, there are two general exceptions. In grade 8 Mathematics and for both Algebra EOC tests, the test is maximized at a higher point along the ability scale. This suggests the items comprising these tests are somewhat challenging relative to the tested population for this initial test administration. Because the testing of Algebra 1 is relatively new and the statewide testing of Algebra 2 is entirely new to these populations, this atypical curve is not unexpected. As students continue to learn these required skills, it is probable that this SEM curve will shift to reflect the expected, normally distributed SEM curve over time.”

What does this mean in English? We are not statisticians but we think this tells us that the 2015 FSA 8th grade Math and BOTH Algebra EOCs DID NOT fit the expected SEM curves, demonstrating the items on these tests are more challenging for the tested population. Is this the result of students not knowing the required skills or is this a problem with test design? No one knows because there was no comparison to an established, valid assessment. No one knows because there was no evaluation as to whether the test assesses math skills or computer skills.

The SEM results should surprise no one. The “curve” on the algebra test has always been huge, with students passing while answering less than 50% of the questions correctly. If you add technology enhanced “gotcha” questions to the previous EOC, you should expect even worse performance. The question is whether it is appropriate, or necessary, to test Algebra 1 students this way. If the DOE believes students will adjust to the new exams in time, should the high stakes be waived while allowing students to catch up on the required skills? Is there a deficit in the earlier (Common Core) math standards that contributes to the poor performance on these exams? Has anyone even looked? We hope these questions are being asked.

We believe, in a standards based system, tests could be written to assess essential Algebra knowledge without making students cry.

What is the goal here? Students can be challenged without being crushed. Exams can be rigorous without being punitive. A high school graduation requirement should not be full of “gotcha” questions. Accelerated middle school math students should feel proud of their accomplishments, not demoralized. No test scores are that important.

Is the goal of this EOC to assure student proficiency or to blame teachers for their inadequacies? Is this exam an educational tool or a political weapon?

Destroying children’s confidence should concern the Department of Education.  It is not enough to wait and see if students acquired new skills in the future. There are children taking these tests now. They are stressed and crying now. Waiting a year or two will only assure next year’s students are equally tortured.

We urge you to contact your Florida legislators and demand that these exams, and the high stakes attached to them, undergo review. Creating a test that demoralizes even the most academically talented students and then blaming their teachers for the outcome is accountabaloney.

Just prior to publishing, this was reported in the Tampa Bay Times: “Florida Department of Education leaders will be setting up an online forum this summer to collect ideas about how to revise the state’s educational accountability system.” They will have their work cut out for them because there is A LOT in need of revision. The problems with the Algebra 1 EOC are just the tip of a very large iceberg.