FEE Panel Agrees: Raising Proficiency Levels Demands Accountabaloney Pause

On January 4, 2016, Florida’s State Board of Education will be setting the cut-scores/determining proficiency levels for the Florida Standards Assessment (FSA) that was administered in early spring 2015. The commissioner of Education has reviewed the cut-score levels determined by a panel of education experts, reviewed by a community “reactor panel” and presented to the public. At the last Board of Education meeting, she presented her recommendations, which raised the proficiency bar higher than the educators recommended but apparently not high enough to please several members of the board. On 12/4/2015, the BOE will meet for the last time before the cut score process is complete. You can find information regarding that meeting here.  One would expect the conversation regarding proficiency levels will come up. Will the Board be convinced to agree with the commissioner’s recommendations or will they push for higher achievement levels, resulting in lower passing rates for Florida’s students?

The question “Why Proficiency Matters” is currently a hot topic in reform circles. In this blog, we will attempt to get into the mind of a reformer and see the issue from their perspective. The best place to start is the Foundation for Excellence in Education’s (FEE’s) annual meeting.

On October 22, 2015, education reformers from around the nation converged on Denver, Colorado, for the Foundation for Excellence in Education’s 8th annual Education Summit. Strategy Session VII was focused on “The Proficiency Gap: Why It Matters for Your State.” This session was particularly timely for Florida because, less than a week later, its Board of Education members would be meeting to discuss determining the proficiency levels (or “cut scores”) for the new, 2015 Florida Standards Assessment (FSA). (You can watch Session VII here)

Members of the Florida Board of Education were present during this strategy session so I thought it might be interesting to try to view the session from their perspective, to try to understand what lessons they might have learned and how those lessons might influence the upcoming FSA cut score process. To do that, I am going to ask everyone to bear with me and, for the time being, put on a reformer hat. In doing so, I need you to accept a few basic assumptions. These are a few of the things that essentially all reformers, and most of the audience for this session, believe:

1. The current standards (Common Core State Standards, Florida Standards, College and Career Ready standards, whatever you choose to call them) are developmentally appropriate, well constructed, internationally benchmarked (not just informed by international benchmarks), rigorous standards that will lead to better academic achievement by all students.
2. The current state assessments are a high quality, effective and efficient way of determining whether students are on the path toward Career and College Readiness.
3. The current assessments have been psychometrically evaluated and are fair, valid and reliable for all types of students, including Special Education and English Language Learners.
4. Raising proficiency cut scores on a state assessment will lead to higher levels of academic performance.
5. Reformers are on the right path and should not be distracted by naysayers. What reformers are doing is working and we can’t “go backward.”

The debate regarding all of the above will be left alone for the time being (I know it will be difficult; try your best).

Here is a recap of the highlights of “The Proficiency Gap: Why It Matters for Your State” from the 2015 FEE Education Summit:

Introductions were made by the moderator Dr. Christy Hovanetz, Senior Policy Fellow of Accountability, Foundation for Excellence in Education. She explained why she felt now was the right time to have the conversation about proficiency levels (because, having spent so much money on new standards, new tests, new curriculum and professional development, “it would all be for naught” if we didn’t raise the standards). She claimed the current “proficiency gap” (the difference between proficiency levels on state tests and proficiency level on “the Nation’s Report Card” or NAEP assessment) was a result of states not being honest with their students. The proficiency gap, she claimed, was really an “honesty gap”, a result of states, essentially, cheating the system, and it needed to be corrected.

The first panelist was Karen Nussle, Executive Director of the Collaborative for Student Success.  She explained what the proficiency gap was and blamed it on politicians who wanted their state’s scores to “look good.” She explained “there were politicians in the mix, it was hard to be honest.” Ms. Nussle explained “The Collaborative” is a PR firm that could help share information about the honesty gap and she shared a very clever commercial they had produced to influence the FSA cut-score process in Florida, showing darling children saying things like “When I grow up I want a dead end, low wage job.”  Really, the kids are cute and the commercial is funny (I’m pretty sure I heard the distinct chuckle of one of Florida’s BOE members in the audience) and “The Collaborative” can help your state get similar commercials to lobby your Governor to influence the setting of cut scores beyond educators’ recommendations, because “parents deserve the truth.”

The second panelist was Alabama State Superintendent of Education, Dr. Tommy Bice.  Under Dr. Bice’s leadership, Alabama adopted standards 4 years ago, chose to assess students with the ACT Aspire and have raised their proficiency bar to eliminate the honesty gap.  He was present as an example of how the process could be done successfully.  He had lots of interesting things to say, so pay close attention.  During this transition  process he had spent a good deal of time developing relationships with Alabama’s 2 and 4 year colleges and with its business leaders.  He frequently asked them “What skill set is missing in Alabama high school graduates when you get them.”  The overwhelming response was kids graduating from Alabama high schools “lack intellectual curiosity.”  He was not surprised by this because “We have been preparing kids to take a meaningless test rather than to think, solve problems, think analytically and collaborate with peers.”

Dr Bice spent a good deal of time explaining why they chose the ACT Aspire, how they communicated the new expectations to parents and what preliminary growth data looked like.  He, also, shared what he felt was the key to Alabama’s success :

“The biggest challenge… is providing this policy safe space in these first few years to not misuse assessment results.  We misused them for decades under No Child Left Behind, to rank and divide and conquer the haves and have nots, we need to use them and we need to use them instructionally to make sure we are improving the practice for our teachers, and improving the learning for our children but we want to be very careful and very protective that we don’t use them for policy agendas that don’t lend itselves (sic) to those things, at least in the first few years and then when we see what this really means to us, we can begin to guide some of the policy decisions that need to be made.”

He repeated that it was important, in Alabama, that they not “misuse these scores in the first few years until we can really see what that growth outcome needs to look like.” During a brief question period, Ms. Nussle pointed out that not everyone will be happy with the new, “more honest scores” but “we are really changing the system here and there is a transition period to that here and there needs to be public understanding of what that involves.” Dr. Bice agreed:

“That’s the most important message… it would be real easy to fall back into how we did it under No Child Left Behind, to put people on lists and rank and divide. We need to use this to really look at what we can do instructionally to move  children in a more accelerated rate to be college or career ready and you don’t do that in a policy environment that uses assessment scores in other ways.”

Next up was Scott Sargrad, Director for Standards and Accountability, Center for American Progress. He spoke about content standards and proficiency standards and explained how, back in 2009, “there was widespread agreement that proficiency standards, and content standards, were too low.” Since then, there has been a lot of work done to increase standards and proficiency expectations.  He explained how raising the proficiency bar can expose previously hidden achievement gaps, allowing more resources and supports to go to the schools that need them. He explained how teachers’ expectations have a significant impact on students. He, also, recognized “serious buy in” has to happen to prevent pushback when raising the proficiency bar. Then (at 31:40) he said:

“One of the things that’s really important is… being patient with the scores and not trying to use them too soon.  So, when I was at the DOE, we offered states this flexibility to take a pause on using some of these new results for accountability purposes for schools, for teachers, while they are getting their feet under them. Now not every state took us up on that… but there is also value in taking a little break from the rating and the ranking of the schools and saying we’re going to keep making during the kids are going to get the supports and interventions they need but we’re going to see how these tests play out, we’re going to see how the growth plays out over 2 years, we’re going to see what that means for teachers, see what that means for school.”

Mr. Sargard concluded by noting that we will know whether this works if we see smaller gaps with NAEP (he was hoping to see this in the 2015 NAEP scores), lower remediation rates in college, higher college graduation rates and students getting good jobs after college… then we will know if we have been successful.

Cornelia Orr, former Executive Director of the National Assessment (NAEP) Governing Board, was the final speaker. She shared NAEP research, explained NAEP mapping to state assessments (which is not the same thing as matching percentages) and gave this advice:

“I think you should set your proficiency cut score at a level that will be challenging for students to reach…  My advice is that you use the proficiency level to communicate about student learning and that you use your accountability system to monitor your system and, if changes need to be made there, you do that in the accountability system, or the way that you use those scores, but you set your proficiency mark as high as you can because that is what communicates to students.”

In response to an audience question regarding fairness of these proficiency levels for older students who have not been exposed to Common Core during their entire school careers (from my own Monroe County School Board member, Captain Ed Davidson), Dr Bice reiterated:

What is important “in these first few years of implementation, rather than adjusting scores, is be careful on how we use them, is not use them especially at the upper grade levels as a high stakes, get out of high school sort of assessment, which I would adamantly oppose at this point and we don’t use those in that way in Alabama… we strictly use those to make instructional decisions, to make sure that we’re providing a curriculum and instruction that get kids more college and career ready. We don’t use them in any sort of punitive or any other way for students.”

The final audience question was from Florida BOE Vice Chair, John Padget. He felt this was a really great panel and he reiterated the need for honesty (possibly his question is a plant from the Foundation?). He claimed he has repeatedly seen students who have finished high school and are told when they show up at college “you are not college ready.” He wanted to know what can we do to be honest at an earlier point so students don’t face disappointment at that time. (Going to take off my reformer hat here for a moment to say: personally, I would suggest attending parent-teacher conferences and meeting with the high school counsellor, to get a good idea as to whether your child is on track for success in college… putting the hat back on now). Dr. Bice replied with  some comments about how community colleges actually make a lot of money providing remedial classes and there might be a financial incentive for them to recommend remediation.

Karen Nussle wanted to end on a positive note. According to her, the testing products are better than ever.  Millions of kids have taken the PARCC tests and we can compare data across states.  The tests are great and can only get better  Parents are going to get more information about their kids than ever before. The amount of data collected is fantastic. This is working and we have to keep at it. We need to stop allowing others to distract us from where we are going because this is working.

Lots of applause. Dinner will be served at 6.

Remember, you are still wearing your reformer hat.  As Mr. Padget said there was lots of great info here.  As Ms. Nussle said, we should focus on it and not allow ourselves to be distracted. In the end, there was little debate. All 5 experts seemed to agree on the issues and the main points of the session can be summed up:

1. Raising the achievement bar to NAEP levels is something we all want.
2. The key to Alabama’s success  was providing this policy safe space in these first few years to not misuse assessment results.
3. Test scores should be used to make instructional decisions, ensuring students are provided a curriculum and instruction that will get them more college and career ready.
4. Guard against using test scores for policy agendas that don’t lend themselves instructional improvements.
5.  One of the things that’s really important is being patient with the scores and do not try to use them too soon.
6. Use the proficiency level to communicate about student learning and use your accountability system to monitor your system and, if changes need to be made there, you do that in the accountability system, or the way that you use those scores.
7. Do not use them, especially at the upper grade levels, as a high stakes, get out of high school sort of assessment.

So that summarizes the Foundation for Excellence’s panel discussion on “Why Proficiency Matters”.  Everyone was pretty much in agreement and Alabama was held out as the shining example of “how to do it right.”

You can take off your reformer hat now.

Fast forward a few days to the October 28, 2015, Florida Board of Education meeting in Orlando (you can watch public comments here). During public comments, Patricia Levesque, the extremely influential C.E.O. of the Foundation for Excellence in Education, advised the board to do two things:

1. Proceed with the issuing of school grades based on the new FSA. Do NOT listen to the School Superintendents who are asking for a pause.
2. Do “the right thing” and set “an honest cut score for children.”

So, Ms. Levesque suggested that Florida set a cut score aligned with NAEP scores (an “honest” one) AND issue school grades, in direct conflict with what essentially every Session VII panelist recommended. Why didn’t she listen to her own experts and recommend a pause in the use of test scores for anything except improving instruction during the transition period? Did she miss the strategy session just a few days earlier or does she disagree with its conclusions?

We recommend that Florida use the initial few years of the FSA administration as recommended by the FEE panelists and institute a policy free zone.  This may take the doing of the legislature and/or the Governor but, as Dr. Bice strongly recommended, it is necessary part of the process. A complete re-evaluation of the accountability system should be done.  Listen to the experts and use the FSA scores only to improve instruction. Eliminate the high stakes. With raised proficiency levels the cut scores will no longer reflect expected grade level performance. Do legislators intend to retain and remediate third graders who are reading at or above grade level? Should these tests be used as a high stakes, get out of high school sort of assessment? Does our Board and Department of Education really want our schools and teachers to continue to focus almost entirely on test scores for fear of punitive repercussions or should they use those test scores, as Dr. Bice strongly suggested, “to make instructional decisions, to make sure that we’re providing a curriculum and instruction that get kids more college and career ready”?

One final question: Why would Ms. Levesque advise the Florida Board of Education to do something that her entire panel of experts would disagree with?