Connect with us

What are you looking for?

Education

One Day, One Test: The promise and perils of Tennessee’s new third-grade reading gate

A student at Cherokee Elementary School volunteers to read. Third graders in public schools who “flunk” next spring’s TNReady reading test — and generally two-thirds of them in Tennessee do — are eligible to be retained in third grade next year. (Jim Weber/The Daily Memphian file)
A student at Cherokee Elementary School volunteers to read. Third graders in public schools who “flunk” next spring’s TNReady reading test — and generally two-thirds of them in Tennessee do — are eligible to be retained in third grade next year. (Jim Weber/The Daily Memphian file)

Tennessee’s  opens in a new windowtoughened third-grade retention law, passed in early 2021, goes into effect this month.

Third graders in public schools who “flunk” next spring’s TNReady reading test — and generally two-thirds of them in Tennessee do — are eligible to be retained in third grade next year.

What does it mean to be “retained?” It’s what most people call being held back a grade.

While much focus is put on the test results of the Memphis-Shelby County Schools, the problem is not unique to Memphis. Fully two-thirds of Tennessee’s third graders — about 50,000 children — flunk the TNReady reading test.

Third grade will be more complicated this year.

The new law is raising the pressure on students, teachers, schools and districts to improve Tennessee’s low reading scores on the National Assessment of Educational Progress (NAEP).

The  opens in a new windowNAEP, given every other year to fourth- and eighth-grade students, is the standard by which all states are compared on the Nation’s Report Card. About 65% of Tennessee’s fourth graders are reading below “proficient” levels on the NAEP. Tennessee’s score hasn’t improved since 2013.

“We are proposing a third-grade reading gate, which means that we make sure students are prepared before we pass them through to the fourth grade,” Gov. Bill Lee told legislators when he introduced the tougher retention proposal in January 2021.

But Tennessee’s new retention law also raises questions and concerns about its purpose and efficacy.

The Institute for Public Service Reporting at the University of Memphis examined the state’s new third-grade retention policy, as well as similar policies across the country, a range of research on their effectiveness, and concerns raised by educators.

Among the concerns:

  • The unfairness of retaining an 8-year-old child based on one test score from an annual standardized achievement test that is subjectively scored, inconsistently administered and widely questioned.
  • Misperceptions that TNReady and other standardized achievement tests measure “grade level” performance, and misunderstandings about what reading at “grade level” and “proficient” reading really mean.
  • Studies that show elementary school retentions are counterproductive and even harmful, can widen achievement gaps, and disproportionately impact students of color.
  • The failure of third-grade retention policies to address students with learning disabilities, which is the cause of many if not most reading deficiencies.
  • The sheer number of third graders who could be retained because they “flunk” the annual TNReady reading test — 64% statewide in 2021-2022 and fully 77% of third graders in Memphis-Shelby County Schools (MSCS).

Why third grade?

Third grade is hard.

In its  opens in a new windowGuide to Third Grade, the education publisher Scholastic explains, “In third grade, students progress from practicing basic skills to mastering them, and move on to develop more complex skills.”

What are these complex skills?

Skills such as learning how to multiply, divide and estimate.

Topics such as basic molecular science and U.S. history.

Tasks such as reading and analyzing longer nonfiction texts and fictional chapter books.

“After mastering basic literacy skills in earlier grades, third graders become better and more independent readers,” Scholastic explains. “Third-grade reading focuses on teaching kids how to think and talk about what they read in deeper and more detailed ways.”

Children who struggle to read in third grade are more likely to struggle academically — and otherwise — as they grow older.

The consequences are profound. These struggling 8-year-olds are four times more likely to drop out of high school. And, studies show, they are much more likely to experience delinquency, unemployment, violence and incarceration.

“The ability to read by third grade is critical to a child’s success in school, life-long earning potential and their ability to contribute to the nation’s economy and its security,” the Annie E. Casey Foundation concluded in “ opens in a new windowEarly Warning,” a 2010 special report on third-grade reading.

Because early-childhood education is so critical to the academic and personal success of children, the federal government began pushing states to adopt more rigorous learning standards for all grades beginning in the 1990s.

But there was a special focus on third grade.

Guthrie Elementary School students participate in a reading circle. (Lance Murphey/The Daily Memphian file)
Guthrie Elementary School students participate in a reading circle. (Lance Murphey/The Daily Memphian file)

The No Child Left Behind Act, signed by President George W. Bush in 2002, required states to test the reading and math skills of every student every year — starting in third grade.

That same year, Florida Gov. Jeb Bush, the president’s brother, signed a law requiring third graders to be retained if they flunked the state’s reading test.

Since then, 37 states — including Tennessee — have joined the third-grade retention brigade, passing laws aimed at making sure students are reading “proficiently” by the end of third grade.

Twenty of those states, including Tennessee, Mississippi and Arkansas, require third graders to be retained if they flunk their state’s reading test.

Tennessee’s original third-grade retention law was passed in 2011, but it was vaguely worded and rarely enforced.

But in 2021, Gov. Lee and Republican legislators revised the law. They linked retention directly to a third grader’s reading score on the annual TNReady achievement tests.

Which is why so many third graders in the Memphis area are at risk of being held back at the end of this year.

Retention policies highlight disparities — and carry risk

Tennessee’s new retention law does offer alternatives to being held back for third-graders who fail the test. It establishes “a full-time tutoring corps, after-school camps, learning loss bridge camps and summer learning camps” for struggling third-grade readers, and promises “reading interventions and supports for students who are identified as ‘at risk’ for a significant reading deficiency.”

A student who “demonstrates adequate growth” in the summer camps, or who accepts and completes approved after-school tutoring in fourth grade can avoid retention.

Those alternatives are critically important. Studies show that holding back a student carries incredible risks to the child. And studies show that there are serious disparities when it comes to the backgrounds of children being retained.

Roughly 10% of U.S. public school students are retained at least once between kindergarten and eighth grade.

But the American Academy of Pediatrics (AAP)  opens in a new windowreports that children are most successful when they are supported to advance grade levels with their peers while the reasons behind their lack of progress are addressed.

Repeating a grade can be considered an adverse childhood experience (ACE). In  opens in a new windowone study, sixth-grade students rated having to repeat a grade as “the most stressful life event.”

Elementary students who are held back a year rarely show long-term academic progress, according to a number of studies over the years.

Several studies showed that achievement gaps between retained and non-retained students only widen.

Retained students are five times more likely to drop out of school, and more likely to have problems with aggressive behavior and to get into trouble.

Students of color and poverty, and especially boys, are  opens in a new windowdisproportionately more likely to be held back.

In 2016, 8% of children from households with incomes at or below the federal poverty level repeated a grade compared to 3% of children from higher-income households.

Why some children struggle to read

Despite two decades of expensive, expansive education reform — much of it focused on high-poverty neighborhoods and schools — the needle has barely moved on national and state reading scores.

“The socioeconomic achievement gap remains essentially as large as in the mid-1960s,” researchers from Harvard and Stanford reported in 2020. “In terms of learning, students in the top quarter of the socioeconomic distribution continue to be around three years ahead of those in the bottom quarter by eighth grade.”

Twenty years after No Child Left Behind, the federal law designed to close the socioeconomic achievement gap, the gap persists.

That has held true despite billions of dollars spent expanding charter schools, raising learning standards, toughening accountability laws, supporting new teacher training programs and developing a new, nonprofit educational support system.

One of every five children in the U.S. has dyslexia, ADHD, or some other learning, thinking or attention difference that impedes their ability to learn to read.

For children of poverty or trauma, the number is one in three.

“The pandemic has exacerbated the limitations of standardized tests, which reward a narrow set of skills and more affluent students who have access to specialized instruction,” the  opens in a new windowEconomic Policy Institute reported in 2020.

Standardized achievement tests can show that third graders aren’t reading as well as they should at age 8. The scores don’t show why.

The American Academy of Pediatricians lists a number of factors that can explain a child’s “reading deficiency.

Children with learning disabilities may chronically underperform academically because of their neurologically based learning differences.

On the other hand, acute academic decline in a child who previously was performing well may indicate onset of a physical condition or an acute stressor (bullying, ostracization, change in teachers or schools, family concerns, death of a close friend or relative, substance use, etc.).

Alternatively, behavioral or emotional challenges may present concerns before the neurodevelopmental disability that is causing the academic problem is discovered.

Challenging behaviors may include hyperactivity, inattention, anxiety, irritability, sadness, aggression, oppositionality and/or social isolation.

The childrens’ book “Shark School” was part of summer reading kits offered to 1,300 area children in 2020. (Mark Weber/The Daily Memphian file)
The childrens’ book “Shark School” was part of summer reading kits offered to 1,300 area children in 2020. (Mark Weber/The Daily Memphian file)

A child who displays oppositional behavior only at school but is compliant at home, where parents are not placing academic demands, may have an undiagnosed learning disability. Another child might behave well in the classroom but decompensate emotionally or behaviorally at home while completing homework.

Other factors that explain reading difficulties:

  • The parental level of education below a bachelor’s degree, little or no shared reading at home, food insecurity, family history, medical risk factors and fair or poor parental health.
  • Inadequate sleep duration and quality. Growing evidence reveals that average sleep time per night for children has gradually been decreasing over the recent decades.
  • Social history, including household composition and adequacy of housing, family income sources and adequacy, transportation, food security, social network and/or degree of social support, and interpersonal violence or personal trauma history, can help to identify social determinants of health and well-being that may contribute to academic problems.

Children with a history of adversity and/or trauma (poverty, homelessness, abuse, neglect, parental mental health issues, etc.) deserve special consideration, educators say.

Sometimes their historical or ongoing emotional trauma and anxiety may be interfering with learning. In other cases, children with these histories may also have specific learning issues related to the impact of trauma on the developing brain.

Children in foster care or kinship care or children who have had involvement with child welfare not only have experienced trauma but also have possibly had multiple school changes. Each school change can essentially result in a loss of four months of academic skills.

AAP says children experiencing one or more of these factors may need the help of child psychologists or psychiatrists, neurologists or neuropsychologists, speech or language pathologists, occupational therapists, or physical therapists, or developmental-behavioral pediatricians.

Educators say third graders in Tennessee who flunk the annual TNReady reading test may need more help than a summer reading camp, an after-school tutor, or another year of third grade.

“It’s a misguided approach, but we’re going to throw millions and millions of dollars at that,” Mike Winstead, director of schools in Maryville., said when the state toughened its third-grade retention law in 2021. “We just need to scale it down, laser it to the kids who need the help, back here at the 20th and 30th percentile, who are not on track to be successful in life. Put the money there, and we’ll see some growth there.”

Mississippi miracle or mirage?

The third-grade retention law that goes into effect in Tennessee beginning with this school is part of a trend.

In some ways, that trend started 20 years ago with the passage of The No Child Left Behind Act, signed by President George W. Bush in 2002. It required states to test the reading and math skills of every student every year — starting in third grade.

Since then, 37 states — including Tennessee — have joined the third-grade retention brigade, passing laws aimed at making sure students are reading “proficiently” by the end of third grade.

Twenty of those states, including Tennessee, Mississippi and Arkansas, require third-graders to be retained — or held back — if they flunk their state’s reading test.

The results have been mixed, with advocates highlighting increased reading scores among some students, and critics lamenting a system that has created many problems.

“There’s a lot of ways to raise your (NAEP) test scores, and one way is to just make sure that kids that can’t pass it don’t take it,” Maryville City Schools Director Mike Winstead said after the new laws were passed in 2021.

Winstead is one of many school leaders who see problems with the state’s new third-grade retention policy.

“It’s about keeping the kids in third grade so that they don’t take the fourth grade NAEP test so that your NAEP scores look better and you have a ‘Mississippi miracle,’ ” Winstead said in 2021.

The “Mississippi Miracle” is a reference to a dramatic increase in that state’s NAEP reading scores since 2013.

That’s the year Mississippi Gov. Phil Bryant signed four education reform bills, including mandatory third-grade retention for students who flunk the state’s third-grade reading test.

Since then, Mississippi’s fourth graders have shown dramatic gains in reading scores in the 2015, 2017 and 2019 NAEP tests. From 2017-2019, for example, the state’s average fourth-grade reading score rose four points, more than any other state.

Since 2013, when then-Mississippi Gov. Phil Bryant signed four education reform bills, the state’s fourth graders saw dramatic gains in reading scores on the 2015, 2017 and 2019 NAEP tests. (Rogelio V. Solis/AP file)
Since 2013, when then-Mississippi Gov. Phil Bryant signed four education reform bills, the state’s fourth graders saw dramatic gains in reading scores on the 2015, 2017 and 2019 NAEP tests. (Rogelio V. Solis/AP file)

But skeptics point out that Mississippi retains a higher percentage of third graders than any other state — about 10% statewide and up to 40% in some counties.

By comparison, Florida retained 14% of its third graders the first year, but retention rates have leveled off to about 8% annually since then.

(Both states suspended retention policies for the 2020-2021 school year due to COVID-19.)

Florida’s average fourth-grade reading scores have increased seven points overall to 225 since its third-grade retention policy was instituted in 2003.

Mississippi’s has increased 10 points to 219, the national average, since it began retaining third-graders in 2013.

Tennessee’s increased five points after its initial third-grade retention policy was passed in 2011, but has since fallen a point to 219.

“Holding back low-performing third graders creates the illusion of huge one-time testing gains, and implementation of the bill would take place just in time for the 2023 NAEP tests,” Amy Frogge, former Nashville school board member, texted after the law was passed in 2021. “This is not about best serving the children of Tennessee; it’s about gaming the system.”

Even advocates for the new laws say third-grade retentions only partially explain Mississippi’s rising reading scores.

“After being held back, (students) receive a variety of supports, including ‘intensive reading intervention’ and being assigned to a high-performing teacher,” the  opens in a new windowFordham Institute reported in 2019. “Assuming that those policies improve their achievement, they should certainly score better once reaching fourth grade than they otherwise would have.”

Tennessee’s new retention law establishes “a full-time tutoring corps, after school camps, learning loss bridge camps and summer learning camps” for struggling third-grade readers and promises “reading interventions and supports for students who are identified as ‘at risk’ for a significant reading deficiency.”

A student who “demonstrates adequate growth” in the summer camps, or who accepts and completes approved after-school in fourth grade can avoid retention.

The American Academy of Pediatrics (AAP)  opens in a new windowreports that children are most successful when they are supported to advance grade levels with their peers while the reasons behind their lack of progress are addressed.

The Florida experience

Florida’s third-grade retention law was signed in 2002 by then-Gov. Jeb Bush. Since that time, studies have shown the policy was applied inequitably. (Robert F. Bukaty/AP file)
Florida’s third-grade retention law was signed in 2002 by then-Gov. Jeb Bush. Since that time, studies have shown the policy was applied inequitably. (Robert F. Bukaty/AP file)

In 2002, Florida Gov. Jeb Bush signed a law requiring third graders to be retained if they flunked the state’s reading test.

Since then, studies showed that Florida’s retention policy was applied inequitably.

Students of color, English learners, students with disabilities and boys, in general, were more likely to be held back. So were students whose mothers have less than a high-school degree.

But advocates for retention policies argue that those inequities merely reflect the reality that students from homes with lower incomes and education levels struggle more in school.

They also argue that early-grade retention helps, not harms.

Repeating third grade improves that student’s preparedness for high school and performance while enrolled in school, according to a 2017  opens in a new windowstudy by the National Bureau of Economic Research.

The study followed 75,000 students who had been retained under the Florida law beginning in 2003.

The retained students experienced substantial short-term gains in both math and reading achievement.

They were less likely to be retained in a later grade and better prepared when they entered high school. They took fewer remedial courses in high school and improved their grade point averages.

Being held back delayed their graduation from high school but did not reduce their probability of graduating or enrolling in post-secondary education.

“Third-grade retention in Florida has no impact on student absences or special education classifications,”  opens in a new windowwrote Martin West, a Harvard University education professor.

Learning ‘deficiencies’ or ‘disabilities’?

There’s a gaping hole in Tennessee’s new third-grade retention law: the challenge of children with learning disabilities.

The law can result in students who fail to score high enough on the TNReady reading test being held back in third grade — that’s some 50,000 third graders across Tennessee in past years, and 77% of students in Memphis-Shelby County Schools.

The new law requires schools to identify students who are “at-risk of significant reading deficiency.”

Those are students who demonstrate “a lack of significant progress and/or skills significantly below grade-level based on universal screening data.”

But the new law doesn’t address students with learning disabilities, which is the cause of many if not most reading deficiencies.

Kids who flunk third-grade reading assessments generally are kids with learning disabilities, either diagnosed or undiagnosed, studies show.

Mandatory retention laws generally exempt special education students. But receiving special education “requires a diagnosis, and a child’s underachievement may be significant before one can be made.”

As many as one in five children in the U.S. have learning, thinking and attention differences such as dyslexia or ADHD.

These differences, often labeled learning disabilities, are caused by variations in how the brain develops and how it processes information.

Learning disabilities are unrelated to intelligence. Kids with LDs just need strategies and supports to help them thrive, educators say.

But kids who learn and think differently often don’t get early or effective interventions. Many don’t get diagnosed at all.

Only one in 16 public school children qualify for special education services. A majority of them have their basic deficits in language and reading.

As many as six in 10 adults with significant literacy deficits in the U.S. live with an undiagnosed learning disability, according to the National Center for Learning Disabilities.

A large study found that half of young adults with diagnosed learning disabilities had been involved with the justice system.

Students with disabilities are more than twice as likely to be suspended, three times more likely to drop out of high school, and half as likely to enroll in a four-year college, and twice as likely to be jobless as adults.

Learning disabilities range in severity and may interfere with:

  • oral language (listening, speaking, understanding)
  • reading (phonetic knowledge, decoding, reading fluency, word recognition, comprehension)
  • written language (spelling, writing fluency, written expression)
  • mathematics (number sense, computation, math fact fluency, problem solving)

The Learning Disability Association of America lists seven specific types of learning disabilities and four disorders that are often classified as related.

They include:

Dyslexia

Dyslexia affects areas of the brain that process language. A child with dyslexia has difficulty identifying speech sounds and learning how they relate to letters and words.

Most children with dyslexia can succeed in school with tutoring or a specialized education program. An estimated 70% to 80% of people with poor reading skills are likely to be dyslexic.

Attention Deficit Disorders

Symptoms including disorganization, impulsive behavior, poor time management and difficulty with focusing on tasks.

Inattention often leads to a diagnosis of ADHD, but it can also be a symptom of obsessive-compulsive disorder, post-traumatic stress disorder, and other learning disorders.

Dysgraphia

This affects a child’s ability to write legibly. They may have messy handwriting, hold pencils incorrectly and take much longer to write than most children.

Auditory, visual and language processing difficulties

This affects a child’s ability to pay attention, process sounds and learn. They have trouble following and remembering instructions.

Learning disabilities are neurologically based, and disproportionately impact children of poverty and trauma.

So do other factors that aren’t addressed by the new third-grade retention law.

Keeping score isn’t as easy as 1, 2, 3

Of all the concerns about Tennessee’s new third-grade retention policies — policies that could result in thousands of local students being held back in the summer of 2023 — one of the most difficult issues to understand is the arcane way that standardized test scores are calculated.

So confusing are these calculations that there is widespread misunderstanding about what it even means to read at “grade-level.”

And maybe most concerning of all, these complicated formulas can penalize children just on the cusp of meeting the standard.

“A lot of those students just miss hitting the proficient mark by one or two questions and are labeled as significantly deficient in this bill,” Maryville City Schools Director Mike Winstead said after the new laws were passed in 2021.

That’s just one of the concerns school officials have with the new third-grade retention law: Large-scale standardized achievement tests don’t measure “grade level” performance.

Defining ‘grade level’

“The legislation is attempting to address third graders who can’t read at grade level, but the TCAP test doesn’t test to see if students can read at grade level,” Lakeland Supt. Ted Horrell  opens in a new windowtold The Daily Memphian in 2021.

TNReady tests measure “performance levels” based on educated but very subjective judgments. They don’t determine whether students are reading at grade level.

The same is true of the National Assessment of Educational Progress (NAEP), which is divided into three performance levels — Basic, Proficient and Advanced.

NAEP results are reported as a percentage of students who read “at or above proficient.” Two-thirds of fourth graders nationally and in Tennessee fall below that standard.

“ ‘Proficient’ on NAEP does not mean grade level performance. It’s significantly above that,” wrote Tom Loveless, director of the Brown Center on Education Policy at the Brookings Institution. “Using NAEP’s proficient level as a basis for education policy is a bad idea.”

Lakeland Schools Superintendent Ted Horrell (at the grand opening of Lakeland Preparatory High School, Saturday, July 30, 2022) said in 2021, “The legislation is attempting to address third graders who can’t read at grade level, but the TCAP test doesn’t test to see if students can read at grade level.” (Lucy Garrett/Special to The Daily Memphian)
Lakeland Schools Superintendent Ted Horrell (at the grand opening of Lakeland Preparatory High School, Saturday, July 30, 2022) said in 2021, “The legislation is attempting to address third graders who can’t read at grade level, but the TCAP test doesn’t test to see if students can read at grade level.” (Lucy Garrett/Special to The Daily Memphian)

The National Academy of Sciences agrees.

“NAEP’s current achievement-level-setting procedures remain fundamentally flawed,”  opens in a new windowthe academy concluded in 2005. “The judgment tasks are difficult and confusing; raters’ judgments of different item types are internally inconsistent; appropriate validity evidence for the cut scores is lacking; and the process has produced unreasonable results.”

The confusion is compounded by the way test scores are reported, which can lead to misunderstandings about what reading at “grade level” really means.

“Grade level has been defined as the average reading achievement at any particular grade,” wrote Richard Allington, a nationally known reading researcher at the University of Tennessee, and former president of the International Literacy Association. “As with any average, half the population is, by definition, above average and half is below average.”

But a “below-average” reader might be just slightly below or way below.

The same holds true for the state’s TNReady tests.

Scores may vary

Every spring, students in grades 3-8 take TNReady achievement tests in reading, math, social studies and science.

The tests are timed. Third graders get 195 minutes for the reading exam (known formally as the English language arts test), 115 minutes for math, 104 minutes for science and 50 minutes for social studies.

That adds up to 464 minutes of testing, or about 7½ hours over the course of several school days. Each test is broken into parts that last no longer than 50 minutes.

On the reading test, students are asked to read brief passages of text, then answer multiple-choice questions about the text.

Here’s an example:

Select the two sentences from the passage that best represents the idea of the New Year as “a time of fresh beginnings.”

  1. “This image … was important to cross through the ‘right way’ to produce favorable outcomes.”
  2. “A resolution is simply an agreement with yourself to change something about your life for the better.”
  3. “A new year is like a clean sheet of paper to draw to draw blueprints for a fresh vision.”
  4. “However, you should not wear black, as that will bring you sorrow in the New Year.”
  5. “There is an old story that exactly at midnight animals are able to speak for one minute…”

The correct answer is 2 and 3.

‘Grade-level’ and ‘proficient’

TNReady scores are cut into four levels that measure a student’s understanding and ability “to apply the grade/course level knowledge and skills defined by the Tennessee academic standards.”

Level 4 (Mastered) demonstrates “an extensive understanding and expert ability.”

Level 3 (On-Track) demonstrates a “comprehensive understanding and thorough ability.”

Level 2 (Approaching) demonstrates “approaching understanding and partial ability.”

Level 1 (Below) demonstrates “minimal understanding and ability.”

According to the new law, third-grade students “achieving a performance level rating of ‘Approaching’ or ‘Below’ on the reading portion of the student’s most recent TCAP test” will be held back.

Students achieving “On-Track” or “Mastered” levels are promoted to fourth grade.

Based on 2022 TCAP results, about 23% of Tennessee’s third graders scored Below” grade-level expectations in reading, and about 41% scored “Approaching.”

Students who score Level 3 “On Track” are generally working above grade level, school officials say.

“Indeed, a high level of performance is required to achieve the ‘On Track’ rating,”  opens in a new windowDale Lynch, executive director of the Tennessee superintendents group, wrote in 2021. “Even third graders who are performing at grade level could be subject to retention. A lot of our teachers believe you can be proficient but still be categorized as ‘Approaching’ grade level.”

It’s also notable that standardized achievement tests like TNReady aren’t graded like regular classroom tests.

On a regular classroom test versus a standardized test, if a third grader correctly answers 85 out of 100 questions on a classroom test, the student’s “raw score” of 85% will be converted to a grade. That generally means that a score between 80-90% will convert to a B.

However, on standardized tests, “a raw score by itself has no meaning … because tests may differ in content and difficulty (year to year),” the  opens in a new windowMississippi Department of Education explains on its website.

Changing the questions and cutting the scores

Yet another challenge for standardized tests is that test makers change up to a third of the questions every year. Otherwise, the answers to last year’s questions might find their way into this year’s classrooms.

That’s why, each year, test makers try to write new questions that are equally challenging. Yet that also means that this year’s questions might randomly be a tad easier, or a tad more difficult.

“On some tests, the student is lucky (knows more answers) and gets higher scores. On other tests, the student is unlucky (knows fewer answers) and gets lower scores,” Mississippi explains.

In other words, a third grader who scores a 69 on this year’s reading questions might have scored a 70 on last year’s questions. What does that mean? If test results aren’t “equalized” from year to year, that one point could be the difference between being promoted to fourth grade or not.

And there are even more elements of subjectivity built into the testing process.

For instance, to ensure that each year’s questions are equally valid, testing officials “cut” raw scores into scale scores. This means that if a third-grader correctly answers 85% of the questions on the TNReady test, his raw score of 85 will be converted or “cut” into “performance levels.”

The “cut score” separates one performance level from another.

The old TCAP test cut scores into four “performance” categories: below, basic, proficient and advanced.

The current TNReady cuts scores into below, approaching, on-track and mastered categories.

“There is no ‘right’ way to set cut scores, and different methods have various strengths and weaknesses,”  opens in a new windowwrote Andrew J. Rotherham, co-founder and co-director of Education Sector.

Or, as  opens in a new windowBoston College explained in its “Scores May Vary” study of cut scores: “Regardless of the method, the cut-score setting process is subjective.”

Tennessee’s ‘bookmarked’ scores

There are other ways that experts question Tennessee’s TNReady approach to standardized tests. That includes an arcane statistical technique called item response theory (more  opens in a new windowcommonly known as the Bookmark method, developed by CTB/McGrawHill in 1996.)

Under the Bookmark method, test questions are ordered along a scale of difficulty, from easiest to hardest, based on that year’s actual scores.

A question that 99% of students answered correctly would be considered least difficult.

A question that 1% of students answered correctly would be considered most difficult.

A committee of 16 or so judges (all certified grade-level teachers) evaluates each question along the scale of difficulty.

Then the judges “bookmark” the question they believe separates one performance level from another — based on the definitions of each level.

This process continues for three rounds. And after this process, the cutting of scores comes back into play. The final cut score is determined by the median value of each judge’s bookmarks.

This story first appeared at dailymemphian.com under an exclusive use agreement with The Institute.

Written By

David Waters is Distinguished Journalist in Residence and assistant director of the Institute for Public Service Reporting at the University of Memphis.

Support The Institute

Please help keep public service journalism alive in Memphis. Your donation funds costly original reporting on subjects important to Memphis and is greatly appreciated.

You May Also Like

Education

HOPE IN MEMPHIS Thousands of people got up before dawn Friday morning and used candles and cell phones and their collective spirits to light...

Watchdog

If MPD had arrested Cleotha Abston on rape charges last year, would he have been on the streets this month when Eliza Fletcher disappeared?

Watchdog

Police say the same man attacked two women. One white. One Black. But if they'd taken Alicia Franklin's case more seriously, would Eliza Fletcher...

Politics

As the state comptroller endorses a possible ouster of Wanda Halbert, a new dispute erupts between her office and the Shelby County Assessor's Office.