The financial investment starts well before the first formal day of kindergarten. The Harlem Children's Zone spends almost as much per child in its Harlem Gems preschool, $13,500, as the city spends on a typical older student. Gems tykes are carefully cultivated and groomed for school; they're in the Promise Academy pipeline already, because Harlem Children's Zone planners hold kindergarten lotteries when a cohort of students is 2 or 3 years old—effectively holding seats until they are old enough to attend kindergarten. In addition, HCZ spends $5,000 per child each year for after-school and extracurricular programs for students who don't attend the Promise Academies but live within the Harlem Children's Zone. Some of the money goes to direct payment of middle school children, for good grades and participation in HCZ programs.
The school day begins at Promise Academy I and II at 8 a.m., even for the youngest students. At Harlem Gems, the lottery admission pre-K program that feeds into the Promise Academies, the day stretches from 8 a.m. to 4 p.m. After-school programs, which include 4- and 5-yearolds, run until 6 or 7 p.m. There's Saturday school every weekend, and some teachers and students meet as early as 7 a.m. for intensive test preparation.
"Every single child has to make it," says Shana Brodnax, senior manager of early-childhood programs at the HCZ. "It's an entirely no-excuses-accepted policy that takes an almost incomprehensible amount of resources and support." "Failure is not permitted," vowed Canada, speaking to a public gathering in Springfield, Mass., in November. "No excuses. Failure is not permitted, because funding is tied to success, not failure."
In the world of education, success has many definitions. But the HCZ schools are simply too new to be able to measure success in the vocabulary of graduation or college enrollment—no students have yet graduated from the Promise Academy's high school, so there's no graduation rate to discuss. Regents scores from 2009 are encouraging but preliminary, as only one cohort of students has taken the exams. Nearly 500 young adults who participated in nonschool HCZ programs are now in college, but not much is known about that group.
Instead, at the Promise Academies, success has an explicit benchmark: "We are judged by the New York State tests," says HCZ spokesperson Marty Lipp. "We literally live or die by that test."
Like all other public school students, those at the Promise Academies take statewide assessments every year. The Promise Academy schools have recently posted strong results in math: In 2009, 87 percent of Promise Academy eighth-graders scored at or above grade level, compared with 61 percent overall in District 5. On the state math test, 91 percent of Asian students and 86 percent of white students citywide scored at or above grade level, as did a mere 62 percent of black students in the city's schools. Since the Promise Academy is 91 percent black, its high scores suggest a far narrower racial achievement gap than might otherwise be expected.
On the 2009 English-language arts (ELA) test, 57 percent of Promise Academy eighth-graders met or exceeded grade-level standards, compared with 46 percent in District 5 at large and 50 percent of black students in New York City. While HCZ students' scores exceed city averages for black students, a substantial and significant race gap persists: Citywide, 76 percent of both white and Asian eighth-graders scored at or above grade level. (Promise Academy eighth-graders bested their District 5 counterparts in 2007 and 2008 on math and English, as well.)
In April 2009, Harvard economists Roland Fryer and Will Dobbie released a study asserting that "the Harlem Children's Zone is enormously effective at increasing the achievement of the poorest minority children," based on their analysis of 2007 state test score data. In middle school, they documented gains that "reverse the black-white achievement gap in mathematics." Grade school results are even stronger, Fryer and Dobbie say, and "close the racial achievement gap in both subjects [math and English-language arts]."
Test scores are the single most powerful measure in the city's annual progress reports about each school. Yet both the city's Department of Education and New York State Regents chancellor Merryl Tisch recognize that the Level 3 score— widely translated as "at grade level" or "proficient," which is where most HCZ students scored—does not actually predict academic success. In fact, students who score Level 3 in eighth grade have only a 52 percent chance of graduating from high school in four years, according to Tisch and analysts at the city Department of Education.
Fryer and Dobbie based their conclusions on gains made by a single class on a single test in a single year. In other years, and for other grades, state-exam scores at the Promise Academy have not always been impressive. The fifth-graders scored lower than the district average on the 2009 math test. Only a third of the school's eighth-graders were at grade level on the 2008 English test.
On non-state exams, the results are even more mixed. On the Iowa Test of Basic Skills (ITBS), the eighth-graders' average score was 41, well below the HCZ-set target of 50 and a score that correlates to an achievement ranking on the 33rd percentile nationally. (ITBS scores since 2007 have risen but still do not meet HCZ-set goals.) On the Terra-Nova English assessment, HCZ's goal was for 65 percent—the tipping point—of students to score 80 percent or above, a goal that the school has not yet been able to achieve. A similar target was set for math; again, the organization's testing goals were unmet, despite three-month delays in testing that should have translated into extra gains. The fact is, any test one looks at, whatever result is shown, is of limited use in judging whether the Promise Academy model works or not.