Report tallies gains from United Way’s Collective Impact initiative

It’s a simple concept: Focus on fewer goals and bigger results. That’s the idea behind the Collective Impact approach United Way of Central Carolinas (UWCC) has adopted. For United Way that goal for the next 10 years is to increase the graduation rate among the more than 13,000 at-risk children who receive services from 16 United Way-funded agencies.

That idea – to set measurements to show, over time, whether goals are being met – is at the heart of the Collective Impact initiative.

The UNC Charlotte Urban Institute is supporting UWCC in its Collective Impact study, using data from the Institute for Social Capital (ISC) community database.

For its Collective Impact Baseline Report from early 2013, the institute’s researchers analyzed baseline measures, so that future years’ results could show any changes. The most recent results released show changes to students’ academic performance and absenteeism in the 2011-12 school year compared to the year before participants received agency services.

The newest report, presented to the UWCC Board of Directors and to Mecklenburg County agencies by the institute’s director of research, Amy Hawn Nelson, looked at students before and after receiving United Way agencies’ services. The results showed:

• A majority of elementary and middle school students improved or remained stable in mathematics and reading from the year before they began receiving services and 2011-2012.
• 82 percent of participants remained stable or improved on their end-of-grade reading exam and 85 percent remained stable or improved on their end-of-grade math exams since they had begun receiving agency services.
• An analysis of absenteeism found 53 percent of participants had the same number or fewer absences in 2011-2012 than in the year before they received services.
• The number of out-of-school suspensions decreased or remained stable for 80 percent of participants in 2011-2012 compared to the year before they received services.

One key component of the Collective Impact initiative has been creating a common measurement system across the 16 UW agencies.

The baseline study examined academic performance, attendance, and suspensions of participants in United Way funded agencies before they began to receive services from a UW-funded agency. The recent Collective Impact report furthers that work by examining both the year prior to the student’s enrollment in services from a UW agency, and the 2011-2012 school year.

The 16 agencies involved in the study have missions to serve at-risk youth in our community. Their services range from after-school tutoring services to character building to legal services.

The Collective Impact report found that most participants in United Way funded programs were African–American (73 percent) and in elementary school (38 percent). Seventy percent attend a Title I school (schools with at least 72 percent of the student population designated economically disadvantaged) and 20 percent attend a Project LIFT school. (Project LIFT is a public/private partnership serving West Charlotte High School and its eight feeder schools.)

More details of the findings:

Math and reading EOG scores: A majority of elementary and middle school students improved or remained stable in mathematics and reading between the year they began receiving services and 2011-2012. Students in grades three through eight take end-of-grade exams (EOG) in reading and math and are deemed proficient if they score a Level III or IV. The study found that 52 percent of participants were proficient on the reading exam and 69 percent were proficient on the math exam in 2011-12.

The study also looked at the change in EOG exam scores from the year before participants received agency services, compared to 2011-2012. The study found 82 percent of participants remained stable or improved on their EOG reading exam, and 85 percent remained stable or improved on EOG math exams since they had begun receiving agency services.

Some high achieving eighth graders (in math) and all high school students (grades 9-12) take end-of-course (EOC) exams. Like EOG results, students are identified as proficient if they score a Level III or IV on the EOC exams. Students only take EOC exams once unless they are required to repeat a course; therefore, the change in EOC scores is not calculated. The study found that 67 percent of participants were proficient on EOC English exams and 62 percent of participants were proficient on EOC math exams in 2011-2012.

Indicators of future success: Student attendance and out-of-school suspensions are important indicators of future success as they can be most directly affected by enrichment and character-building programs. During the middle grades, rates of chronic absenteeism increase and continue to climb during high school. Chronic absenteeism is most pronounced in high-poverty schools. Absenteeism, along with out-of-school suspensions and failing courses, is considered a warning sign of disengagement in school and lowers the likelihood of graduating high school.

Attendance: In the 2011-12 school year, 36 percent of participants were chronically absent, meaning 10 or more absences from school. On average, participants were absent 10 school days—though this varied from no absences to a maximum of 150 days absent. The analysis found that 53 percent of participants had the same number or fewer absences in 2011-2012 than in the year before they received services.

Out-of-school suspensions: In the 2011-2012 school year, 24 percent of participants received an out-of-school suspension. Overall, participants averaged two out-of-school suspensions. While many experienced no suspensions, one participant received 86 during the 2011-2012 school year.

One caution in interpreting the results is that the study is descriptive and not designed to show causality.

In addition to the Collective Report, each agency received a report with the data for its participants. The study is now moving into its third year, and is expanding to include the school systems in Cabarrus County and Kannapolis.