Active Learning Leads to Literacy Program

Executive Summary

Learning through an Expanded Arts Program, Inc. (Leap) was founded in 1977 to enhance public education in the New York City school system by integrating arts throughout the curriculum. Beginning with four schools in Manhattan, Leap today works with 8,500 teachers and 200,000 students in 300 schools across the New York metropolitan area.

In 2002-2003 Leap began a one-year pilot program intended to test the effectiveness of its hands-on, arts-based strategies to promote early childhood literacy. Anecdotal evidence supported our belief that strategies to teach literacy to young children would be enhanced through our "Active Learning Leads to Literacy" (ALLL) program. But anecdotal evidence is not enough, and so Leap and NYU applied for and received a federal grant (later supplemented with private foundation funding) to compare and evaluate the improvements in literacy for randomly selected students in classes employing ALLL strategies to randomly selected students in classes using the traditional literacy strategies. Both pilot and control classes used the same literacy curriculum.

Our study included a total of 5,337 children from 25 Title I randomly selected schools over a three-year period. The instructors included both the students' regular classroom teacher and Leap teaching consultants, both of whom received professional development training in Leap's ALLL program. The arts-based instructional approach included music, visual arts, drama, creative movement, cooking, games, and other "active" techniques to enhance literacy education.

The research design was straightforward. Classes were randomly divided into those who were to be instructed using ALLL's arts-based strategies and those who were to be instructed using traditional literacy instruction. That is, both the control classes and the ALLL classes used the same early childhood literacy curriculum, but with different teaching strategies. Students were tested via the Early Childhood Language Assessment System (ECLAS) at the beginning and end of their school year. Data were collected in a manner that would allow the researchers to employ statistical techniques that would highlight the importance of variables such as race and gender for each grade level, and that would measure 23 different elements of literacy skills (phonics, vocabulary, reading, writing, etc.).

The positive results of the study were quite startling. They are illustrated by several charts in the appendix of this report. For all three grade levels, students whose literacy classes used the ALLL strategies achieved greater gains than those in the control group in at least 16 (70 percent) of the 23 literacy skills noted above. Overall, students using the ALLL strategies scored some 18 percent higher on their ECLAS tests than students using the traditional curriculum. Additional findings suggest that ALLL is most effective in promoting literacy for limited-English-proficient kindergartners, for non-foreign students, and for English-proficient first and second graders. As the program participants were overwhelmingly Hispanic and African American and in Title I schools, the results suggest that it would be advantageous to use ALLL in urban, ethnic, low-income kindergartens and first and second grades.

Leap's Intervention into the Learning Process

The core idea of the ALLL program is that children can gain literacy competence more quickly through a process of "active learning," that is, a multi-modal course of instruction that enhances the traditional curriculum with activities based in the arts. To this end, Leap developed a great number and variety of imaginative tools and techniques to make the teaching/learning process more effective. The strategies we used include music, the visual arts, creative movement, drama, cooking, and games. Secondly, Leap provided 57 hours of professional development training for our artist-educators, who then adapted this curriculum to their particular artistic skills: painting, theater, dance, and much more. Finally, Leap provided each participating classroom teacher with 102 hours of professional development training.

Once in the classroom, the Leap artist-educator provided one hour of instruction three days a week for 20 weeks, and the classroom teacher used ALLL strategies to instruct students two hours per week for 20 weeks, for a total of 100 hours of ALLL. In addition to the actual classroom instruction, the artist-educator's work provided the regular classroom teacher with instruction models that they then adapted for their own teaching of the ALLL strategies. Team-teaching exercises were also part of the ALLL program.

Finally, upon the completion of the 20-week artist-educator's residency, Leap supplied the schools with a package of 300 Early Childhood Lesson Plans/Strategies linked to the 23 literacy skills that make up childhood literacy, and a four-hour training DVD on the use of ALLL.

The Methodology Used to Evaluate the Effectiveness of ALLL

During each of the three years of the ALLL Pilot Program we collected extensive data on student performance. Using a control group of students that was demographically almost identical to the ALLL program participants, we attempted to measure the impact of ALLL. Both the students in the control group and those using the ALLL strategies took the ECLAS test in the fall and again in the spring. We did this for each cohort of students, for each of the three years of the program. By comparing improvements in the ECLAS test scores of ALLL students with the improvements in ECLAS test scores in the control group, we could then measure the impact of the ALLL program on literacy gain.

Because we also collected extensive demographic data on the students, we were able to isolate different variables that might point to more tightly defined effects of the ALLL program. For example, did it have a significantly different effect on students whose main language was other than English? Or, was it more effective for boys than it was for girls? Or, did it have a greater impact at the kindergarten level than it did in the second grade? Or, was there any particular variance of the impact of ALLL for students who were repeating a grade, who were holdovers?

We also wanted to find out how exposure to the ALLL strategies affected students' achievement in different components of literacy and learning. The ECLAS test itself isolates 23 different components of literacy and measures student performance in each of them. These components include very basic skills such as initial and final consonants, up through more sophisticated skills such as reading accuracy and reading comprehension. Therefore, we were in a position to investigate more closely which components of literacy were clearly impacted by ALLL, and which were not. And again, because we collected extensive demographic information about the students, we were able, for example, to make distinctions within each of the literacy component gains to see if greater gains were made by students with one or another demographic characteristic, such as grade level, gender, or language background.

Finally, because the ALLL Pilot Program was a three-year study, we could measure gains in student achievement over a period of more than one year. Thus we could find out whether the gains made by ALLL students in kindergarten carried over into their first and second grade achievements, even if they no longer received ALLL instruction.

The Results of the Evaluation

Data from the ALLL program were collected and analyzed by Professor Robert G. Malgady of the Center for Research in Teaching and Learning at the Steinhardt School of Education at New York University. Professor Malgady and his assistants designed the demographic surveys and analyzed student performance data from each of the three years of the Pilot Program, as well as assessed the impact of the ALLL program over its entire three years. Professor Malgady's reports are of particular interest to those with a sophisticated understanding of statistical methods, and are available upon request from Leap. Here we will only summarize Professor Malgady's findings.

Program Year I (2003-2004) — During the first year of the Pilot Program we included in our study 1,951 students in 22 schools, ranging from kindergarten to second grade. Testing found significant differences in outcomes between the pilot students in the ALLL program and those in the control group. Looking more closely at the results, we found that the ALLL program had its greatest impact on lower-order literacy skills at the kindergarten level, and on higher-order literacy skills at the first and second grade levels.

Program Year II (2004-2005) — The outcomes for the second year were very interesting. There appeared to be no significant difference in literacy gains for pilot students in the ALLL program at the kindergarten level, and first-grade students in the control group outperformed those in the ALLL program in a majority of the 23 ECLAS skills. This was reversed for students in the second grade, where students using the ALLL strategies did better than those using the traditional literacy instruction. Further analysis of the data showed that the effectiveness of the ALLL program was highly specific, depending on grade level, particular skill, and particular types of students.

Program Year III (2005-2006) — The test results for the 1,625 students in the third year of the program showed that pilot students in the ALLL program improved their performance more than the students in the control group in 15 of 23 skills. Once again looking at the data more closely, we find that the gains of pilot kindergarten students in the ALLL program were greater than those in the control group in 9 of 15 age-appropriate ECLAS skills; pilot students in the first grade surpassed students in the control group in 9 of 12 grade-appropriate skills; and pilot students in the second grade did better on the ECLAS test than students in the control group in 11 of 14 skills. Further investigation showed that, while ALLL students made greater improvements in all three grades, kindergarten students made the greatest relative gain.

Though ALLL's impact was positive each year, the results for each year varied. Therefore, Leap has applied for a new federal grant in order to determine which strategies are most effective. Our goal will be to create a set of strategies that will significantly improve literacy skills.

Longitudinal Study (2003-2006) — The Pilot Program allowed us to compare the ECLAS achievement of students who remained in the control group for two years with students who had either one or two years of exposure to ALLL. [see endnote 1]

Looking at the first "wave" of the Pilot Program, comparing the school years 2003-2004 and 2004-2005, we found that students in the ALLL classes scored 7 to 14 percent of a grade level higher on the most elementary skills and from 20 to 75 percent of a grade level higher on the more complex literacy skills, as compared to their counterparts in the control groups. In terms of the 23 literacy components, kindergarten students in ALLL classes made greater improvement than the students in the classes using the traditional curriculum in 17 of 23 skills; among first graders the ALLL students made greater gains in 21 of 23 skills.

The evaluation of this first wave of the Pilot Program was affected by a significant change in the early childhood literacy curriculum used in the New York City public schools. The public schools used a wide variety of literacy curricula during 2003-2004, but between 2004 and 2006 they used only three significantly improved early childhood literacy curricula. Also, the literacy block was increased from 60 minutes to 90 minutes. Nevertheless, the ALLL strategies, when used with these new curricula, continued to yield better results overall, though the margin of difference was smaller.

Looking at the second "wave" of the Pilot Program, comparing the school years 2004-2005 to 2005-2006, we found significant differences in the impact of the ALLL curriculum based on grade level. Here we found that of the kindergarten students who were in the control groups and used the traditional curriculum made greater gains in their ECLAS scores than students who were in the ALLL classes. The outcome was reversed, however, when we looked at students in the first and second grades. Here we found that students who were in ALLL classes in either the first grade or second grade or both outperformed students who were in the control groups for either one or both years.

Summary and Conclusions — The three-year ALLL Pilot Program showed that the ALLL curriculum was more effective in developing literacy skills than was the traditional curriculum. Looking at the data more closely, it appears that ALLL is most effective in promoting literacy for limited English proficient kindergartners; for non-foreign students and non-limited English proficient students in grades one and two; "remedially" at the first grade level; and for second graders who have been held over from the previous year. We believe the results described above recommend ALLL strategies to school systems throughout the nation.

ALLL Pilot Program, 2003-2006

Demographic Characteristics of Student Participants

ECLAS Reading Comprehension Scores for Second Grade Students in the ALLL Program Compared with the Students in the Control Group - 2005-2006

The greatest concern of the NYC Department of Education and the focus of No Child Left Behind is that all children learn to read. We feel that the ALLL strategies strongly support these goals. You can see from this graph that the ALLL students' mean test score is 18% points higher than that of the control-group students. This means that only 32% of the control students performed above the ALLL mean in reading comprehension in second grade and 68% of the students in the ALLL program performed above the control mean. Thus over two-thirds of the students in the ALLL program out performed the students in the control group in reading comprehension.

Summary of ALLL vs. Control Pupil Gains In Literacy Skills By Grade Level and By Literacy Skill, 2003-2006