Connections between Education in the Arts and Student Achievement

The theme of education in the arts can be found throughout GIA's programs. The role that the arts can play in education is one of four primary themes that will be explored at our 2002 conference, Creative Connections; and the “Bookmarks” column in this issue of the Reader concentrates on “Arts Education Resources on the Web.” The following two articles take a look at recent research, specifically research that explores the connections between education in the arts and student learning in other realms. The theme of education in the arts can be found throughout GIA's programs. The role that the arts can play in education is one of four primary themes that will be explored at our 2002 conference, Creative Connections; and the “Bookmarks” column in this issue of the Reader concentrates on “Arts Education Resources on the Web”. The following two articles take a look at recent research, specifically research that explores the connections between education in the arts and student learning in other realms. The writers of both articles — Nick Rabkin, who reviews Critical Links, and Dale Rose and Michaela Parks, who review a special issue of The Journal of Aesthetic Education — highlight the need for additional research.

Critical Links: A New Compendium of Research

Nick Rabkin

Educators may disagree about how to best teach children to read and write, or learn science, history, and math, but there are no serious questions about whether these subjects should be taught to all kids. They are the core of academic programs. In most schools, the other subjects — including the arts — are squeezed in, or they are not taught. These subjects are threatened when budgets are trimmed, when new demands are made on schools to improve academic performance, or when new subjects claim a share of the school day.

Advocates for arts education have long made an essentialist argument for the arts: they are such an important dimension of life that they must be included among core academic subjects. Their efforts have been rewarded by inclusion of the arts as a core subject in federal legislation, most recently in the No Child Left Behind legislation and earlier in Goals 2000 legislation.

But most people think of the arts as expressive, creative, emotive, and recreational, not as academic. They may agree that the arts are an important part of life, but this does not make them essential to the enterprise of education. Including the arts in federal education legislation is, in the end, lip service. No states have funded mandates for arts education, and there are no standardized tests in the arts. Schools teach what's tested, and the arts aren't tested. Arts education may have enjoyed some growth during the boom of the past decade, but high-stakes testing and budget tightening jeopardize it now.

If the arts are going to find a place at the education table, more persuasive arguments must be made. Arts educators have long reported that the arts are connected to a wide range of benefits to students beyond their learning in the arts — academic achievement, positive social development, habits of mind, and inclinations of thought. Some have speculated that if these connections were documented, an instrumental case could be built for the arts that might have broader appeal and the potential to affect policy.

Now the Arts Education Partnership, a broad association supported by the National Endowment for the Arts, the Department of Education, and some private funders, has published a compendium of research on these connections. Critical Links: Learning in the Arts and Student Achievement and Social Development presents summaries, findings, and commentary on sixty-two studies, the best of recent research. The studies were carefully selected from a universe of thousands by James Catterall of the University of California at Los Angeles, Lois Hetland of Project Zero at Harvard's Graduate School of Education, and Ellen Winner of Project Zero and Boston University. These three and several other distinguished researchers summarize each featured study, evaluate its methods and findings, and comment on its significance.

Critical Links addresses the general question of what the arts contribute to education and development by looking at particular contributions from each arts discipline. The book is organized into sections on dance, drama, multi-arts, music, and visual arts, each concluding with a thoughtful essay summarizing what we know from research, what we can speculate about, and what we need to learn more about.

Critical Links makes the case for a great many links between learning in the arts and student achievement. In a summary essay, Catterall catalogs them. Each discipline is connected to significant outcomes. For example, in the visual arts, there are findings about how drawing supports writing skills and how visualization training supports interpretation of text. In music, researchers found strong connections to spatial reasoning and math, and between instrument instruction and SAT scores. Dance instruction was connected to fluency in creative thinking and to reading skills. Drama in the form of dramatic enactment was connected to story comprehension, character understanding, and writing proficiency, and is shown to be a better way for students to process a story than teacher-led discussion. Multi-arts programs, as you might expect, had multiple connections: to reading, verbal, and math skills, and to creative thinking.

Similar connections are present between arts learning and social and emotional development. Dance is connected to self-confidence and persistence; music to self-efficacy and self-concept; drama to concentration, comprehension, conflict resolution, and self-concept; multi-arts to achievement motivation, cognitive engagement, self-confidence, risk-taking, perseverance, and leadership. Several studies show that children become more engaged in their studies when the arts are integrated into their lessons. Others show that at-risk students often find pathways through the arts to broader academic successes.

There is not likely to be much controversy about the desirability of the educational and developmental outcomes found in Critical Links. But are these connections merely a matter of correlation or are they causal? In educational terms, can we legitimately say that learning in the arts “transfers” to other contexts of learning? Can we credit the arts as the cause of all this good stuff for kids? Proving transfer in education is a pretty tough assignment. Schools are complex settings, and learning is an enormously complex enterprise. Research has rarely proven causality in any domain of learning. In a concluding essay, Catterall writes, “Children may persist for years studying Latin or rote mathematics under assumptions that general mental discipline will result. Available studies say it does not… We might even think that…learning to judge the area of a rectangle would show up in ability to judge the area of a circle. Not likely say researchers.”

The web of connections between learning in different domains is extraordinarily dynamic and may well include psychological development, motivations, and attitudes, as well as skills and knowledge.

So it should not come as a surprise that there is disagreement among the experts who contributed to Critical Links on the question of transfer from the arts. Ellen Winner and Lois Hetland are among the skeptics. They conducted several of their own meta-analyses of research on transfer from the arts that are included in the compendium. The title of one, “Mute Those Claims: No Evidence (Yet) for a Causal Link between Arts Study and Academic Achievement,” sums up their estimate. They argue that when subjected to the rigorous statistical demands of meta-analysis (a technique designed to draw broader conclusions than any single study could make by interpreting data from diverse studies), the connections, while real, fall short of the requirements of causality. [See the article “The Arts and Academic Acheivement” for another perspective on this meta-analysis research.]

Other contributors to Critical Links do believe that causal links are established. Commenting on drama research, Robert Horowitz and Jaci Webb-Dempsey argue that meta-analysis is insensitive to the most important connections, but that qualitative research establishes transfer between the arts and “positive cognitive, personal, and social outcomes…represent(ing) capacities central to the goals society typically articulates for public education — productive social membership, critical and higher order thinking, and commitment to the skills for lifelong learning.” Larry Scripp, commenting on music research, makes an even greater claim, arguing that it has produced “generative neurological and cognitive frameworks for learning transfer.”

Confusion over the nature of transfer itself may be at the root of the disagreement. The standard model of transfer is linear and mechanical: one learning input leads quickly and directly to a different learning output. But the web of connections between learning in different domains is extraordinarily dynamic, and may well include psychological development, motivations, and attitudes, as well as skills and knowledge. So, if one looks for quick and direct transfer from learning in the arts, one is likely to find nothing at all.

How People Learn, a recent publication of the National Research Council edited by John Bransford, Ann Brown, and Rodney Cocking, begins a serious effort to reconceptualize transfer, which it places at the very heart of the learning process. “All learning involves transfer from previous experiences,” it claims, describing a dynamic process in which people plumb what they already know, identify and evaluate what may be relevant, and translate it for new circumstances. These activities may be entirely unconscious, but they are quite real. Good teachers have a pedagogical repertoire that helps students perform these tasks.

Scripp follows a similar line of reasoning into the world of teaching and learning in school. He proposes that if learning is a process of integrating knowledge from multiple domains, then teaching will be most effective when it, too, is integrated. The new frontier in arts education will be maximizing transfer through curriculum and pedagogy that is “circular” rather than linear. A musician himself, Scripp runs a music charter school in Boston for the New England Conservatory of Music. He hypothesizes an integrated music/math curriculum that will maximize learning in both subjects by strategically linking “concepts shared by both disciplines.”

Scripp, Horowitz, and others in Critical Links remind us that if the arts are going to have the power to improve learning more generally, instruction in the arts must be rigorous and learning must be deep. This should reassure the essentialists who fear that the instrumentalists' demands for arts education to support other learning will be detrimental to the quality of the arts instruction. Transfer only occurs when the quality is high. On this the essentialists and the instrumentalists can agree: all children need high quality arts instruction.

One of the ironies of Critical Links is that few of the studies investigate what happens when children actually receive high quality arts instruction. Like education research generally, most Critical Links' studies explore small questions over short periods: Does dramatic enactment improve story comprehension? Does keyboard instruction improve spatial reasoning?

Some exceptional studies transcend these limits. Steve Seidel's study of a Shakespeare program for high school students lasted two years and considers a multitude of variables. Seidel needs to use poetic language to capture the richness of the teaching, learning, and achievement he found. Several studies in the multi-arts section, particularly those on the A+ Schools and the Chicago Arts Partnerships in Education, capture the fullness of the arts as a catalyst for improving school culture, raising standards, building links between schools and communities, enriching learning environments, motivating teachers, and engaging reluctant learners.

Sixty-two studies may seem like a great many as the reader pours through the compendium, and collecting them in one place focuses their power. But there are 20,000 members of the American Education Research Association, and nearly 1,000 education-related journals in the U.S. The studies in Critical Links are less than a thimbleful in an ocean of education research and policy discourse. And it is not certain how much influence that ocean actually has on education policy in any event.

The case for the arts may not be fully made until a new comprehensive theory of learning is developed that acknowledges the many ways of thinking, knowing, and representing that are available through the arts. But Critical Links has made terribly important contributions. It has established a basis — high quality arts education — on which to reconcile the essential and instrumental cases for arts education. It has shown clearly two paths for new research. One path will investigate the structural and neurological relationships between learning in the arts disciplines and other learning. The other will explore how to deliver high quality arts education in real educational settings that maximize those relationships.

Nick Rabkin is executive director, Chicago Center for Arts Policy at Columbia College Chicago, where he leads Learning and the Arts, a program to advance professional understanding in philanthropy of the value of arts education. He was senior program officer for arts and culture at the John D. and Catherine T. MacArthur Foundation from 1991 to 2001.

Critical Links: Learning in the Arts and Student Academic and Social Development is edited by Richard J. Deasy, published in 2002 by the Arts Education Partnership. Copies are available from AEP, One Massachusetts Avenue N.W., Washington, D.C., 20001-1431, 202-638-5770, www.aep-arts.org

The Arts and Academic Achievement

What the Evidence Does (and Doesn't) Show

Dale Rose and Michaela Parks

The impact of the arts on academic outcomes has become one of the truly hot topics in the world of arts education. The most recent evidence of this is the compendium of research summaries, Critical Links, mentioned above. This extremely valuable effort was preceded late in 2000 by a report from researchers at Harvard's Project Zero that included nearly all of the same studies reviewed in the Critical Links compendium. Project Zero researchers, however, took a different approach to reviewing the research. Their review was a heavily technical analysis that combined findings from all of the studies that they could find to date regarding the impact of the arts on cognitive and academic outcomes. An advantage of the method used by Project Zero (meta-analysis) is that it allowed the reader to draw singular conclusions from large groups of studies.

The Project Zero review, titled “The Arts and Academic Achievement: What the Evidence Shows,” appeared in an invited double issue of The Journal of Aesthetic Education published in the fall, 2000. Edited by Ellen Winner and Lois Hetland, the special issue summarized all the research they could find on the relationships between studying arts disciplines (such as music and drama) and achieving specific educational outcomes (such as reading achievement and math scores). Having conducted dozens of program evaluation studies in arts education over the last eight years, we were quite enthusiastic about this technical review of the research (though it definitely took us a while to read its over 300 pages!). It would be difficult to overstate the value of locating, coding, and analyzing the results of all these studies.

Having read the entire issue, it occurred to us that not everyone interested in the findings would read all 300 plus pages, especially since much of the report involved technical language. This occurred to the Project Zero authors as well, and, in addition to the ten papers included in the Journal, they also wrote several less-technical summaries of the findings in which they presented their conclusions about the implications for the field of arts in education. Unfortunately, from our perspective the interpretations and conclusions offered by Project Zero researchers often missed some of the more salient findings of the research they reviewed. Additionally, many of the meta-analyses included far too few studies to allow for reliable conclusions. As a consequence, we felt that arts educators and education policy makers who read the Project Zero summaries might be making ill-informed decisions regarding the role art should play in contemporary education. Additionally, much of the technical information in the journal might be missed by readers who are not statisticians.

What follows is a brief, non-technical summary of what the research reviewed by Project Zero tells us about the impact of the arts on academic achievement and other non-arts outcomes. We include a summary of both equivocal and unequivocal findings and conclude with implications for educators and funders.

The Review Process

Project Zero researchers generated ten articles summarizing the findings from over 250 published and unpublished studies. Each article covered a different topic area. All but one of these articles used a method known as meta-analysis that combines the findings of similar studies in order to draw conclusions based on a large group of studies. The method was first introduced in the 1980s and has become a very useful tool for advancing knowledge because it allows for “meta conclusions” about a particular phenomenon. Basically, the meta-analysis method involves 1) identifying specific research questions, 2) finding every study ever written that attempts to answer these questions, 3) reducing the large initial pool of studies down to a set that are similar enough to be meaningfully combined, 4) boiling each study down to one number (an “effect size”) that indicates the strength of the relationship that was found (i.e. how strong was the music-study/math-achievement association?), and then 5) combining the “effect sizes” from among all the studies. The result is one grand “effect size” that indicates how strong an association there is between, say, music instruction and achievement in math.

Unequivocal Findings: What we know about arts-academic skills

A finding is “unequivocal” if there were enough studies in the meta-analysis to draw unambiguous conclusions and if the findings show a clear impact (one way or another) of the arts on well-articulated outcomes. Though there are no hard and fast rules for the minimum number of studies, we used the Project Zero researchers' own notions of “too few” to guide our selection. For example, the Project Zero researcher Butzlaff, in discussing the relationship between music instruction and reading skills, indicated that the six relevant studies were “a very small number.” (p. 176).

The following conclusions are clear and unambiguous findings from the body of research on arts-academic achievement. The clearest findings were related to the impact of music and drama — on math and reading, respectively.

Music

Three research questions produced clear results related to the impact of music. Each question and the subsequent findings are presented below. For each research question, we pulled quotes directly from the Project Zero researchers to summarize the clear and largely unquestioned conclusions from the research.

Research question: Does music instruction increase spatial skills?

• “Music instruction clearly enhances spatial skills.” (Hetland, p. 226)
• “The effect [of music on spatial skills] cannot be explained away by a Hawthorne effect, nonequivalence of experimental groups, experimenter bias, or study quality. It is a solid finding.” (Hetland, p. 220)

These conclusions were based on two meta-analyses containing a total of twenty-three studies. Hetland suggests that the finding is so clear that we should use it to inform instruction in the classroom as well as leveraging the information for political lobbying.

Research question: Is studying music associated with higher math scores?

• “Yes. A small [statistically significant] association between the voluntary study of music and mathematics achievement was found when twenty studies with correlational designs were combined.” (Vaughn, p. 163)

Research question: Does listening to music improve performance on spatial temporal tasks?

• “Is there a ‘Mozart Effect?’ Yes, there is. It is limited, however, to a specific type of spatial task that requires mental rotation in the absence of a physical model.” (Hetland, p. 136)

Drama

The Project Zero researchers reviewed hundreds of studies addressing a variety of topics related to drama's impact on academic achievement. The primary effects researchers found were related to drama's impact on verbal ability. As with the findings for music, we provide direct quotes from the Project Zero researchers regarding clear conclusions related to the impact of drama.

Research question: Does classroom drama have an effect on children's verbal ability?

• “The results of the seven meta analyses [containing 107 studies] show clearly that the answer is yes. Drama instruction has a positive, robust effect on a range of verbal outcomes.” (Podlozny, p. 264)
• “The results of these meta-analyses are very encour-aging for educators who wish to use drama in the classroom to promote deeper learning in a variety of verbal domains. Clearly, drama is an effective tool for increasing achievement in story understanding, reading achievement, reading readiness and writing.” (p. 268)

Equivocal Findings: Questions left unanswered

Many of the meta-analyses conducted by Project Zero included far too few studies to allow drawing any reasonable conclusions. Whereas there are no hard and fast rules about the minimum number of studies to include, most meta-analyses in other fields summarize twenty-five or more studies. For example, a well-known meta-analysis that examined the effectiveness of employment interviews included 245 studies, and another that specifically looked at racial differences in employment interviews (a far smaller arena) included thirty-one studies. In the case of arts education, the number of high quality studies1 available is quite limited. The largest meta-analyses among the ten articles in this journal involved twenty or so studies and most of the meta-analyses contained about ten studies.

The bottom line is that a meta-analysis summarizing as few as six studies is of dubious use. In many cases, Project Zero drew conclusions from meta analyses with far too few studies to justify strong statements. In some cases, Project Zero researchers even point out the limited numbers of studies available, yet proceed with their analysis and with drawing unfounded conclusions. Indeed, a more appropriate treatment of these cases would have been simply to point out the infrequency of high quality studies and to call for more research.

In the following areas more research is needed before a clear meta-analytic finding will be appropriate. The answers to the following research questions are not yet clear.

Music

Research question: Does music instruction increase general IQ? Hetland only found three studies that addressed this question — far too few to warrant drawing any conclusions.

Research question: Does music instruction increase reading skills? Butzlaff found that music instruction did significantly increase reading skills, but dismissed the finding as “neither large, robust, nor reliable” (p. 176) because he found too few relevant studies (six) to be confident in the findings.

Research question: Does music training cause increases in math achievement? Vaughn concluded the answer to this question was a qualified “yes.” “A small causal relationship was demonstrated when six studies were combined. However, it is noteworthy that six studies is a very small number.” (p. 163)

Visual Arts

Research question: Do students who study visual arts score higher on paper-pencil tests of creativity than students who don't study visual arts? Moga, Burger, Hetland, and Winner found only four studies addressing this question, meaning that drawing conclusions is unwarranted. The combined findings suggested “a modest association between studying arts and performance on creativity measures.” (p. 102)

Research question: Does studying visual art increase student's creativity using verbal measures of creativity? What about using figural measures of creativity? Moga, Burger, Hetland, and Winner found only three studies addressing each of these questions, meaning that drawing conclusions is unwarranted. The authors, however, concluded there was “modest evidence for a causal relationship between arts study and creativity measures, but only when the creativity was figural. When the measure was verbal/conceptual, no evidence for a causal relationship was found.”(p. 102). The authors went on to point out that “conclusions are strongly limited by the dearth of experimental studies found.” (p. 102)

Research question: Does studying visual art increase student achievement in reading? According to Burger and Winner, “When [visual] art instruction is not integrated with reading, such instruction has no effect on reading achievement scores, but has a moderate effect on reading readiness scores.” (p. 291)

This finding was based on nine studies, five of which used reading achievement as the outcome. The five achievement studies did not demonstrate positive results, but the reading readiness studies did. In their own words the “effect [on reading skills] was carried entirely by studies whose outcomes were reading readiness test scores.” Thus, the story here is muddled, and we need more conclusive information before drawing any strong conclusions.

Dance

Research question: Can dance instruction improve reading? According to Keinanen, Hetland, and Winner, “The results of this small meta-analysis [containing four studies] are equivocal and do not support a conclusion that dance instruction serves as an effective means of teaching reading. While our combined studies revealed a significant relationship between dance instruction and reading achievement that would likely have been found again had other subjects been selected for these studies, we cannot generalize this result to new studies. The conclusions are further limited by the very small number of relevant studies found.” (p. 300)

In other words, the four studies they found indicated that dance instruction does improve reading, but because they could only find four studies they don't feel confident in drawing a conclusion.

Research question: Can dance instruction improve non-verbal reasoning? Keinanen, Hetland, and Winner found, “Dance instruction does lead to improved visual-spatial skills. However, our conclusions are strongly limited by the fact that our analysis was based on only four studies.” (p. 303) In other words, the initial data suggest that dance may have an impact on non-verbal reasoning, but we really need more data before we can be confident in saying so.

So, what's an educator to do?

First, use the results we have. The clearest findings suggest that music instruction can increase math ability and that drama is an effective tool for enhancing reading skills. Also, stop playing Mozart during tests. (Well…at least stop hoping that doing so will enhance math and reading results!)

Next, ignore the results we don't have. Educators looking to confirm their own beliefs about how arts can/can't/should/shouldn't be used in the classroom may be tempted to over-generalize from the inconclusive findings of the Project Zero meta-analyses. We suggest not giving too much weight to conclusions from too few studies. One of the greatest mistakes we can make is to emphasize too strongly the findings that confirm our opinions and to dismiss too easily the findings that contradict our opinions. Rome was not built in a day, and neither will our knowledge in this area. We need to wait until adequate data are available before drawing conclusions.

So, what's a grantmaker to do?

First, use the results we have. If you are funding arts programs in schools but also would like to see some “basic skills” benefits, here is your chance to get your cake and eat it too. Grantmakers can use the Project Zero findings to select arts programs targeted at non-arts achievement-oriented outcomes. For example, the data clearly demonstrate that learning to play music can enhance math-related abilities but just listening to music is not likely to achieve similar results. It is important to point out, however, that arts programs have the greatest impact on academic achievement when they are part of an arts-integrated curriculum developed explicitly to enhance academic outcomes.

From a policy perspective, the Project Zero findings have tremendous potential for increasing the role of the arts in education in the United States. For instance, if students currently spend two hours of every school week engaged in art-making and the rest of their time working on “basic skills,” there is a legitimate argument to be made based on these findings that some of the “basic skills” time should involve arts-based instruction. In fact, contrary to the view that arts instruction takes away from time spent on basic skills training, such a shift may actually increase student achievement.

Next, DON'T ignore the results we don't have. Many of the meta-analyses described here should not have been done. If only four studies address a specific research question, there is no point to a meta-analysis that computes a combined, quantitative result — except as a way to assess the need for future research. When so few studies are found, the conclusion might better be “we need more studies,” and that's about all we can reasonably say. For example, we only have six studies that examine the causal link between music instruction and math achievement. These six studies show promise, but we need more research to know what the true effect is. Also, we need more studies to explain why these effects occur. Grantmakers have an opportunity to sponsor new, well-designed research to fill the gaps in knowledge that have been identified by Project Zero researchers.

Conclusion

Overall, we were very encouraged by the Project Zero studies. Although more questions were raised than answers given, the answers we did get were clear regarding linkages between the arts and achievement in other fields. While we know for ourselves the value of the arts in and of themselves, Project Zero research provides strong evidence for art's more distal effects, that is, for effects found far from its intrinsic value. Instruction in (and through) music and drama can have a clear effect on non-arts disciplines such as reading and math. However, considerably more research is needed before drawing any strong conclusions beyond drama and music. Here, grantmakers can make a real difference by funding evaluation and adding the needed research to extend the initial evidence to other arts disciplines.

We were also encouraged by Vincent Marron's comments (GIA Reader, vol. 12, no. 2, summer 2001) suggesting that a large body of new research is under way that explores more deeply the role of arts in school reform. As Marron pointed out, the lens of the studies included in the work by Winner and colleagues was necessarily limited in scope. It will ultimately be the combination of these efforts with the more multi-faceted studies Marron described that will complete the tapestry of knowledge about the role art can play in education.

Dale Rose, Ph.D. is president, 3D Group in Berkeley, California. Michaela Parks, Ph.D. is vice president of evaluation, 3D Group in Berkeley, California.

Published references

Ellen Winner and Lois Hetland were guest editors of a special issue of The Journal of Aesthetic Education: “The Arts and Academic Achievement: What the Evidence Shows,” volume 34, numbers 3-4, fall/winter 2000. This special issue includes the following articles, many of which are quoted here.