Singing Our Praises

2004, 171 pages. Commissioned by Association of Performing Arts Presenters, Washington, D.C. 20063.

While reading Suzanne Callahan's enlightened and enlightening book Singing Our Praises: Case Studies in the Art of Evaluation, I underlined passages that I thought made the book significant. Then I sat down to write a review of the book for the Reader and casually glanced through a reprint of a review by Norma Fleischman and Kathlyn M. Steedly, published in the American Journal of Evaluation. The quote from Singing in the first paragraph caught my eye. It was a passage I had underlined. I looked closer at the review. Fleischman and Steedly had pulled out the same quotes, the same points I intended to write about, and they were drawing the same conclusions. There is an art to evaluation and Callahan has captured that in her book. So, I thought, I don't need to write a review that makes the same points. I will simply write a preface to that review, which I have done. And here is the review.

Sarah Solotaroff is program director,
Chicago Arts Education Initiative.

Bravo, Suzanne Callahan, for writing a comprehensive guide to arts evaluation that clarifies the evaluation process for arts and cultural organizations. Singing Our Praises has a simple and profound message: "Once informed, you can take control of the evaluation process rather than feeling that the process controls you" (p. 9). Singing Our Praises is a valuable tool kit that details evaluation vocabulary, methods, and resources focused on building the evaluation capacity of arts and cultural organizations.

Singing Our Praises seeks to demystify the evaluation process through providing concrete examples from the performing arts world. Using two case studies (Chicago Dancemakers Forum and Improbable Theatre) as the basis to explain the technical aspects of evaluation, the book directly addresses the evaluation needs of arts organizations and a wide range of nonprofit, community-based organizations that are faced with the seemingly daunting task of defending their very existence through evaluation. It is important that through the case study examples, Singing Our Praises details the benefits of evaluation to multiple dimensions of an arts organization's well-being, such as audience development; market research; impact on artists, audiences, and staff members; and, all-important, fund-raising. With deep sensitivity to the cultural context, economics, and daily life of arts and cultural organizations, Singing Our Praises provides a systematic and accessible map for navigating several fundamental yet complex evaluation processes. The book addresses head on the artistic dilemma of measuring the seemingly "intangible" nature of the creative process and its outcomes and provides real examples that highlight how such intangibles can be measured.

The philosophy of participatory evaluation provides context and tone for the technical aspects of the book. Artists are equal partners in the evaluation process, and evaluation questions are collaboratively and locally defined in the spirit of relevance, not as a reaction to an outside agenda. Evaluation data, within a participatory framework, are more broadly understood to be information useful to an organization rather than an externally driven set of answers to seemingly random questions. Finally, participatory evaluation asserts that rigor is found in carefully crafted evaluation questions, evaluation designs that respond to local contexts from multiple perspectives, and thoughtful analysis and reporting of evaluation information. Singing Our Praises shows how the evaluation process parallels the creative process. The knowledge, skills, and understandings critical to the artistic process translate into the knowledge, skills, and understandings necessary to design a participatory evaluation. For example, the ability to think critically, observe closely, and make new connections with ideas and information are fundamental to both making works of art and evaluation. In championing a participatory evaluation strategy, Callahan seeks to shift the evaluation discussion to one in which evaluation becomes a tool for programmatic progress, information gathering and sharing, and reflective practice.

Most important, Singing Our Praises provides a comprehensive set of evaluation tools and resources. Using the case study frame as organizer, the book is a guide that ties the technical aspects of evaluation to the case study narratives. Singing Our Praises consolidates years of qualitative and quantitative evaluation research and clarifies the value and uses of both these complimentary traditions. The book provides easy-to-follow templates as guidance for arts organizations to develop their own logic models and also provides realistic guidance about when to secure and how to work with outside evaluators. Singing Our Praises takes care of the many foundational aspects of evaluation for arts and cultural organizations. The chapter titled "Try This at Home" addresses evaluation planning, implementation, analysis, and reporting. Callahan's careful attention to the details of evaluation and her caring voice provide a solid base for arts organizations to get a jump start on the evaluative dimensions of their creative work.

Singing Our Praises does not address arts evaluation within context other than performing arts organizations or evaluating arts-related work that occurs within disciplines such as education or health care. Callahan understandably focuses Singing Our Praises on participatory evaluation strategies within performing arts organizations, a large enough topic to comfortably digest. Although they are not directly addressed, arts and cultural organizations, not just performing arts organizations, will find Singing Our Praises useful. Librarians and museum curators, for example, will find technical aspects of this work particularly helpful as they seek to draw from their specific disciplines' literature and practice and develop arts-centered, locally driven evaluation practice. The art forms may be different, but the evaluation needs are definitely related.

Perhaps a larger leap will need to be made by arts educators who read Singing Our Praises. Arts educators often struggle with the competing demands of the arts and education worlds. Callahan and other authors who assert participatory evaluation strategies un-earth common ground while respecting the unique needs of the disciplines. Callahan forwards participatory evaluation practices that allow for communication across disciplines, validate the multiple perspectives of artists and educators, and meet their evaluation needs. The participatory nature of Callahan's work is the key to its broader utility. Arts educators will need to take Callahan's suggestions (mindful of arts education-specific issues, resources, and literature) and tailor their evaluations to the educational context that is often less concerned with audience participation than measuring standards-based learning, for example. Arts educators can evaluate their work using participatory strategies and meet the diverse requirements with which they are confronted.

Singing Our Praises pushes the evaluation conversation farther than the title suggests. Callahan does not suggest that evaluation serves to simply laud an arts organization in a facile attempt to secure funding. Methodologically speaking, Callahan encourages arts organizations to position themselves at the center of the evaluation conversation, drawing on best practices from the evaluation field. As previously articulated, Callahan provides thoughtful examples and tools that can be applied by arts organizations in the pursuit of strengthening programming. Singing can be understood to be sharing the answers to complex, authentic questions, and praise involves locally identifying programmatic goals and outcomes, individually defining measures of success, and reporting the progress toward those goals and outcomes. Callahan puts the notion of artistic integrity front and center as she urges the arts field to tell its own story. Singing Our Praises successfully contributes to the body of literature that seeks to build evaluation capacity of organizations, and it does so in an aesthetic and thoughtful way.

The review by Norma Fleischman and Kathlyn M. Steedly is reprinted, with permission, from the American Journal of Evaluation, Vol.27 No. 1, March 2006 134-135.