Ai-ling Chang
Researcher. Strategist. Creative.
V716F8UMWI.jpg

Measuring Impact in the Arts

 
  Photo  by  Freestocks.org  is licensed under  CC0 by 1.0

Photo by Freestocks.org is licensed under CC0 by 1.0

 

project

M.A. Arts Administration Thesis Project, 2013

preface

Arts funding is a topic that made its way to the forefront of nonprofit issues as a result of the economic recession. However, nonprofits have always faced funding challenges—recession or not—an issue exacerbated by an over-saturation of the market. In 2009, more than 1.4 million nonprofit organizations were registered with the IRS. A large number of these nonprofit organizations are working towards similar causes and competing for the same funds. To be competitive in the quest for funding nonprofits must be prepared to identify and differentiate their organizations' impacts. For arts administrators this is particularly complicated. What is the impact of the arts on individuals and communities and how can that impact be measured? This project is situated directly in this space, attempting to identify the value of the arts to both individuals and communities and to provide a framework for arts organizations to demonstrate this impact.


background

Manifest was founded in 2004 to support the “quality presentation, experience and documentation of the visual arts, engaging students, professionals and the public from around the world through accessible world-class exhibits, studio programs and publications”. Granting agencies and funders require Manifest to demonstrate how its contributions result in vibrant neighborhoods, connected communities and thriving economies. 

Manifest's current assessment methodology measures program participation and tracks media exposure. Through these methods Manifest:

  • demonstrates how many people benefit from its services, 
  • showcases the extent of its reach across the world and, 
  • provides proof of its successes as evidenced through repeat and growing participation. 
  • Manifest also solicits participant feedback to continually research participant needs and improve the quality of its services. 
1.jpg

problem

Manifest's existing assessment practices do not demonstrate its impact on individuals or communities, or provide information on how Manifest might strategically grow its participant base.

opportunity

To develop an assessment methodology that allows Manifest to demonstrate its impact, grow its participant base and as a result, increase its access to funding.

hypothesis

By developing an assessment methodology and toolkit that targets two users: current constituents and non-participants, Manifest can demonstrate the positive effects of its programs to granting agencies and funders while also finding solutions to grow its participant base.


stakeholder research

In collaboration with the Executive Director, internal and external stakeholders were identified for participation in interviews and surveys. The goal for this research was to identify how key stakeholders defined Manifest's goals, program outcomes and impacts. 


A framework for measuring outcomes & impact

Logic models were created for each of Manifest’s programs to demonstrate how Manifest creates change. 


deconstructing outcomes

After defining Manifest’s program outcomes and impacts in logic models, outcomes were linked directly to impacts by deconstructing outcomes into measurable indicators of change. This process was informed by stakeholder feedback and secondary research.


Toolkit Development

Toolkits were created so Manifest could generate customized surveys for each of its programs. Each toolkit provides survey questions that can be used to measure a specific indicator of change, which is in turn linked to a specific outcome and impact. 


testing the toolkit

Surveys were tested on-site and online.

The following list provides a summary of key insights derived from the results of the soft launch:

  • All outcomes prescribed to participating artists and the creative public received an average score greater than four, except for Career Growth and Professional Development, which received an average score of 1.79.
  • All outcomes prescribed to viewers received an average score between three and four, except for Provide Quality Visual Art Experiences, which received an average score of 4.88.
  • All outcomes prescribed to the community received an average score above three. The lowest scoring outcome of the three that were measured was Grew Participation in and Supporting of the Arts, which received a score of 3.27.

Based on these results, Manifest can identify key areas of strength and opportunity. 

In order to use the preceding data effectively, Manifest will need to establish a baseline against which to measure each score. Manifest may decide a goal is to achieve a four or higher in all outcomes. Or, Manifest may set goals for specific outcomes.  For example, Manifest may decide success would be defined by growing Career Growth and Professional Development by one point over the course of one year.

By identifying areas of strength and opportunity, Manifest can used the data from surveys to not only communicate successes to stakeholders and potential supporters, but also to identify ways to change or even create new programs to support desired outcomes.


conclusion

Though Manifest’s mission is first and foremost to provide quality visual art experiences, the process of creating this assessment plan revealed Manifest's constituents experience many benefits. Manifest can use this plan to continually assess its performance against internal standards, prioritizing outcomes and identifying key areas of strength and opportunity. Utilizing the National Endowment for the Arts’ five year research agenda as a model, Manifest can then demonstrate how its outcomes align within recognized impact initiatives.

This plan also provides tools to identify opportunities for audience growth. The key considerations here are: what are Manifest’s goals for audience growth and to what extent will non-participant feedback impact marketing and program design decisions? 

In summary, this assessment plan and toolkit will:

  • increase Manifest's likelihood of securing funding,
  • aid Manifest's Board of Directors in evaluating performance,
  • increase organizational transparency with stakeholders and,
  • grow Manifest's participant base.

The key significance of this plan is its ability to provide Manifest with a framework of data to support the strategic decision making process.