How to Measure and Assess

How To Measure and Assess

Determine what to measure

Start by determining what you want to measure. Align what you want to measure with the problem statement, so you have a sense of whether the problem is addressed by the actions you take.

Using the Theory of Change Framework (Figure 1), map out questions like:

  • What longer-term outcomes do you seek to accomplish (impact)?
  • What interim outcomes and conditional conditions are necessary to produce these longer-term outcomes?
  • What activities and supports are needed to achieve early and intermediate outcomes?

(FIGURE ONE) 

Identify sources of data

Be intentional with what data you will collect and what that data can tell you. Most people lean immediately to quantitative measures. However, quantitative data are limited and biased towards measurable considerations. Some issues cannot be captured solely through quantitative measurements, such as happiness or resilience. Look for non-quantitative measures, such as stories and visuals, which tend to be more prevalent in high-context cultures. Balance seeking information from diverse sources with being selective in what you measure to avoid information overload. Too much information can reduce efficiency in making decisions.

Determine how information is being communicated

When we measure and communicate what is being measured through numbers, it limits our understanding to only those things that be quantifiably reduced to numbers. To ensure a holistic story of impact, try supplementing the numbers with creative forms of communicating ideas, such as visuals like pictures, diagrams, infographics, and videos. Also, consider storytelling as a means of communication. Not only are stories more memorable than data, but they also communicate norms and values, develop trust and commitment, sharing tacit knowledge, facilitating unlearning, and building emotional connections.

Determine what will be assessed

Whereas measurements are often taken after the innovation is complete, measuring and assessing should be applied throughout the innovation process. Innovators should measure and assess various aspects of the context before innovating to assess its impact later. Knowing what outcomes or impacts are desirable can influence the problem definition and shape what is innovated. Not all data are equal, so prioritize the most important issues and the data related to those issues.

Analyze the data

When analyzing data, think of how the community defines success. Think beyond your organization to stakeholders directly and indirectly impacted by your innovation. Consider looking at trends over time and compare with prior experiences or with competitors’ experiences. Be careful not to be dogmatic about desired outcomes, as the outcomes of a complex system are often not within your control. Sometimes changes are subtle, such as how people tell their stories, but still provide important insight.

Determine how the insights will inform decisions and impact the future

Many conventional measurement and assessment tools focus on collecting and analyzing information in the past and project these insights into the future. However, when measuring and assessing the impact of innovation, we may not have past data or comparables readily available. It is important to use the data to inform future decisions, but do not simply project the information into the future. Use your understanding of the system to assess how the information you have collected will impact the future. Stay anchored in the desirable future and make decisions based on your assessments that will get you there.

Related