Introduction and Overview of Method

Work is busy. Resources are scarce, time is limited and the next Monday is waiting. A common challenge many executives are facing is keeping focus in a busy life and making efficient use of the limited resources available… not only for the everyday operations and processes, but especially to drive the organisation forward by improving the status quo, overcoming current challenges and tapping into new opportunities.

If you are in the same situation, the Impact Experiment Canvas might be a tool to try out. We know: Not every challenge is the same and work is different. But over the last couple of years many executives reported back to us that a structured Impact Experiment Canvas helped them to make better use of their resources at hand and boost progress towards desired results. In case you are now scared of all the work required to learn about experimentation boards, no worries, it’s mostly common sense – nicely packed.

Impact Experiments are a method that combines best practices from design thinking, improvement iterations, and entrepreneurial venturing. It fosters a learning mindset that seeks progress over absolute answers. It’s a tool that aims to move forward and try things out, whilst at the same time gaining clarity over the results and required processes. Impact Experiments also seek to establish the foundation of a learning organisation as their intention is not only drive results but rather focus on the learning and benchmarking of results against desired success criterions. By doing so, it helps to evaluate the results against our own predictions and hence brings us into a position in which we are not interpreting results but benchmarking them. Hence, Impact Experiments can help not only to structure initiatives better, but also create clarity in the organisation and foster required organisational skills needed in the 21st century.

Let’s start by looking at some of the ideas behind Impact Experiments, beginning with the advantages of divergent and convergent thinking. Each and every step of the Impact Experiment Canvas follows the divergent and convergent thinking method which has been made popular by design thinking schools. This means that Impact Experiments help us widening our perspective through divergent thinking by forcing us to think beyond the first – and often obvious – answers.

In contrast, once we have widened our perspective, convergent thinking helps us to gain clarity along the potential opportunities we created. It’s the analytical and rational mind being at work figuring out what is feasible,viable and also desirable amongst all the great thoughts we just had.

Further to that, Impact Experiments are following an iterative process throughout. This means not only to review experiments regularly, but also to learn constantly about what has already been done. The iteration follows three basic steps:

• Step 1: Preparation of the experiment by gaining clarity of problem statements and potential solutions
• Step 2: Designing the experiment by defining required ressources and the intended impact
• Step 3: Evaluating the process regularly by accessing results and reviewing learnings

After Step 3 we are iterating over the whole process again and updating for the next iteration. Keep in mind that maybe you do not need to update anything and continue as per the previous iterations in case you see your experiment is working as intended.

Download the Canvas as PDF here: GILT Impact Experiment Canvas

Working with the Impact Experiments Canvas

Step 1: Prepare your experiment

1.1 Problem Statement 

A solid house can’t be built without a foundation, right? So building one that is able to hold the construction is indispensable. The foundation can’t be changed later without tearing down everything that was built afterwards, so one should have reflected very well on how to set it up. The foundation of the Impact Experiments Canvas is the problem statement.

Before we start thinking about anything else in our Impact Experiment, we want to understand the problem at hand and formulate a problem statement with for example a strong “How-might-we-Question”. This question and the problem statement becomes the foundational input for all other steps of our Impact Experiment and will lead us accordingly.

To work out a good problem statement, it’s important to deeply understand the problem or challenge you are facing. A typical mistake being made within the problem space is the description of a superficial – almost symptomatic – problem that is not addressing the root cause at all. This happens due to the urge of formulating the problem space quickly, the cognitive bias of thinking what the problem is without any evidence or proof of this or simply the fact of laziness working towards the first principle of the problem space. 

To overcome this and design more meaningful problem spaces some simple techniques have proven to be working excellently (amongst other more complex methods).

The simple techniques are basically asking good questions like:

• What is the problem? Followed by asking five times “Why?”.
• When does the problem emerge?
• Why is it a problem?
• Are there times when the problem is not existing?

If these tools are too simple for your liking, you can try out a more advanced technique like Context Mapping.

1.2 Customer/Persona

After you gain clarity of the problem space and the problem statement you want to understand who is affected by it. It’s a simple question that is often not that easy to answer. For a reason a whole research discipline about customer centricity, targeting and user segments has been in the spotlight over the last few years. A often used concept is “customer centricity” which unfortunately is often being put into the corner of marketeers and business developers. However, the idea is not necessarily limited to a customer that is buying a product or service from your organisation.

The customer should be more seen as someone that is affected by a problem space that you are willing to solve. This could be anything from an inefficient process within the organisation that makes your employees (customers) slow in the operations to structural challenges that hinder innovation.

When working with customers or more generic personas, we want to understand whom we can serve best in the first place. Even better, we want to understand who might be the early adopters of our solution, the ones that suffer the most from the problem or that are so excited to get their hands on a needed solution that they gratefully test unfinished or rough first prototype solutions.

Again, you can work with simple questions like for example:

• Who has the problem? How is your user/ persona affected by it? 
• What are the customer’s demographics?
• What are their interests?
• What do they hope to archive?
• What pains do they have?

You might also want to work with more advanced techniques like the Empathy Map or the Persona Canvas. 

1.3 Potential Solution

If you manage the first two steps you come to the fun part of a fun process: Creating and developing solutions. It all starts fairly easy with brainstorming as many ideas as possible based on your problem statement and the Persona profile you have archived.

Important for this particular step, you do not need to necessarily deliver your solution yet. This is part of running the experiment after completing Step 2. Here, we only want to conceptualise and only draw out the first paper prototype at the maximum. Describe a potential solution and then move on to 1.4 and 1.5.

This part is all about being creative, a simple brainstorming combined with paper prototyping will likely be sufficient but if you want to use more advanced techniques consider following the Walt-Disney-Method or the 6-3-5 Method.

1.4 Assumptions / Hypothesis

Great, you have a solution in mind that may solve the problem at hand. Let’s start building, right? Not quite, we need to gain clarity of the assumptions we are going to build our solution on.

In general, everything that you can’t prove is an assumption and as the solution we just defined is the result of a creative process, we can’t prove it (unless it’s a solution that we can copy exactly the same under the same circumstances). Hence, we are in need to gain clarity over the critical assumptions that are part of the solution we defined. The reason we want to define these assumptions is to cluster them into their level of riskiness for the success of the solution in order to validate them first before investing a lot of time, energy and other resources just to figure out that our assumptions do not hold true.

Defining assumptions is not the easiest task as it’s often difficult for us to identify what we believe to hold true and what is actually true. There are two questions that can help you identify assumptions in your solution. These are: 

• Which part of the solution do we feel most uncomfortable with?
• Which part of the solution gets challenged most by people that we present the solution to?

In case this does not help to gain significant clarity, Hypothesis Boards (similar to what is being done to define research questions) and the Riskiest Assumption Canvas are suitable tools and frameworks to work with.

1.5 Success Criterion 

Last step of the preparation is to define your Success Criterion. How will you measure the success of your solution? Even though it may be the easiest part of the overall preparation process, it’s one of the most important things for the iterations you are going to facilitate.

The Success Criterion is the benchmark for the whole experiment and everything will be measured against it. It’s already mentioned here, that the main purpose is not to reach this success criterion at all cost, but to have a cornerstone that helps us interpret our results because numbers alone are not only what counts, they need to be measured within context.

To make this easier to understand, let’s take a simple example of running a 5k race. You define your Success Criterion as finishing the race within 25 minutes. It’s the race day and you finish your race in 30 minutes. In pure measurement terms, you come to the conclusion that you underperformed and didn’t reach the Success Criterion. However, it was not your day and mid-race your running shoes decided to tear on the right side of your foot and for the rest of the race you struggled to keep pace. Not nice, still no success but you may come to the conclusion that you still ran a good race.

The same we want to do with our experiment in the future, we do not only want to measure in absolute terms but within context. Also, in contrast to the running example where we can establish rather precise success criterions based on our training performance, in business context it’s often much harder to predict a “success”. How fast can a process really go? When are your goals stretched and when not?

Again, Impact Experiments are progress driven and aim at learning. Reaching the goal is great but not the main priority of the experiment. One of the best ways to define Success Criterions is to work with North-Star Metrics.

Step 2: Design for Impact

2.1 Ressources

This step is exactly what it sounds like: Make a list of all the resources you need. Typically, this includes time, material, budget, and so on. This is the obvious thing. But don’t forget about the less obvious input, either. This includes means like the buy-in of critical stakeholders, space, knowledge etc. After all the time you spent on reflecting and evaluating, you can decide best which resources are needed to realise your idea. Think about the best but easiest way to test it. Bring this down in a structured way, transparent for everyone to see and commit to your final input.   

Questions to ask:

• What are you working on?
• What will be done?
• Is everybody committed?
• Do we need external advice?
• Which materials will be needed?

2.2 Action

Taking action needs to be structured. Therefore, before you go ahead, make a plan. Define critical steps, tasks and milestones. Think also ahead and define the checkpoints at which you want to evaluate your progress and also benchmark against your success criterions. 

Questions to ask:

• What exactly needs to be done?
• What are the single tasks that need to be accomplished?
• Who is responsible for doing what?
• What is the timeline?
• How will we be working?
• Are there dependencies that we need to consider?
  

2.3 Output 

What are you aiming for? Or, in other words, what Output will be provided by the required resources identified, paired with the actions, you just defined? Output in these matters means no other than tangible results. This could be a prototype of a product, a strategy paper, a new work organisation or similar. Think about what you intend to “hold in hand” when done. Make it as concrete and precise as possible.

Questions to ask:

• What exactly are we aiming for?
• What features should our prototype have?
• What are the minimum requirements?

2.4 Outcome

The outcome describes the way things will change for the better within a shorter time frame. In contrast to the output, outcomes are already looking at implications of our intended solution and not only the pure “facts” like the features we have built. The main question to answer is: “What do we wish to see happening as soon as the solution is implemented?”. Try to make a clear forecast if and how the solution will bring the expected benefits.

Questions to ask:

• How will the outcome lead to positive results?
• What are maybe also negative impacts?
• What exactly will change by implementing the output?
• What will be a benefit?
• How will this benefit make an impact on the overall company, the team, and so on on a short term basis?
   

2.5 Impact 

Think ahead, what impact do you want your solution to have in the long run? What changes do you hope to see? Impact is all about accessing and predicting the mid- to long-term implications of your solution. Like many things in life, next to the short term gains you will likely also see long-term gains that need time to materialise within the organisation.

Question to ask:

• What will change over time?
• Does your solution cause a cascade of consequences or will it be an isolated event?
• Is there any risk to consider?
• Does the solution have long-term consequences on our persona? 

Step 3: Evaluate your experiment

Finally, you prepared and designed your experiment and have been working on it the last weeks. Time to reflect!

3.1 Learning Experience

If you ask 20 people to tell you something about a very simple object, for example a stone, you will get 20 different stories. Accessing the experiment on a regular basis helps to take different perspectives into consideration and make sure progress is being evaluated.

One of the key characteristics of working with Impact Experiments is that we are working in areas of uncertainty and based on assumptions that we aim to validate (or refute) over the course of the experiment. Hence, learning is considered one of the most important outcomes out of the experiment. Every learning is progress and the main purpose of working with Impact Experiments.

Questions to ask:

• Did we have surprises along the way?
• What was expected?
• What was unexpected?
• Are the insights that are worth sharing?
• How can the last iteration help us with the next one?

3.2  What is your result? 

And finally, you also want to access the results from the current iteration and benchmark these against the defined success criterions, output, outcome and impact.

The success criterion helps you to identify how well your experiment delivered over the iteration. Especially in this step it becomes very clear why defining success criterions before the start of the experiment is so important: The assessment has no wiggle room for speculation. You either reached, overperformed or underperformed against the success criterion. This brings you into a strong position to then look at the output, outcome and impact.

Let’s assume you underperformed against the initial criterion but at the same time the output, outcome and impact delivered already the results you have been aiming for, then you might accept the experiment as successful and review your success criterion as overly ambitious and not needed to that extent to archive the results. The same goes into the other direction if you over perform but don’t find the intended results in your assessment.

And again, this draws the relevance for progress over goals when working with Impact Experiments.

Final thought

Impact Experiments and the Impact Experiment Canvas are nothing complicated nor difficult to use. Its purpose is not to introduce you to a fancy new way of working but rather to support your focus, reflect and use your resources as meaningfully as possible. It’s a tool to try things out and learn along the process. It certainly won’t solve all challenges but why not give it a try and add it to your leadership toolbox?