Member login  
Forgot Password? | Not a member?
Member login
Picture of Ron Evans

A question of bias and balance #Digilab

How many times is too many times when it comes to testing an idea until it ‘succeeds’?  Ron Evans, a mentor on the Digital Lab, examines the layers that encapsulate our attitudes and approaches to being flexible when we are giving things a ‘good go’.

Picture of Ron EvansAfter working with hundreds of nonprofit cultural organisations, I’ve been in a lot of conversations about marketing strategy. “Oh, we tried that, and it didn’t work” is something I hear often. When I probe more deeply, I find out that the strategy was tried perhaps a couple of times, and then abandoned, and the person implementing the idea wasn’t enthusiastic about it in the first place.

This is a particularly nasty form of confirmation bias, which is defined as the tendency to interpret new evidence as confirmation of one’s existing beliefs or theories. When we don’t believe something will work, it’s easy to find evidence to that effect. But there are a variety of other factors at play. Was the idea implemented correctly? Was there an objective method of measuring effectiveness? Was the time that the experiment ran long enough to get usable results?

When we don’t believe something will work, it’s easy to find evidence to that effect.

As Digital Lab researchers, one of our objectives is to combat personal bias. We do that by recognising and correcting for emotions we feel about a particular experiment, and actively exploring ways we may be influencing the outcomes. For example, it’s useful for us to frequently ask ourselves: “Are my feelings about this experiment influencing its execution, or my interpretation of the results?” Another way of looking at this question, which is often asked in academia, is: “Is another researcher going to see any issues of personal bias that may affect my results?” This type of introspection will often lead to dramatically improved experiments.

The science of experimentation that mentees get to practice in Digital Lab is a primary benefit of the programme. But equally important is the exploration of how we mentally approach experimental design, execution, and interpretation of results. A good researcher learns to balance both