- Venanzio Figliolino

What Do I Tell My Leaders When Experiments Fail?

Asked by Venanzio Figliolino


by John Drogosz
August 23, 2016


I’m dealing with the fallout of a failed experiment with set-based concurrent engineering (SBCE). As the product development operations specialist, I understand that LPPD experiments don’t always result in success – but my leadership team doesn't. How do I help them understand that a failure in LPPD isn’t a total loss?

The first step we always talk about is, “What was the purpose of the experiment and were we very clear with the management team about what we were trying to prove or disprove?” A lot of times at the start of a project we communicate, “We’re going to do an experiment on set-based design to see its benefits,” and we leave it very open-ended. And like anything that you leave open-ended, that leads to different expectations from different people in the organization – those who are more familiar with LPPD obviously will have a more realistic expectation of what will come out of set-based design; senior managers will certainly be more interested in the end result, which may happen in the short term or, in accordance with product development, may happen years later.

So from their perspective, if they’re not seeing instant results, they may see the experiment as a failure when the jury may still be out. Or this may be a failure in that there was an experiment conducted with a specific target outcome, and we didn’t achieve that.

In the latter case, it’s important to remember that there is ALWAYS a silver lining that occurs. Very rarely do we have a complete and utter lack of learning in an experiment. If we do, we probably didn’t do something or just stuck with the status quo.

The first thing to usually do is do a good reflection with the experiment team to analyze what went well and what did not.

Then you can hold a report-out with the management team to share your findings. And I think this is where we start getting some clarity that the glass is either half-empty or half-full.

I remember a time where we had a failed experiment that was viewed as a failure. But when the group went through the reflection, it wasn't quite as much of a failure as it was perceived. One team did their first set-based experiment on a given project and partway through, the regulatory requirements kind of changed up the project. And when the requirements changed, the parameters changed as well and the team was forced to change their design direction at that point.

Now the perception by the organization – the "talk around the water cooler," if you will – was that the team was really struggling with set-based design since they’d obviously just had to change directions. They thought that was a failure in set-based design.

Interestingly, when we sat down with the team to discuss this failure, the team did not call it a failure – they called it a hiccup! They said, “Sure, it’s true we changed design direction but only with two out of the five sets we were working with.” But the real perception from the organization on why they thought the team had failed was because the project’s cost metrics went up. People interpreted the result not being achieved as a reflection on how poorly the team executed set-based design. But once the team had set the record straight with the leadership team, the response was “Oh. That makes sense. No problem – keep working on the project.” One of the directors even repeated the team’s explanation to the rest of the organization.

It’s my experience you need at least three or four experiments before organization starts to deeply understand the nuances of SBCE. And they learn enough from their successes as well as their failures to be able to really define their own way of what SBCE looks like in their own product development community.

Now, let’s talk about what you can do to manage your leadership’s expectations while the experiment is going on. You don’t want to wait until the end of the experiment to tell management about your progress. If you do, you may be disappointed, and frankly it may be several months or years – and as we all know, management likes to see results quickly.

You want to have a series of certain touch points when you’re going through the process. When we’re doing an experiment in product development – or in lean in general – we should be just as interested in the process as the metrics. We should be checking in with management more periodically when we’re going through the process in order to inform them when the process is in control or deviating. And that’s certainly where it can be helpful to have a steering committee or some sort of leadership champion that you can go to, because if the team has a problem, there needs to be a specific organization or group they can escalate it to when needed. It is a very effective way to avoid telling management about your “failure” in retrospect.

The views expressed in this post do not necessarily represent the views or policies of The Lean Enterprise Institute.

0 Answers

Venanzio Figliolino

Process Excellence Lead & Founder of Lean Six Sigma Community

Management engineer with 15+ years of experiences in multinational companies, employed in different roles but with common goals: Continuos Improvement, processes standardization & optimization.

I'm trainer and consultant about Lean & Six Sigma methodologies, I love to help organizations to improve their performances in terms of times/costs/defects/spaces reductions.

I founded Lean Six Sigma University & Community, two synergic projects developed for creating a common place where learn and divulgate knowledges about Lean and Six Sigma methodologies applied to different ambits.

The application of Lean & Six Sigma methods are not simply methods learned and applied, it is a cultural change.