5. Process Improvement
5.4. Analysis of DOE data

5.4.6.

How to confirm DOE results (confirmatory runs)

Definition of confirmation runs

When the analysis of the experiment is complete, one must verify that the predictions are good. These are called confirmation runs.

The interpretation and conclusions from an experiment may include a "best" setting to use to meet the goals of the experiment. Even if this "best" setting were included in the design, you should run it again as part of the confirmation runs to make sure nothing has changed and that the response values are close to their predicted values. would get.

At least 3 confirmation runs should be planned

In an industrial setting, it is very desirable to have a stable process. Therefore, one should run more than one test at the "best" settings. A minimum of 3 runs should be conducted (allowing an estimate of variability at that setting).

If the time between actually running the experiment and conducting the confirmation runs is more than a few hours, the experimenter must be careful to ensure that nothing else has changed since the original data collection.

Carefully duplicate the original environment

The confirmation runs should be conducted in an environment as similar as possible to the original experiment. For example, if the experiment were conducted in the afternoon and the equipment has a warm-up effect, the confirmation runs should be conducted in the afternoon after the equipment has warmed up. Other extraneous factors that may change or affect the results of the confirmation runs are: person/operator on the equipment, temperature, humidity, machine parameters, raw materials, etc.

Checks for when confirmation runs give surprises

What do you do if you don't obtain the results you expected? If the confirmation runs don't produce the results you expected:

1. check to see that nothing has changed since the original data collection
2. verify that you have the correct settings for the confirmation runs
3. revisit the model to verify the "best" settings from the analysis
4. verify that you had the correct predicted value for the confirmation runs.

If you don't find the answer after checking the above 4 items, the model may not predict very well in the region you decided was "best". You still learned from the experiment and you should use the information gained from this experiment to design another follow-up experiment.

Even when the experimental goals are not met, something was learned that can be used in a follow-up experiment

Every well-designed experiment is a success in that you learn something from it. However, every experiment will not necessarily meet the goals established before experimentation. That is why it makes sense to plan to experiment sequentially in order to meet the goals.