Dangers in testing statistical hypotheses

Most experiments are intended for the estimation of the size of effects rather than for the testing of a hypothesis of whether or not an effect occurs. Hypothesis testing is often inapplicable, is over-used and is likely to lead to misinterpretations of results. The two types of error possible in hypothesis testing are discussed. Whereas Type I error is usually examined as a matter of course, Type II error is almost always ignored. Investigations in which zero differences are important should recognise the possibility of Type II error in their interpretation. A nonsignificant result should not be interpreted as evidence of a lack of effect. Statistical significance is not synonymous with economic or scientific importance. The importance of choosing the most appropriate design is emphasised and some suggestions are made as to how important sources of error can be avoided.