Level of significance in Machine Learning

The probability level below which we reject the hypothesis is known as the level of significance. The region in which a sample value falling is rejected is known as the critical region. We generally take two critical regions which cover 5% and 1% areas of the normal curve. The shaded portion in the figure corresponds to 5% level of significance. Hence the probability of the value of the variate falling in the critical region is the level of significance.

Level of significance

Depending on the nature of the problem, we use a single-tail test or double-tail test to estimate the significance of a result. In a double-tail test, the area of both the tails of the curve representing the sampling distribution are taken into account whereas on the single tail test, only the area on the right of an ordinate are taken into consideration. For instance, to test whether a coin is biased or not, double-tail test should be used, since a biased coin gives either more number of heads than tails ( which corresponds to right tail), or more number of tails than heads (which corresponds to left tail only).