Forensic evidence is often given in terms of a mathematical probability. However, it is easy to make a particular type of error, 'the prosecutor's fallacy', in the interpretation of such probability. This paper highlights the source of this error and briefly introduces Bayes Theorem as the correct theoretical basis for interpreting such evidence. The first Californian legal case in which this error was committed is discussed, followed by all cases in the UK in which this error was highlighted and rectified by the Court of Appeal. It is important for all parties concerned - expert witnesses, lawyers and judges - to understand these principles correctly. Furthermore, the criminal courts face dilemmas when juries are involved. This paper concludes by discussing recent guidance from the Court of Appeal on how such evidence should be presented to the jury and the implications of this guidance.