Page: 13 |
![]() ![]() |
Cancellation |
Cancellation is probably the most serious type of floating-point problem. However, cancellation is not a type of error per se. |
Cancellation occurs when two numbers with identical leading digits are subtracted. The subtraction leaves behind the difference of the remaining digits, which is stored as a new double precision number. |
Cancellation does not degrade precision if there is no rounding error in the numbers but magnifies rounding error if the numbers are imprecise. |