Overflow

Occurs when the result of an arithmetic operation is finite, but larger in magnitude than the largest FP number representable using the given precision. The standard IEEE response is to set the result to (round to nearest) or to the largest representable FP number (round toward 0). Some compilers will trap the overflow and abort execution with an error message.

Example (Demmel 1984, from D. Goldberg, p. 187, adapted): In our toy FP system let's compute

whose result is , a "normal" FP number. A direct use of the formula

returns the result equal to 0, since the denominators overflow. Using the scaled formulation

we have , and .

Sometimes overflow and infinity arithmetic mat lead to curious results. For example, let and compute

Since the denominator overflows it is set to infinity; the numerator does not overflow, therefore the result is !. If we compute the same quantity as

we obtain a result closer to the mathematical value.