It is often the case that we have a real number that is not exactly a floating point number: falls between two consecutive FP numbers and .
In order to represent in the computer, we need to approximate it by a FP number. If we choose we say that we rounded down; if we choose we say that we rounded up. We can choose a different FP number also, but this makes little sense, as the approximation error will be larger than with . For example, is in between and . and are successive floating point numbers in our toy system.
We will denote the FP number that approximates . Then
Obviously, when rounding up or down we have to make a certain representation error; we call it the roundoff (rounding) error.
The relative roundoff error, , is defined as
What is the largest error that we can make when rounding (up or
down)? The two FP candidates can be represented as
and
(this is correct since they are successive FP numbers).
For now suppose both numbers are positive (if negative, a similar
reasoning applies).
Since
we have
Now, we need to choose which one of , `better'' approximates . There are two possible approaches.