Upload
kendall-stevens
View
36
Download
0
Embed Size (px)
DESCRIPTION
Floating-Point and High-Level Languages. Programming Languages Fall 2003. Floating-Point, the Basics. Floating-point numbers are approximations of real numbers, but they are not real numbers. Typical format in a machine is sign exponent mantissa Exponent determines range available - PowerPoint PPT Presentation
Citation preview
Floating-PointFloating-Pointandand
High-Level Languages High-Level Languages
Programming LanguagesProgramming Languages
Fall 2003Fall 2003
Floating-Point, the BasicsFloating-Point, the Basics
Floating-point numbers are Floating-point numbers are approximations of real numbers, but approximations of real numbers, but they are not real numbers.they are not real numbers.
Typical format in a machine isTypical format in a machine is sign exponent mantissa sign exponent mantissa
Exponent determines range availableExponent determines range availableMantissa determines precisionMantissa determines precisionBase is usually 2 (rarely 16, never 10)Base is usually 2 (rarely 16, never 10)
The Notion of PrecisionThe Notion of Precision
Precision is relative. Large numbers have Precision is relative. Large numbers have less absolute precision than small less absolute precision than small numbersnumbers
For example if we have a 24 bit mantissa, For example if we have a 24 bit mantissa, then relative precision is 1 in 2**24then relative precision is 1 in 2**24
For 1.0, this is an absolute precision ofFor 1.0, this is an absolute precision of1.0*2**(-24).1.0*2**(-24).
For 100, this is an absolute precision ofFor 100, this is an absolute precision of100*2**(-24).100*2**(-24).
Representing NumbersRepresenting Numbers
Some numbers can typically be Some numbers can typically be represented exactly, e,g. 1.0, 2**(-represented exactly, e,g. 1.0, 2**(-13),13),2**(+20) [assume 24 bit mantissa].2**(+20) [assume 24 bit mantissa].
But other numbers are represented But other numbers are represented only approximately or not at allonly approximately or not at all
Problems in RepresentationProblems in Representation
2**(-9999999)2**(-9999999)Too small, underflows to 0.0Too small, underflows to 0.0
2**(+9999999)2**(+9999999)Too large, error or infinityToo large, error or infinity
0.10.1Cannot be represented exactly inCannot be represented exactly inbinary (repeating fracion in binary)binary (repeating fracion in binary)
145678325785.25145678325785.25Representable in binary, but 24-bit Representable in binary, but 24-bit
mantissamantissatoo small to represent exactlytoo small to represent exactly
Floating-Point OperationsFloating-Point Operations
Result may be representable exactlyResult may be representable exactlyr = 81.0;r = 81.0;s = 3.0;s = 3.0;x = r / s;x = r / s;
Machines typically have a floating-Machines typically have a floating-point division instruction point division instruction
But it may not give the exact result But it may not give the exact result
Floating-Point OperationsFloating-Point Operations
Result may not be representable Result may not be representable exactlyexactly
r = 1.0;r = 1.0;s = 10.0;s = 10.0;t = r / s;t = r / s;
Result cannot be precisely corrrectResult cannot be precisely corrrectWill it be rounded to nearest bit, or Will it be rounded to nearest bit, or
perhaps truncated towards zero, or perhaps truncated towards zero, or perhaps even more inaccurate, all perhaps even more inaccurate, all are possible.are possible.
Unexpected ResultsUnexpected Results
Let’s look at this codeLet’s look at this codea = 1.0;a = 1.0;b = 10.0;b = 10.0;c = a / b;c = a / b;if (c == 0.1) printf (“hello1”);if (c == 0.1) printf (“hello1”);if (c == 1.0/10.0) printf if (c == 1.0/10.0) printf
(“goodbye”);(“goodbye”);if (c == a/b) printf (“what the %$!”);if (c == a/b) printf (“what the %$!”);
We may get nothing printed!We may get nothing printed!
Why was Nothing Printed?Why was Nothing Printed?
if (c == 0.1) …if (c == 0.1) … In this case, we have stored the result In this case, we have stored the result
of the run-time computation of 0.1, but of the run-time computation of 0.1, but it’s not quite precise, in c.it’s not quite precise, in c.
The other operand has been converted The other operand has been converted to a constant by the compiler.to a constant by the compiler.
Both are good approximations of 0.1Both are good approximations of 0.1But neither are accurateBut neither are accurateAnd perhaps they are a little bit And perhaps they are a little bit
differentdifferent
Why Was Nothing Printed?Why Was Nothing Printed?
if (c == 1.0 / 10.0) …if (c == 1.0 / 10.0) …The compiler may compute 1.0/10.0 The compiler may compute 1.0/10.0
at compile time and treat it as at compile time and treat it as though it had seen 0.1, and get a though it had seen 0.1, and get a different resultdifferent result
Really ends up being the same as Really ends up being the same as last caselast case
Why Was Nothing Printed?Why Was Nothing Printed?
if (c == a/b)if (c == a/b)Now surely we should get the same Now surely we should get the same
computation.computation.Maybe not, compiler may be clever Maybe not, compiler may be clever
enough to know that a/b is 0.1 in one enough to know that a/b is 0.1 in one case and not in the other.case and not in the other.
Now Let’s Get Something Now Let’s Get Something Printed!Printed!
Read in value of a at run timeRead in value of a at run timeRead in value of b at run timeRead in value of b at run timeCompiler knows nothingCompiler knows nothingNow we will get some output or else!Now we will get some output or else!
c = a / b;c = a / b;if (c == a/b) printf (“This will if (c == a/b) printf (“This will
print!”);print!”);
Still Nothing Printed!!!Still Nothing Printed!!!
How can that beHow can that beFirst a bit of backgroundFirst a bit of backgroundTypically we have two or more Typically we have two or more
different precisions of floating-point different precisions of floating-point values, with different length mantissasvalues, with different length mantissas
In registers we use only the higher In registers we use only the higher precision form, expanding on a load, precision form, expanding on a load, rounding on a store.rounding on a store.
What Happened?What Happened?
c = a / b;c = a / b;if (c == a/b) printf (“This will print!”);if (c == a/b) printf (“This will print!”);
First compute a/b in high precisionFirst compute a/b in high precisionNow round to fit in low precision c, Now round to fit in low precision c,
loosing significant bitsloosing significant bitsCompute a/b in high precision into a Compute a/b in high precision into a
register, load c into a register, register, load c into a register, expandingexpanding
Comparison does not say equalComparison does not say equal
Surprises in PrecisionSurprises in Precision
Let’s compute x**4Let’s compute x**4Two methods:Two methods:
Result = x*x*x*x;Result = x*x*x*x;Result = (x*x)**2Result = (x*x)**2
Second has only two multiplications, Second has only two multiplications, instead of 3, must be more accurate.instead of 3, must be more accurate.
Nope, first is more accurate!Nope, first is more accurate!
Subtleties of RoundingSubtleties of Rounding
Suppose we insist on floating-point Suppose we insist on floating-point operations being properly rounded.operations being properly rounded.
What does properly rounded mean for What does properly rounded mean for 0.50.5
Typical rule, round up always if half wayTypical rule, round up always if half way Introduces BiasIntroduces BiasSome computations sensitive to this biasSome computations sensitive to this biasComputation of orbit of pluto Computation of orbit of pluto
significantly off because of this problemsignificantly off because of this problem
Moral of this StoryMoral of this Story
Floating-point is full of surprisesFloating-point is full of surprises If you base your expectations on real If you base your expectations on real
arithmetic, you will be surprisedarithmetic, you will be surprisedOn any given machine, floating-point On any given machine, floating-point
operations are well definedoperations are well definedBut may be more or less peculiarBut may be more or less peculiarBut the semantics will differ from But the semantics will differ from
machine to machinemachine to machine
What to do in High Level What to do in High Level LanguagesLanguages
We can punt. We just say that floating-We can punt. We just say that floating-point numbers are some approximation of point numbers are some approximation of real numbers, and that the results of real numbers, and that the results of floating-point operations are some floating-point operations are some approximation of the real results.approximation of the real results.
Nice and simple from a language definition Nice and simple from a language definition point of viewpoint of view
Fortran and C historically did thisFortran and C historically did this Not so simple for a poor application Not so simple for a poor application
programmerprogrammer
Doing a Bit BetterDoing a Bit Better
Parametrize the machine model of Parametrize the machine model of floating-point. What exponent range floating-point. What exponent range does it have, what precision of the does it have, what precision of the mantissa.mantissa.
Define fpt model in terms of these Define fpt model in terms of these parameters.parameters.
Insist on results being accurate where Insist on results being accurate where possible, or one of two end points it not.possible, or one of two end points it not.
This is the approach of Ada This is the approach of Ada
Doing Quite a Bit BetterDoing Quite a Bit Better
What if all machines had exactly the What if all machines had exactly the same floating-point model?same floating-point model?
IEEE floating-point heads in that IEEE floating-point heads in that directiondirection
Precisely defines two floating-point Precisely defines two floating-point formats (32-bit and 64-bit) and formats (32-bit and 64-bit) and precisely defines operations on them.precisely defines operations on them.
More on IEEEMore on IEEE
We could define our language to require We could define our language to require IEEE semantics for floating-point.IEEE semantics for floating-point.
But what if the machine does not But what if the machine does not efficiently implement IEEEefficiently implement IEEE
For example, x86 implements the two For example, x86 implements the two formats, but all registers have an 80-bit formats, but all registers have an 80-bit format, so you get extra precisionformat, so you get extra precision
Which sounds good, but is as we have Which sounds good, but is as we have seem a possible reason for suprising seem a possible reason for suprising behavior.behavior.
IEEE and High Level IEEE and High Level LanguagesLanguages
Java and Python both expect/require Java and Python both expect/require IEEE semantics for arithmetic.IEEE semantics for arithmetic.
Java wants high efficiency, which Java wants high efficiency, which causes a clash if the machine does causes a clash if the machine does not support IEEE in the “right” way. not support IEEE in the “right” way.
Java is potentially inefficient on x86 Java is potentially inefficient on x86 machines. Solution: cheat machines. Solution: cheat
Python requires IEEE too, but does Python requires IEEE too, but does not care so much about efficiency.not care so much about efficiency.