Welcome to the Treehouse Community
Want to collaborate on code errors? Have bugs you need feedback on? Looking for an extra set of eyes on your latest project? Get support with fellow developers, designers, and programmers of all backgrounds and skill levels here with the Treehouse Community! While you're at it, check out some resources Treehouse students have shared here.
Looking to learn something new?
Treehouse offers a seven day free trial for new students. Get access to thousands of hours of content and join thousands of Treehouse students and alumni in the community today.
Start your free trial
rileypaxton
440 PointsThe float precision really bugs me
So, I am still going through the beginner courses, and I watched to video when you were talking about floats, namely when you were talking about this line in python:
>>> 0.1 + 0.1 + 0.1 - 0.3
5.551115123125783e-17
And it returns a number close to 0, but not equal to 0. Why does this happen? Why? Ruby (irb) and Lua return the exact same number. Are these scripting languages flawed at properly handling float operations? Or maybe this has to be due to the fundamental way the CPU instructions handle floats on x86-based systems. I'd like to know how ARM or MIPS systems would handle these kind of things.
When doing a conversion with printf() in C, I get more correct results:
#include <stdio.h>
// test.c
int main()
{
printf("%f", 0.1 + 0.1 + 0.1 - 0.3);
return(0);
}
I compiled with:
gcc test.c
And it gave me 0.00000
So I guess what I'm asking is does that make C or printf a better way to handle floating point numbers? I don't know of another way to print raw number with C but... I still think this kind of behavior should be corrected. I don't like programming languages that allow flaws like this.
1 Answer
Steve Hunter
57,712 PointsHi Riley,
Based on those results, the C result is potentially less close to zero than the Python result, isn't it? It is just presented in a better way.
Python gave you:
0.000000000000000005
or thereabouts, while the worst-case scenario for C is:
0.000004
So, if you output the Python with a formatter, something like
"{10.5f}".format(var)
you'd get an identical output to that which C gave you, I think.
On a similar topic, I found this article which goes some way to explaining, but not fixing, the phenomenon. And here talks through the binary process of floats-handling that cause the (tiny) inaccuracy/discrepancy.
I hope that helps! It probably didn't ... have a great New Year!!
Steve.