Floating point roundoff issues

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view

Floating point roundoff issues

Dear all,

I am experiencing the following strange behavior:

Given a script main.py:
import math
a = 0.999999999999999888977697537484
print("%.30f" % a)
print("%.30f" % math.sqrt(a))

That is, 'a' is the double preceding 1.

Then the output of running 'python main.py' is:

However, after calling pyinstaller to main.py, the resulting main.exe gives the following output:

This seems to be incorrect rounding of sqrt(a).

This is observed using an Anaconda Python distribution on a 32-bit windows system.
I have also already found that on Ubuntu, the 'executable' does give the correct result.

Does anyone have any idea why this is happening, and how it can be fixed? Is this something to do with msvcr...dll versions?

Kind regards,

You received this message because you are subscribed to the Google Groups "PyInstaller" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To post to this group, send email to [hidden email].
Visit this group at https://groups.google.com/group/pyinstaller.
For more options, visit https://groups.google.com/d/optout.