windows calculator 152362126.30733179848
ubuntu calculator 152362126.307331798
-------------------------- as float----------------------------------------------
Code: Select all
f32 xyz = 123456789.0f;
f32 xcv = 1.23413323432;
f32 ccc = xcv * xyz;
Code: Select all
xyz 1.2345679e+008 float
xcv 1.2341332 float
ccc 1.5236213e+008 float
diff: -3.69266820152
-------------------------- as mixed ----------------------------------------------
Code: Select all
f32 xyz = 123456789.0f;
f32 xcv = 1.23413323432;
f64 ccc = xcv * xyz;
Code: Select all
xyz 1.2345679e+008 float
xcv 1.2341332 float
ccc 152362131.15056992 double
diff: -4.84323812152
-------------------------- as double ----------------------------------------------
Code: Select all
f64 xyz = 123456789.0f;
f64 xcv = 1.23413323432;
f64 ccc = xcv * xyz;
Code: Select all
xyz 123456792.00000000 double
xcv 1.2341332343200000 double
ccc 152362130.00973150 double
diff: -3.70239970152
ok, looking at it, I would have expected float and mixed to mess up, but I'm little surprised the double result is so far off myself,.
BTW: I have no idea about visual basic...
^bump for the double result, that really makes me wonder