wrong computation result when dividing two integers in C/C++

[Patrice Terrier]

I know C/C++ is doing optimization based on the native type of numbers, but there is something i couldn't find how to let it work like in PB.

Example:
UINT u1 = 71, u2 = 10;
float result = u1 / u2 (the result here is 7, because both u1 and u2 are unsigned integers)

now using
float result = (float) u1 / u2 (the result becomes 7.09999, because u1 has been casted first to float)

when i would like to get 7.1 just like in PB.

if i use
float result = 71.0f / 10.0f then the result is good: 7.1

What should i do to solve this problem?

[Anand Kumar]

This should do the trick...

unsigned int a = 71, b = 10;
float c = 0;
c = (float) a / (float) b;

[John Spikowski]

ScriptBasic is written in C and seems to return the correct results.

Code Select Expand


PRINT 71/10,"\n"


jrs@laptop:~/sb/sb22/test$ scriba divint.sb
7.100000e+00
jrs@laptop:~/sb/sb22/test$

[Patrice Terrier]

John--

I am speaking of VS2010 C/C++, i am not concerned with any other exotic.

Anand --

Your syntax does exactly the same that what i am using, and the result is 7.099999 not 7.1 (like with PB).

To convert the float into a string to show it on screen, i am forced to use
swprintf(ws, sizeof(ws), L"%3f", floatnumber)
instead of using
_ltow_s(floatnumber, ws, 10)

in PB i can write STR$(U1 / U2) and display correctly the result 7.1, without extra brain storming.

I did try using:
float result = (u1 / (float)u2) + 0.000001f;
but that is just a kind of joke, and couldn't be used in real programming.

Try this: c = (float) (u1 / (float) (U2*100)) *100;
and you should get 7.1000004, that is more close to what i want.

That is typical of rounding error.

[Charles Pegge]

Hi Patrice,

I tested this expression in OxygenBasic and saw the same rounding error that you saw.

Even when the variables are upgraded to double, there is still a rounding error. However when using Extended (80 bit) floats the answer resolved to 7.1.

One possible solution is to hide the rounding error by displaying formatted numbers with a restricted number of decimal places

[Patrice Terrier]

Yes, this is what i am doing with the STR$ function below, but that is the kind of thing hard to guess when you move from one language to another 

Code Select Expand

wstring STRF$(IN float N) {
    WCHAR ws[33] = {0};
    swprintf(ws, sizeof(ws), L"%3f", N);
    return (WCHAR*) ws;
}

By the way i shall convert my zTrace utility to let it work in 64-bit, to help me in further debugging of 64-bit code.

[Charles Pegge]

Where precision matters: does VC10 support long double?

[Patrice Terrier]

Charles--

see MSDN : http://msdn.microsoft.com/en-us/library/9cx8xs15.aspx

[Anand Kumar]

Patrice,

When I try this,

   unsigned int a = 71, b = 10;
   float c = 0;
   c = (float) a / (float) b;
   if (c == 7.1f) printf ("yes %f", c); else printf ("no %f", c);
   getch ();

My results are  (both in release as well as in debug mode)

yes 7.100000

PS: I am using VS 2008...

[Patrice Terrier]

Yes it seems that it does work as soon as one of the UINT member is casted to float, like this
wprintf(L"%f", a / (float) b);

note: better to use wprintf Unicode nowdays, rather than old Ansi printf. 

[John Spikowski]

I am speaking of VS2010 C/C++, i am not concerned with any other exotic.

ScriptBasic 2.2 for Windows is compiled with Microsoft VC10 from the latest Windows SDK. I don't understand what you mean by exotic.

My previous post used gcc under Ubuntu 64 bit. I assumed since ScriptBasic only has one source tree that the results would be the same.

Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.

C:\Documents and Settings\John>cd \sb22\test

C:\SB22\test>scriba divint.sb
7.100000e+000

C:\SB22\test>scriba -v
ScriptBasic v2.2
Variation >>Windows32<< build 2
Magic value 859012665
Node size is 16
Extension interface version is 11
Compilation: Nov  9 2012 18:29:05
Executable: C:\SB22\bin\scriba.exe

C:\SB22\test>