Ventuz Forum

Hi

i found this precision problem when using a float expression to divide 0.01 by 10
i know that mathematically 0.0009999999 is considered 0.001 but when you use the result in a textnode it is rather annoying to get rid of the error

partyboarder

You can't guarantee something like that in float. Depending on your usecase, you should round your results or limit the decimals.

cheers,
rou

You can use a ToText node to convert one or more floating point values into a more human-friendly form.

This node is controlled by a "Format" string that is a bit cryptic. In your case, the format string "{0:0.0000}" will convert 0.00099999 to 0.0010. The zeros after the ':' are placeholders for digits. the zero before the colon specifies that the Value0 should be used.

A detailed description of how this works can be found at https://msdn.microsoft.com/en-us/librar ... 10%29.aspx and https://msdn.microsoft.com/en-us/librar ... 10%29.aspx.

Hi

Thank you dierk.

I know .... the problem is that it will also add trailing zeros when using an input value of 0.1 .... result 0.0100 ... which is not wanted.

okay ... i will use that and add a script that will remove trailing zeroes ....

partyboarder

Hey,

you should have a look at the C# string formatting documentation - this is not Ventuz specific. You don't need a script! Try:

{0:0.####}

cheers,
rou

that worked .... thx

pb