RAR61 post(s) |
It seems like I am having issues with RoundDecs(value,2) function: it shows say 1.67 as 1.6999999xxx quite few records. How would we get just 2 decimals consistently please? I am using the latest build. Thank you |

That is normally a binary representation issue. (Most decimal fractions can’t be expressed exactly in binary. Only those which can be expressed as a sum of fractions each having denominator equal to a power of 2.) You will need to use a string to get exact representation for all floating-point numbers. I can’t test just now, but I expect it is enough to CAST([field] AS NVARCHAR). You can do this as a new computed field of type NVARCHAR. You may to introduce explicit rounding on the way; hopefully not. |

RAR61 post(s) |
Yes, that cast would solve it but the field will be a nvarchar though. Thank you so much Tim! |

RAR61 post(s) |
Tim, I was wrong: casting is actually rounding it properly in a float64 field. Thank you so much: it solved my problem!!! |

RAR61 post(s) |
Very strange: i was testing it before and it worked. Now it complains about a type mismatch in the real data processing. I don't understand. Thank you again! |

I have only just tested this, and using the value 1.67 (and some others) I don't see a problem. It seems to me that Manifold 9.0.172.x treats even "tricky" floating-point values correctly. (That is, it is applying some cosmetic rounding behind the scenes, to show decimal values as we would expect.) So two questions: what build are you using, and what exact values are you using? Can you supply some sample data so that we can work on this? |

RAR61 post(s) |
Tim, I was trying to create a sample data that you can test but as I mentioned before I am confused as sometimes it does it properly, sometimes not - and that for the same samples. Would it be something in my computer or something happening during the calculation? I truly do not now. Thank you for your help though. |

That is odd! I should have read you more literally before sorry. Maybe if and when you see it again, take a screen picture, so there is at least some context or reminder. We could replicate as much as possible, maybe figure out how to automate some testing. E.g. whether in a query, the result of transform, whether multiple CPU threads or GPGPU are being used, anything is useful. |

RAR61 post(s) |
I definitely will when I see that again; I really want to know why is it happening. Thank you Tim! |

You might want to have a look at this thread, to see if it rings any bells. It might not. |

Manifold User Community Use Agreement
Copyright (C) 2007-2019 Manifold Software Limited. All rights reserved.