Home - Cutting Edge / All posts - RoundDecs problem?
 RAR61 post(s) #17-Sep-20 21:09 It seems like I am having issues with RoundDecs(value,2) function: it shows say 1.67 as 1.6999999xxx quite few records.How would we get just 2 decimals consistently please?I am using the latest build.Thank you
 tjhb9,452 post(s)online #17-Sep-20 21:51 That is normally a binary representation issue. (Most decimal fractions can’t be expressed exactly in binary. Only those which can be expressed as a sum of fractions each having denominator equal to a power of 2.)You will need to use a string to get exact representation for all floating-point numbers.I can’t test just now, but I expect it is enough to CAST([field] AS NVARCHAR). You can do this as a new computed field of type NVARCHAR.You may to introduce explicit rounding on the way; hopefully not.
 RAR61 post(s) #17-Sep-20 21:59 Yes, that cast would solve it but the field will be a nvarchar though.Thank you so much Tim!
 RAR61 post(s) #17-Sep-20 22:09 Tim,I was wrong: casting is actually rounding it properly in a float64 field.Thank you so much: it solved my problem!!!
 RAR61 post(s) #17-Sep-20 22:17 Very strange: i was testing it before and it worked. Now it complains about a type mismatch in the real data processing.I don't understand.Thank you again!
 tjhb9,452 post(s)online #21-Sep-20 22:21 I have only just tested this, and using the value 1.67 (and some others) I don't see a problem.It seems to me that Manifold 9.0.172.x treats even "tricky" floating-point values correctly. (That is, it is applying some cosmetic rounding behind the scenes, to show decimal values as we would expect.)So two questions: what build are you using, and what exact values are you using?Can you supply some sample data so that we can work on this?
 RAR61 post(s) #22-Sep-20 00:39 Tim,I was trying to create a sample data that you can test but as I mentioned before I am confused as sometimes it does it properly, sometimes not - and that for the same samples. Would it be something in my computer or something happening during the calculation? I truly do not now.Thank you for your help though.
 tjhb9,452 post(s)online #22-Sep-20 01:11 That is odd! I should have read you more literally before sorry.Maybe if and when you see it again, take a screen picture, so there is at least some context or reminder. We could replicate as much as possible, maybe figure out how to automate some testing.E.g. whether in a query, the result of transform, whether multiple CPU threads or GPGPU are being used, anything is useful.
 RAR61 post(s) #22-Sep-20 01:38 I definitely will when I see that again; I really want to know why is it happening.Thank you Tim!
 tjhb9,452 post(s)online #22-Sep-20 03:13 You might want to have a look at this thread, to see if it rings any bells. It might not.