python - Decimal module: rounding -


i have trouble getting desired values.

the following 'abc' strings represent angles. 'result' strings represents values i'm looking for.

a = '199.1224' result = '199.122400000000000'  b = '199.0362' result = '199.036200000000010'  c = '-199.9591' result = '-199.959100000000010' 

i use following code:

decimal(float(a)).quantize(decimal('.000000000000001'), rounding=round_half_up)  results in  = 199.122399999999999 b = 199.036200000000008 c = -199.959100000000007 

i'm no math genius, , tried possible in decimal module, can't seem find right way result need.

as crappy workaround use:

decimal(float(a)).quantize(decimal('.00000000000001'), rounding=round_half_up) str(a)+'0' 

so quantize 1 less decimal, convert string (have anyway) , add zero. gets me correct desired results thousands of these values. want find out if there correct way round , end 0 (don't know english words this).

import decimal decimal import decimal  b = '199.0362' result = decimal(float(b)).quantize(decimal('0e-14'), rounding=decimal.round_half_up) print format(result, '.15f')  # a:  '199.1224' ->  '199.122400000000000' # b:  '199.0362' ->  '199.036200000000010' # c: '-199.9591' -> '-199.959100000000010' 
  • if use '0e-14' instead of '0.00000000000001' in quantize, don't have count digits check it's correct.
  • use format function convert string specific number of decimal places.

Comments

Popular posts from this blog

java - activate/deactivate sonar maven plugin by profile? -

python - TypeError: can only concatenate tuple (not "float") to tuple -

java - What is the difference between String. and String.this. ? -