python - Decimal module: rounding -
i have trouble getting desired values.
the following 'abc' strings represent angles. 'result' strings represents values i'm looking for.
a = '199.1224' result = '199.122400000000000' b = '199.0362' result = '199.036200000000010' c = '-199.9591' result = '-199.959100000000010'
i use following code:
decimal(float(a)).quantize(decimal('.000000000000001'), rounding=round_half_up) results in = 199.122399999999999 b = 199.036200000000008 c = -199.959100000000007
i'm no math genius, , tried possible in decimal module, can't seem find right way result need.
as crappy workaround use:
decimal(float(a)).quantize(decimal('.00000000000001'), rounding=round_half_up) str(a)+'0'
so quantize 1 less decimal, convert string (have anyway) , add zero. gets me correct desired results thousands of these values. want find out if there correct way round , end 0 (don't know english words this).
import decimal decimal import decimal b = '199.0362' result = decimal(float(b)).quantize(decimal('0e-14'), rounding=decimal.round_half_up) print format(result, '.15f') # a: '199.1224' -> '199.122400000000000' # b: '199.0362' -> '199.036200000000010' # c: '-199.9591' -> '-199.959100000000010'
- if use
'0e-14'
instead of'0.00000000000001'
inquantize
, don't have count digits check it's correct. - use
format
function convert string specific number of decimal places.
Comments
Post a Comment