jgoizueta / flt Goto Github PK

I have been picking at this issue on and off for weeks, but hadn't been able to make any progress on it until now. I have a bunch of RSpec specs that fail only when I'm using Spork, all with the same error message for the same line of code:

Flt::DecNum can't be coerced into BigDecimal

I followed the instructions in your README just now and added a file to ./config/initializers/flt.rb with these conversion functions in it:

DecNum.context.define_conversion_from(BigDecimal) do |x, context|
  DecNum(x.to_s)
end

DecNum.context.define_conversion_to(BigDecimal) do |x|
  BigDecimal.new(x.to_s)
end

I still get the error though. The mysterious thing to me is that even before I wrote that initializer, the specs all pass without Spork. It's not like I need to use Spork (obviously I've been okay without it for several weeks), but it is pretty annoying having such an arbitrary bug. Can you lend any insight into what's happening here, why it works outside of Spork, and what I can do to fix it?

After regenerating the trigonometry test data, one case is revealed where the result is not correctly rounded (trigonometry tests aspire to achieve same accuracy as HP classic 12-digit calculators).

  1) Failure:
TestTrig#test_cos [/Users/jgoizueta/dev/repos/flt/test/test_trig.rb:42]:
cos(10000000.6817)==-0.969490904178
input: [100000006817, -4] [10 rad 12].
Expected: DecNum('-0.969490904178')
  Actual: DecNum('-0.969490904177')

The problem goes away raising the internal extra precision to 4 digits:

diff --git a/lib/flt/trigonometry.rb b/lib/flt/trigonometry.rb
index 5ecea64..17afbd4 100644
--- a/lib/flt/trigonometry.rb
+++ b/lib/flt/trigonometry.rb
@@ -134,7 +134,7 @@ module Flt
       rev_sign = false
       s = nil
       num_class.context(self) do |local_context|
-        local_context.precision += 3 # extra digits for intermediate steps
+        local_context.precision += 4 # extra digits for intermediate steps
         x,k,pi_2 = local_context.reduce_angle2(x,2)
         rev_sign = true if k>1
         if k % 2 == 0
@@ -161,7 +161,7 @@ module Flt
       sign = x.sign
       s = nil
       num_class.context(self) do |local_context|
-        local_context.precision += 3 # extra digits for intermediate steps
+        local_context.precision += 4 # extra digits for intermediate steps
         x = x.copy_sign(+1) if sign<0
         x,k,pi_2 = local_context.reduce_angle2(x,2)
         sign = -sign if k>1

But It'd be nice to fix this by using better algorithms rather than increasing the brute force...