anz-bank / decimal Goto Github PK
View Code? Open in Web Editor NEWFixed-precision decimal numbers based on IEEE 754 decimal64
License: Apache License 2.0
Fixed-precision decimal numbers based on IEEE 754 decimal64
License: Apache License 2.0
This should return a string with a sign and the type of the decimal number
eg Class(0) --> +0
Class(-420) --> +Normal
Also consider adding a zero flavour in the constants flZero
Rounding is done in different places and users slightly different logic based on which arithmetic function.
It should be cleaned up and generalised to be used in one function
Implement
(Decimal64).IsZero64() bool
E.g.:
It'll make rounding a lot simpler and straight forward
Let's clean this up.
64
.Thus:
const (
decDigits64 = 16
expBits64 = 10
expRange64 = 3 << (expBits - 2)
symmMaxExp64 = expRange64 / 2
symmMinExp64 = 1 - symmMaxExp64
maxExp64 = symmMaxExp64 - (decDigits64 - 1)
minExp64 = symmMinExp64 - (decDigits64 - 1)
)
We end up with symm(Max|Min)Exp64
= 384|-383 and (max|min)Exp64
= 369|-398.
The decimal128 group can then be written in exactly the same form, with the only changes being to the first two constants:
decDigits128 = 34
expBits128 = 14
It could even be extended to decimal32, in the event that we ever wanted to implement it:
decDigits32 = 7
expBits32 = 8
Technically, we could even derive decDigits(32|64|128)
:
decDigits64 = (64 - expBits64) * 3 / 10
64 - expBits64
is the number of bits of precision available for the significand and 3/10 is close enough to log(2)/log(10) to convert bits to digits for the given values of expBits(32|64|128)
. But I think this a bit too abstruse for the reader.
I just noticed that the standard math/bits package has some support for 128-bit operations. It might be worth using these in place of the equivalent uint128_t operations to reduce LoC.
Use prec
to control the number of digits present in the output.
32 bit go compilers (such as go play) result in an integer overflow when using this library with constants such as decimal64Base.
This needs to be changed to be a uint64 or int64 for consistency across compilers.
ieee 754 requires the following error types to be implemented:
invalidOperation
divisionByzero
inexact
overflow
underflow```
These should be returned as an error type in the context struct after an operation
current dec tests don't change the sign on negative NaNs, eg
abs(-NaN) == -NaN
Logic is that this is good in order for all operations on NaNs keep as mush information for diagnostic info as possible
Exponent overflows when adding very large numbers. eg 9e384 + 1e384.
Should add to inf
x := MustParseDecimal64("9e384")
y := MustParseDecimal64("1e384")
z := x.Add(y)
z evaluates to .007199254740992e-383
instead of inf
Having to suffix every name with 64 is annoying. Should we have a decimal64 package (and therefore decimal128) instead?
The counterargument is that most users will have to use the suffix regardless, whether it's on the typename or the package name, so it's really only a headache for the maintainers.
the IEEE 754-r specifies that:
"Four mutually exclusive relations are possible: less than, equal, greater than, and unordered. The last case
arises when at least one operand is NaN. Every NaN shall compare unordered with everything, including
itself"
which means that Nan != Nan
although there isn't a way to set equality behaviour in golang (that i know of), but it's got to do with the bits that make up the Nan
Currently there is only a uint128/uint64 and another function is needed going forward
This is required for approval by the ANZ open source committee to use this library in ANZ systems.
First step is to define what constitutes an acceptable audit.
When subtracting numbers 10000000000000000 + -77
rounding errors occur with the rescaling of 77 as the magnitude of the number decreases by a power of 10 then the truncated 7
would have resolution in the answer, but the number is already truncated.
current error is:
expected value: 0.9999999999999923
calculated value: 0.9999999999999931
There are several loops in the code that might be removable or at least improvable. Investigate and possibly rework to improve performance.
SNan panics in functions such as signalNaN64
aren't needed due to the fact that the calling function can check if a number is a signalling Nan
Code becomes a lot simpler once panics are removed and Snan (or original weighted Nans) are returned
It's basically untested, other than to verify a single round trip.
A key concern is accurate numeric outcomes for all arithmetic operations, including parsing and formatting. The last thing we want is some variant of 0.1 + 0.1 + 0.1 ≠ 0.3.
Some thought needs to go into the completeness of the tests such that all conceivable edge cases are covered. The following is a reasonably thorough but incomplete set of cases to consider.
Currently, rounding status is one of eq0
, lt5
, eq5
and gt5
. Tests then look for different combinations of these. For instance, round()
has the following test:
if context.rndStatus == gt5 || context.rndStatus == eq5 {
This could be made more efficient if the constants were intialised with 1 << iota
. You could then test:
if context.rndStatus & (gt5 | eq5) != 0 {
As described in https://en.wikipedia.org/wiki/Decimal32_floating-point_format. The value of this is likely marginal, but might be useful in some scenarios:
decimal64
.uint64
suffices for internal operations, it will likely be much faster than decimal64
.thanks for this package, does it possible to add database/sql support to allow to Scan to decimal value and store it inside db?
Currently numbers are "normalised" so that there are trailing zeros,
for example 1
is stored as a significand 100000000000000
with an exponent of -15
Generally outputs are normalised from ParseDecimal and arithmetic functions.
Some problems are:
remove trailing zeros
function to avoid integer overflowSooner or later rounding attributes are going to be implemented by parsing some sort of argument
proposal to change function syntax from d.Add(e)
to context.Add(d, e)
where context is the rounding attribute.
Even with this implemented the standard d.Add(e)
could still be implemented with half up rounding, for example
Go has a math.MaxFloat64
but no math.NegativeFloat64
because it's trivial to write -math.MaxFloat64
. Should the decimal library similarly avoid defining negative versions of a bunch of constants?
The main counterargument is that decimal.Max64().Neg()
isn't quite as neat and it requires a computation on each use, whereas defining as a constant means it can be computed once at program initialisation time.
As described in https://en.wikipedia.org/wiki/Decimal128_floating-point_format. The existing decimal64
type suffices for most purposes, but some applications might require the greater precision of decimal128
.
Size gets the magnitude of the significand,
currently using a for loop method that divides till it reaches zero.
@anzdaddy speculates that it might be possible to get the same result using bitwise inspection as used in the renormalise function, although this method currently doens't work for all decimal numbers
numBits := 64 - bits.LeadingZeros64(significand)
numDigits := numBits * 3 / 10
normExp := 15 - numDigits
Might make some operations quicker and allow 1-1 mapping to Class function as well as generally simplifying comparison logic
roundContext bundles a rounding mode, which is stateless (set at construction and never changed) and a rounding status, which is stateful (meaningless at construction and only set later). These don't belong together.
Recommendation:
roundContext
type.roundingMode
.roundingStatus
parameter to those methods that require it.Am I allowed to edit the readme yet?
For example, (Decimal64).Format
allocates a buffer to pass to (Decimal64).Append
but ends up passing it to (fmt.State).Write
, which might be simply copying the buffer into another already allocated buffer.
The solution might be to reverse the roles. Format
implements the algorithm and Append
calls it like so:
type appender struct {
buf []byte
prec int
}
func (a *appender) Write(b []byte) (n int, err error) {
a.buf = append(a.buf, b...)
return n, nil
}
func (a *appender) Width() (wid int, ok bool) {
return 0, false
}
func (a *appender) Precision() (prec int, ok bool) {
return a.prec, true
}
func (a *appender) Flag(c int) bool {
return false
}
// Append appends the text representation of d to buf.
func (d Decimal64) Append(buf []byte, format byte, prec int) []byte {
a := appender{buf, prec}
d.Format(&a, rune(format))
return a.buf
}
// Format implements fmt.Formatter.
func (d Decimal64) Format(s fmt.State, format rune) {
// Declare big enough local array to avoid dynamic allocation.
var data [25]byte
// Use the same algo as Append currently does
buf := data[:]
prec, havePrec := s.Precision()
⋮
if _, err := s.Write(buf); err != nil {
panic(err)
}
}
To reiterate, the above is just one case. There might be others. Also, it's possible the above solution yields worse performance, so don't assume, profile.
I created https://github.com/anz-bank/decimal-reference. You can invoke it from the command line (supports cmdline args and stdin input modes) driving arbitrary operations using an RPN notation. The major caveat is that I based it on libdfp, which only wants to compile on Linux, so you need docker to run it.
It still needs to be hooked in for testing purposes, and if we can replace libdfp with a macOS-friendly library to avoid the Docker dependency, that would be awesome (though then we'd have to ensure some kind of build environment, probably Xcode, which has its own challenges).
Either way, we should stick to the cmdline/stdin approach to keep the moving parts fully decoupled.
Currently doesn't pass all test cases
golang/go#19787 proposes adding decimal types to the language. As of this writing, it is tagged Proposal and NeedsDecision.
The proposal process is documented at https://github.com/golang/proposal. It would probably help to have a proof-of-concept implemented using this library, which might also end up in the Go code base in some form.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.