Overcoming rounding errors

Floating point variables are prone to rounding errors.  (That is, the Number type in ActionScript)  Often these don’t matter.  But in some applications, especially those that process financial information, it is important to avoid errors.  Especially errors that propagate.  I am interested in eliminating rounding errors because of my spreadsheet AIR application.  e2spreadsheet.  The current beta version doesn’t do anything about these errors, but I want to remedy that in the next version.

For example, if you open up e2spreadsheet, and type in the following formula: =74.725*100 what answer do you expect?  7472.5 right?  Well, the answer that you actually get is 74.499999999999

Because the binary representation has a finite length, it cannot accurately represent some decimal values.

So, how do we get around this?  Well, my first thought was a number representation called Binary Coded Decimal (BCD).  I think Cobol used to use this representation.  But that would have involved many program operations to simulate each BCD calculation.  Then this blog by Josh Tynjala gave me another idea.  (Ok, probably not a new idea – but it was a revelation to me).

What if I represent my numbers using a radix-10 (decimal) mantissa and exponent.

Actually floating point numbers are represented internally in binary mantissa-exponent form. So that…

value=mantissa*Math.pow(2,exponent);

In other words, the mantissa is shifted left or right depending on the value of the exponent.

Rounding errors occur because finite fractional binary numbers don’t translate well into decimal numbers.  So the solution is to simulate this mantissa-exponent form in decimal (radix-10).  We would write a class so that value=mantissa*Math.pow(10,exponent);  And this class would contain all the methods necessary to do arithmetic operations on this representation.  Add, Subtract, Multiply, etc..

So, with that idea in my head, I wrote the following code. (previous blog entry which appears below).  Don’t rely on this yet.  I wrote and tested it quickly, and I wanted to see what other developers think about the idea.  Is this a good way to eliminate rounding errors?  Or is there a better way I haven’t considered?

Overcoming rounding errors (code)

package { // Decimal.as  by Daniel Freeman http://www.e2easy.co.cc

	public class Decimal {

		private var mantissa:int;
		private var exponent:int;
		private var fixed:Boolean=false;

		public function Decimal(mantissa:int=0,exponent:int=0) {
			this.mantissa=mantissa;
			this.exponent=exponent;
		}

		public function convert(float:Number):void {
			var strng:String=float.toString();
			var pnt:int=strng.indexOf('.');

			if (pnt<0) {
				mantissa=float;
				exponent=0;
				normalise();
			} else {
				exponent=pnt-strng.length+1;
				mantissa=Math.round(float*Math.pow(10,-exponent));
			}
			fixed=false;
		}

		public function converttoprecision(float:Number,exponent:int):void {
			this.exponent=exponent
			mantissa=Math.round(float*Math.pow(10,-exponent));
			fixed=true;
		}

		private function normalise():void {
			if (!fixed && mantissa!=0) {
				while (mantissa % 10 == 0) {
				mantissa=mantissa/10;
				exponent++;
				}
			}
		}

		public function add(num:Decimal,minus:Boolean=false):Decimal {
			var ret:Decimal=new Decimal();
			var m0:int=mantissa;
			var m1:int=num.mantissa;
			var sum:int;

			if (exponent<num.exponent) {
				ret.exponent=exponent;
				m1=m1*Math.pow(10,num.exponent-exponent);
			} else if (exponent>num.exponent) {
				ret.exponent=num.exponent;
				m0=m0*Math.pow(10,exponent-num.exponent);
			} else ret.exponent=exponent;
			ret.mantissa=minus ? m0-m1 : m0+m1;
			ret.fixed=fixed && num.fixed;
			ret.normalise();
			return ret;
		}

		public function subtract(num:Decimal):Decimal {
			return add(num,true);
		}		

		public function multiply(num:Decimal):Decimal {
			var ret:Decimal=new Decimal(mantissa*num.mantissa,exponent+num.exponent);

			ret.fixed=fixed && num.fixed;
			ret.normalise();
			return ret;
		}

		public function toString():String {
			var ret:String;
			var sign=mantissa<0;
			var pos:int,i:int;

			if (sign) mantissa=-mantissa;
			ret=mantissa.toString();
			if (exponent<0) {
				pos=ret.length+exponent;
				if (pos<0) {
					for (i=0;i<-pos;i++) ret='0'+ret;
					ret='0.'+ret;
				} else {
					ret=ret.substring(0,pos)+'.'+ret.substring(pos,ret.length);
					if (pos==0) ret='0'+ret;
				}
			} else if (exponent>0) {
				for (i=0;i<exponent;i++) ret+='0';
			}
			if (sign) {ret='-'+ret;mantissa=-mantissa}
			return ret;
		}

	}
}