Numerical accuracy of script computation

I'm wondering if someone can advise me if I need to be worried about apparently large numerical errors when I replicate a script in excel, where hand computation confirms Excel as accurate and Optuma as not accurate. Attached is the computation in excel and in optuma based on 1 daily bar, 11 dec 2017 for stock code AGL.

Is this error because the numbers (High, Low, Open, Close) displayed by Optuma on the chart are rough? How can such a simple computation result in a 20% error in 1 step?

If I have a BUY threshold on a number with 20% error (or worse, or a bunch of summed numbers, each with 20% error), then I'm buying wrongly in a random way because of cumulative internal truncation/rounding errors of some kind.

I tried playing with the 'format' and 'decimal type' fields in the script editor ... with no observable effect.

I feel I must have missed something. Please advise.


colin hales

26Apr_Book1.xlsx (10.6 KB)

Hi Colin,

Thanks for your post. What has happened is that the low of the day was actually recorded as 25.485. We can see that by pushing the decimal places (by right clicking on the price scale) to 3 decimal places. I added a Bar Label in this image to show it.


When we display the chart, the prices are rounded to 2 decimal places (I did not think a half cent trade was possible on a $25 stock, but it may be some dividend etc). Point is that the rounding pushes the low up to 25.49 but it also pushes the range up to 0.52.

Once you put in the adjustment into your excel for the low, you will see that the error disappears. There is still a small rounding issue in the final computation, but at $0.0005, slippage will be more of a problem when executing.


Hope that helps


Thanks Mathew.