A test asserts that computed P&L equals zero after buying and selling the same quantity at the same price, and it fails with a value of 1.8e-12. Say why, then give the comparison you would write instead, including the tolerance and why an absolute tolerance alone is wrong.

A test asserts that computed P&L equals zero after buying and selling the same quantity at the same price, and it fails with a value of 1.8e-12. Say why, then give the comparison you would write instead, including the tolerance and why an absolute tolerance alone is wrong.

Approach: Ask which intermediate values are representable in binary and what each operation contributes, then decide what scale a tolerance should be measured against.

Each product and sum rounds to the nearest representable double, so the two legs differ in their last bits and the residue is rounding error near 1e-12; compare with a relative tolerance of about 1e-9 times the notional plus a small absolute floor, rather than testing equality. A double carries 15 to 16 significant decimal digits, so a notional of 1e6 resolves to about 1e-10 and a chain of multiplications and additions accumulates a few units in the last place. An absolute tolerance alone fails at both ends, since 1e-12 is far too tight for a notional of 1e9 and far too loose for a price of 0.0001. The absolute floor covers the case where both values sit near zero and a relative test has no scale to work against. The clean fix for money is to hold quantity in integer units and price in integer ticks so the P&L is exact and equality is meaningful again.

Follow-up: What tolerance would you use to compare two implied volatilities from different solvers, and why is that a different question?

Key concepts: rounding error, relative tolerance, units in the last place, absolute floor.