Proteus tools ยท from probe P-0082

Luck test

A backtest says a rule made money. Would it have made as much on a market with the same moves in a random order? This script shuffles the overnight gaps and the moves inside each day separately, rebuilds the price series, runs the same rule, and counts how often the shuffle does at least as well. A rule that only rides the market's drift scores about the same on shuffles. A real timing edge does not.

What it said about five YouTube strategies

Each dot is one test on SPY daily bars. Green band: p below 0.05. Log scale; 0.001 is the floor at 1,000 shuffles.

0.0010.010.050.11200-day MA filter 1993 onp = 0.0569, profit factor 1.143 v shuffled median 1.103Best MA of 281 lengths 1993 onp = 0.1796, profit factor 1.164 v shuffled median 1.135RSI(5) simple 1993 onp = 0.001, profit factor 1.513 v shuffled median 1.104RSI(5) double 1993 onp = 0.001, profit factor 1.614 v shuffled median 1.101RSI(5) triple 1993 onp = 0.001, profit factor 1.756 v shuffled median 1.102RSI(2) below 30 1993 onp = 0.017, profit factor 1.221 v shuffled median 1.078Best RSI(2) of 50 levels 1993 onp = 0.0319, profit factor 1.417 v shuffled median 1.16RSI(2) below 9 2009 onp = 0.2118, profit factor 1.266 v shuffled median 1.133RSI(2) below 30 2009 onp = 0.1039, profit factor 1.249 v shuffled median 1.133RSI(5) simple 2009 onp = 0.037, profit factor 1.393 v shuffled median 1.15RSI(5) triple 2009 onp = 0.029, profit factor 1.579 v shuffled median 1.158200-day MA filter 2009 onp = 0.3377, profit factor 1.158 v shuffled median 1.145
RulePeriodProfit factorShuffled medianp
200-day MA filter1993 on1.1431.1030.0569fails
Best MA of 281 lengths1993 on1.1641.1350.1796fails
RSI(5) simple1993 on1.5131.1040.001passes
RSI(5) double1993 on1.6141.1010.001passes
RSI(5) triple1993 on1.7561.1020.001passes
RSI(2) below 301993 on1.2211.0780.017passes
Best RSI(2) of 50 levels1993 on1.4171.160.0319passes
RSI(2) below 92009 on1.2661.1330.2118fails
RSI(2) below 302009 on1.2491.1330.1039fails
RSI(5) simple2009 on1.3931.150.037passes
RSI(5) triple2009 on1.5791.1580.029passes
200-day MA filter2009 on1.1581.1450.3377fails

Short-term dip buying inside an uptrend survived. RSI(2), published in 2008, passed before and failed after. The trend filter cut drawdowns but did not beat luck. Full write-up.

Use it

pip install numpy
python3 luck_test.py prices.csv --rule rsi5 --perms 1000
python3 luck_test.py prices.csv --rule sma --n 200 --from 2009-01-01

Repo: github.com/triton-xxix/proteus-luck-test (MIT). luck_test.py needs only numpy. The CSV needs date, open, high, low, close; an adjclose column scales prices to total return. Three rules are built in; add your own as a function returning daily log returns. README.

Method after neurotrader888/mcpt (MIT). No costs or slippage. Not advice. Numbers drawn from results.json by build.py.