10 Reasons why back-testing is pointless

I am always interested in why my endeavours fail, but i usually try to do something about it. Often the knowledge so gained sets me upon a different, hopefully profitable path.

Recently, one of my bots had some issues which made me decide to use "better data" for my backtesting and optimisation. The following is a bullet-point list of the results.

1. You are not using your own broker's live server data:

- It comes from a different broker than the one on which you will forward test, and so

will almost certainly have (even slightly) different prices from that which you would have experienced had you been connected to your broker's platform. Prices differ also between demo and live, and between historic timeframe data (*see footnote*)

2. You are using your own broker's (tick) data, but the copy that you have is not complete and/or accurate:

- This would probably be the most accurate way to test a bot, but relies on you receiving/recording every tick in a timely fashion. This may not be possible due to net latency.

3. You have filtered out duplicate ticks

- Duplicate ticks may have an effect on your bot.

4. You are using a constant spread:

- Spreads are usually only constant when the market is closed.

5. You are using a constant tick value:

- Tick values are sometimes not constant, e.g. when trading a cross pair.

6. Your bot behaves differently in live than in optimisation/testing mode:

- Some platforms provide a way of the software determining if it is "test" mode, e.g. the IsOptimisation() and IsTesting() Metatrader functions. These allow you to legitimately bypass unwanted code (e.g. some console output) but may also cause other side effects.

7. You assume market memory where there is none.

- If the random-walk theory is correct (not my opinion), then technical analysis of any kind is useless.

8. Your bot is over-tuned (curve-fitted) to the price data:

- Some analysts insist a bot must have been back-tested using many years of data, and is only "useful" if it would have been consistently profitable over that period.

9. Your bot is under-tuned (not curve-fitted) to the price data:

- Some analysts believe that regular tuning is necessary to allow the bot to "keep in step" with the changing data distribution.

10. You can't keep your fingers off the finished product:

- OK, so you got 1 to 9 under control, and then put your "holy grail" onto a live account. The bot enters a trade, and it is making money so you just HAVE to close it at a profit, thus ignoring a sound tenet "Let your winners run", and losing 50% of the profit you would have otherwise made (opportunity cost). The bot then enters a trade, and it is losing money, so you "Cut your losing trades short". The second you close your trade, the price action reverses and you lose what would have been a healthy profit. YOU MUPPET!