Dontcha just hate it when you spend the weekend back-testing your latest software, only to get disappointing results when you forward test the next week. Where did I go wrong? How can this be happening?
One often mooted reason is the data used for the back-test. If the data is not representative of the live environment, our test results are not likely to be reflecting anything near reality.
Of course, there is the argument that back-testing is intrinsically useless because the future does not reflect the past. Well, a brazillion (large number) folks would disagree. I include among those any trader that uses a support or resistance point, Fibonacci projection or similar in their strategy. What else do these demonstrate but market memory? I digress ...
The list of problems with data includes:
Difference between demo and live: Even if you recorded tick data yourself from a demo server, it may not reflect what is happening on the same broker's live server. Big and frequent differences point to the need to look for a new broker.
Data not reflecting "market" liquidity provider sentiment: Foreign Exchange is a huge market. Retail brokers only have a (usually small) slice. Their quotes will reflect that small share, plus their own spreads to reflect their trading position (after all the majority are actively trading against you). I don't want to reiterate the list of broker "dirty tricks" in this article. Dump a months worth of M1 data, come back a month later and look at the historic data currently shown. You will often find peaks/troughs disappear. These are artificially jacked quotes designed to knockout trade stop losses.
Data gaps: Wow! look at that hugely profitable renko trade! Unfortunately the untradeable stream of bricks produced by my chart generator were entirely due to the gaps in the data I used.
Data granularity: Using 1 minute closes to test a scalping system may not be the optimal method, especially if your system artificially generates ticks based on the Open, High, Low and Close of that bar. Metatrader, for example, uses a technique called fractal interpolation to generate fake ticks from the summary bar data.
Data obtained from a different broker: It goes without saying that testing with data from a different broker has its problems, results depending entirely on the correlation between them and prices from the target broker.
Applications requiring volume as a key parameter: Some providers dont provide volume at all with their data (e.g. Gain Capital). Some provide bid and ask volume separately (e.g. Dukascopy). It is questionable in many cases whether the volume numbers are representative of anything.
It's all in the percentages
Bwa ha ha ha! Sorry, let me pick myself up from the floor and dust myself off. There is sooooo much BS talked about 99% vs 25% tick data that I have to expose this right away. I refer to the test summary report MT4 produces when doing back-testing. If you use fractal interpolation, the percentage will be low, like 20 to 25%, right? But if you have tested with good quality data, the percentage will be high, like 99%, right? WRONG! The 25% figure is a number that the strategy tester in MT4 writes into the tick data file. The 99% figure is a number that the PROGRAM that generated the data writes into the tick data file. Simple as that. In version 406 of MT4, it is a double value written to bytes 232 to 239 of the file. Any reprobate could fake the "quality" of data used in their testing using a very simple program.
Warning note: MT4 tick data DOES change sometimes from build to build. Please ensure you are using tick data generated for your build, especially if you purchase it.
Solution
There are several solutions to the issues mentioned above, and whichever you take will depend on the type of testing you want to do, whether you are scalping or working on longer time-frames, using time-based or price-based charts.
Some very nice scripts are freely available for downloading, and given time, one can become very proficient at using them. One problem for me is that they usually run very slowly, very frustrating for me if I want to generate a tick data set for several currencies and time-frames on the fly. Being time-conscious, I had to bite the bullet and spend some time, firstly deciding what approach suited me, and then implementing a system. The beauty of porting to a more modern environment such as VB/Java/C++ allows me, as a programmer, to revisit all of the programmer decisions that must have been made multiple times by the various implementers. e.g. Once you move out of MQL4, you need to implement your own StrToTime function, which is fun! As I mentioned previously, quick access to representative source data is important to me, so using Dukascopy with its wacky interface is largely out of the question. My choice of data provider allows me to download in bulk, on the fly, in a matter of minutes. My new system allows me to quickly convert this into a tick data file with 99% quality (or is it 100%?), (big smile .....). BTW, did I mention it can also generate renko tick data?
One often mooted reason is the data used for the back-test. If the data is not representative of the live environment, our test results are not likely to be reflecting anything near reality.
Of course, there is the argument that back-testing is intrinsically useless because the future does not reflect the past. Well, a brazillion (large number) folks would disagree. I include among those any trader that uses a support or resistance point, Fibonacci projection or similar in their strategy. What else do these demonstrate but market memory? I digress ...
The list of problems with data includes:
Difference between demo and live: Even if you recorded tick data yourself from a demo server, it may not reflect what is happening on the same broker's live server. Big and frequent differences point to the need to look for a new broker.
Data not reflecting "market" liquidity provider sentiment: Foreign Exchange is a huge market. Retail brokers only have a (usually small) slice. Their quotes will reflect that small share, plus their own spreads to reflect their trading position (after all the majority are actively trading against you). I don't want to reiterate the list of broker "dirty tricks" in this article. Dump a months worth of M1 data, come back a month later and look at the historic data currently shown. You will often find peaks/troughs disappear. These are artificially jacked quotes designed to knockout trade stop losses.
Data gaps: Wow! look at that hugely profitable renko trade! Unfortunately the untradeable stream of bricks produced by my chart generator were entirely due to the gaps in the data I used.
Data granularity: Using 1 minute closes to test a scalping system may not be the optimal method, especially if your system artificially generates ticks based on the Open, High, Low and Close of that bar. Metatrader, for example, uses a technique called fractal interpolation to generate fake ticks from the summary bar data.
Data obtained from a different broker: It goes without saying that testing with data from a different broker has its problems, results depending entirely on the correlation between them and prices from the target broker.
Applications requiring volume as a key parameter: Some providers dont provide volume at all with their data (e.g. Gain Capital). Some provide bid and ask volume separately (e.g. Dukascopy). It is questionable in many cases whether the volume numbers are representative of anything.
It's all in the percentages
Bwa ha ha ha! Sorry, let me pick myself up from the floor and dust myself off. There is sooooo much BS talked about 99% vs 25% tick data that I have to expose this right away. I refer to the test summary report MT4 produces when doing back-testing. If you use fractal interpolation, the percentage will be low, like 20 to 25%, right? But if you have tested with good quality data, the percentage will be high, like 99%, right? WRONG! The 25% figure is a number that the strategy tester in MT4 writes into the tick data file. The 99% figure is a number that the PROGRAM that generated the data writes into the tick data file. Simple as that. In version 406 of MT4, it is a double value written to bytes 232 to 239 of the file. Any reprobate could fake the "quality" of data used in their testing using a very simple program.
Warning note: MT4 tick data DOES change sometimes from build to build. Please ensure you are using tick data generated for your build, especially if you purchase it.
Solution
There are several solutions to the issues mentioned above, and whichever you take will depend on the type of testing you want to do, whether you are scalping or working on longer time-frames, using time-based or price-based charts.
Some very nice scripts are freely available for downloading, and given time, one can become very proficient at using them. One problem for me is that they usually run very slowly, very frustrating for me if I want to generate a tick data set for several currencies and time-frames on the fly. Being time-conscious, I had to bite the bullet and spend some time, firstly deciding what approach suited me, and then implementing a system. The beauty of porting to a more modern environment such as VB/Java/C++ allows me, as a programmer, to revisit all of the programmer decisions that must have been made multiple times by the various implementers. e.g. Once you move out of MQL4, you need to implement your own StrToTime function, which is fun! As I mentioned previously, quick access to representative source data is important to me, so using Dukascopy with its wacky interface is largely out of the question. My choice of data provider allows me to download in bulk, on the fly, in a matter of minutes. My new system allows me to quickly convert this into a tick data file with 99% quality (or is it 100%?), (big smile .....). BTW, did I mention it can also generate renko tick data?
