Trader’s Tech – Writing Your Own EA Part 39 – Test Data

Forex RobotIf you’re new to this series and want to check it out from the beginning, you can find that here. And look here for a list of all the programming articles.

In Part 38 we talked about the Strategy Tester (ST) report and how to use it. To wrap up our discussion on the ST, we’re going to talk about good test data.

I’m no expert on good test data, there may be others in our readership that could shed more light on the subject. If so, please don’t hesitate share and/or correct anything I may share. In my quest for accurate back testing of strategies I’ve learned a little about getting quality back tests from your your MT4 platform.  I’ll share my sources and software for doing just that.

As you may know, MT4 downloads data as needed. You can change the amount of data that is downloaded by changing settings in the options. For the purposes of getting quality data, you don’t want any of that data downloaded. The only way to do that is NOT sign into MT4 when it opens. Just click Cancel-Cancel when it tries to get you to create an account and log in. The live charts will not work, but the ST will still work fine.

The next step is to find quality data. There are several places you can get a good data feed. If you have a sufficiently sized account with Oanda (I’ve had an Oanda account for many years with very few problems), you can get free data. I’ve found that Dukascopy is much better for free data. You can download the data here. I was only able to download one month of data at a time. It’s possible that I was missing something, but I spent a LOT of time downloading data this way. The Casey sent me a link to the Tick Data Downloader. This software allows you to download all the data you wish without intervention. Just kick it off and walk away. Be sure you have LOTS of space for the data. I downloaded about six and a half years of data for about 10 pairs and it fills 55GB of space on my drive. At home I have a pretty fast Internet connection (typically over 30mbps) and it took quite a while to download.

Trader's Tech - Test Data 1Using the Tick Data Downloader is pretty straightforward. You tell it what data you want, where you want it and click the Download button and in a little while, you’ve got your data in CSV format. You can use the data in a spreadsheet in this format if you wish. If you want to use it in MT4, there are a couple more of necessary steps.

Instead of going into great detail about downloading and preparing data for your MT4 platform (which would be reinventing the wheel) I’m going to give you a link to the site where I got all my information. Birt’s EA Review starting on the Tick Data page. I will give you a few suggestions that helped me out.

Once you’ve downloaded your CSV data, you have to convert it for use in MT4.  This is the link to download the conversion script. You will have to run the conversion script on a live MT4 platform. You won’t be able to run it on the platform you’ve prepared for your testing since it needs incoming ticks to trigger the script (remember when we discussed the fact that EAs and scripts are triggered by incoming data?) So you will need to install another MT4 platform on your computer and set it up with a demo account to get live data. Yes, you can install as many MT4 platforms on your computer as you like. My desktop computer (back in my office) has about 12 different broker demo platforms installed (for testing EAs with different brokers.) I usually put them in a folder called c:MT4s. Then I name the folder for each specific platform that will help me identify the platform, for example, c:MT4sBackTesting or c:MT4sOandaDemo1.  The other thing you should remember is to change the name of the shortcuts that MT4 installs. They all default to the same name (no matter what you put in the installation wizard), so you can get really confused really fast. If you change them right at installation time, you’ll alleviate most of the confusion. Once you’ve downloaded the script, you will have to drop it in the Scripts folder of your live MT4 platform. The details for this procedure are here.

When you execute the script, you will be able to pick the time frames for which you want data converted. To conserve disk space, I recommend only converting the pairs/time frames you will actually use for testing. If you have tons of disk space (terabytes), then I would go ahead and convert them all.

Since you’re using a different platform to convert your tick data than the one you’ll be using for testing, when it asks you whether you want the script to move the files to their proper locations, you will want to say No. You will need to move the data into the proper location.  For the purposes of this discussion, let’s say that your platform for converting the data is c:MT4sConversion and the testing platform is c:MT4sTesting. You will need to move the c:MT4sConversionexpertsfiles*.fxt files (one for each pair) to the c:MT4sTestingtesterhistory folder. These will be humongous files so it may take a little while to move them, depending on what hard drives you’re using. You may want to delete them from the Conversion platform since they can be quite large (gigabytes in size – depending on how many years of data you have.)

That’s it for today. We’ll pick up this topic tomorrow and discuss more suggestions for data conversion and usage.

Thanks for your attention and please follow me on Twitter and LinkedIn.




The following two tabs change content below.
Winners Edge Trading was founded in 2009 and is working to create the most current and useful Forex information and training available on the internet.

Winner’s Edge Trading, as seen on:

Winner's Edge Trading in the news

  • Dave Hanna

    Thanks, Tim. That sounds like extremely useful information.