Published on Wed Oct 16th, 2013
If you came here, you should have already downloaded your tick data either for free as mentioned in the Free Tick Data page from your own private source and now you are trying to use it in Metatrader 4.
Converting tick data file format
Native CSV file containing tick data are incompatible with Metatrader 4, so it can't be used in backtesting. FXT is the proprietary file format that contains ticks and can be read by Metatrader 4, so simply we need to convert CSV files to FXT. There is a script able to convert data from mostly any source to an FXT, it's available in the Downloads area here. HST files can also be used by Metatrader 4 when calculating the indicators, so HST files matching the FXT file are required for accurate tick data backtesting.
CSV2FXT binaries is what you need for the conversion process, it's available in the tick data downloads area too. It has been compiled with MT4 build 225 to widen its working range on MT4 builds.
Finally, here's a short guide explaining how to convert the tick data CVS file to an FXT file and a series of HST files:
- First, if you still don't have CSV2FXT binaries archive, navigate to the tick data downloads area to download it.
- Extract the files from the zip archive to the MT4 installation folder. A directory structure is inside, be certain that they are placed in the proper directories (CSV2FXT.mq4 and CSV2FXT.ex4 must be in experts\scripts, FXTHeader.mqh must be in experts\include, CsvReader.dll must be in experts\libraries). Be certain too to copy them to the correct folder if there are multiple MT4 installations.
- Move the CSV tick data file to experts\files in the same MT4 installation folder.
- Open a chart for the pair that you have data for (e.g. EURUSD chart for EURUSD.csv file).
- Select the timeframe that you want to create the FXT for. For instance, if you want to backtest on M1, you should select M1 timeframe for the chart, while the FXT file created for a certain timeframe won't work for other timeframes; for backtesting on another timeframe you simply have to create a new FXT.
- Be sure now that your terminal is connected to the broker before proceeding (at the lower right corner, if it is “not connected†you have to fix that first).
- Be sure too that DLL calls are allowed. You can do that by opening the Tools menu, select Options, select Expert Advisors then ensure that Allow DLL imports is enabled while Confirm DLL function calls is disabled.
- Double click the CSV2FXT script in the navigation panel, it's available in the scripts area.
- A pop up window will open, configure the parameters there.
- CSV2FXT_version: it's a parameter that just quickly indicates what version has been installed. There is no effect of changing it.
- CsvFile: it can be left blank if the name of the file is just like the symbol and with a CSV extension (e.g. EURUSD.csv); if not, just type the file name.
- CreateHst: for HST files to be created for backtesting, this setting have to be true. If set to false, as you may have already generated HST files and you just need to generate an FXT for a different timeframe. Whenever you changed the GMT or DST, you have to create new HST files.
This setting if enabled will lead to HST files formation for the whole timespan of the tick data file regardless of the time range selected.
- Spread: this is the fixed spread of the resulting FXT file, expressed in pips; 2.3 will result in a 2.3 pips spread. Its default value is 0.0, letting it so will make the converter use the current spread of your broker. You should care about the fact that many brokers are widening their spreads during the weekends.
If you plan to use real spread; the variable spread in the CSV file, you can then leave this parameter set to its default.
- StartDate and EndDate: these settings determine the FXT file time span. The StartDate and EndDate format is YYYY.MM.DD. If you want the converter to use the CSV file earliest and latest date available respectively, just leave these fields blank.
- UseRealSpread: as the name indicates, this parameter if enabled will make use of the CSV file real (variable) spread in the resulting FXT.
By using the loader script, it'll be a must to enable real spread to be able to use such FXT files. The Tick Data Suite can detect automatically if the real spread is used by FXT file, so you don't need to pay attention to this part if you used it.
- SpreadPadding: this setting is to pad real spread - if being used - by a certain pips count; if you want to pad it by 0.8 pips, just add 0.8 here.
- PipsCommission: if you want your resulting FXT file to include a commission, the desired value can be set here. The figure is a back and forth and is expressed in pips.
- MoneyCommission: it's set the commission expressed in money as an alternative to be in pips. The value is expressed in base account currency per lot.
- Leverage: changes the FXT file leverage.
- FXTGMTOffset: to set the FXT GMT offset to other than 0.
- CSVGMTOffset: the CSV file data GMT offset. The conversion script can detect automatically several free tick data providers formats and apply the correct setting here, so it could be safe to leave it set to “autodetectâ€. If the experts log revealed a message that the script is unable to identify the tick data source, you will need to manually set the CSV GMT offset here.
- FXTDST: the FXT file DST setting. 0 means no DST, 1 means US DST, 2 means European DST. Be aware that the DST is calculated if the US DST setting is set, according to the regulations that were applied by the start of 2007.
- CSVDST: the CVS file DST setting. it's safes to leave it to “autodetectâ€, or you can do the same as for the FXTDST.
- TimeShift: This parameter if enabled, will shift all the created data 28 years backwards. This can be done if you suspect an EAs to have hardcoded days aiming to backtesting cheating. It's determined by 28 year shift as the calendar is identical when dealing with the days of the week and leap years. This may be unfair to judge an EA using this method, as some EAs may have legal causes to dissimilar result by backtested with a shifted time.
- CreateM1, CreateM5, CreateM15, CreateM30, CreateH1, CreateH4, CreateD1, CreateW1, CreateMN: These parameters aim to allow the creation of multiple FXT files in a single run. By default, the script will create the FXT for the timeframe of the already running chart, no matter if that specific timeframe parameter is enabled or not. You don't need enable these parameters unless you really need the FXT for a different timeframe.
- Click "Ok". Once done, the data creation process will be launched, typically taking 20 to 45 minutes, depending on the data range and volume, could be even more if a slow machine is being used. A progress indicator will be in the chart upper left side and by finishing, an alert will come out. Throughout the conversion, some data will be printed in the experts log for any problem that could occur.
Once every thing is done and the script finishes processing, you will be prompted to determine if the files will be move to their proper locations. If Yes is selected, restart Metatrader 4 terminal to be certain that the HST files are well synchronized, then directly proceed following the next guide; Using the tick data in your backtests. If No is selected, you there will be an FXT file and a pack of HST files in experts\files folder and before using them, you need first copy and paste them in the correct location, then go on following these steps:
- Exit the Metatrader 4 terminal.
- Move all the .HST files from experts\files to history\[your_server_name].
Be very aware of the number server directories in the history folder, the .HST files have to be in the correct one for active account.
- Move the created FXT file from experts\files to tester\history.
Now, it's all done with the conversion and setup. It's time now to proceed to backtesting, but take in consideration that it isn't as easy as clicking Start in the backtesting user interface. Some additional tools which are described in the Using the tick data in your backtests part, may be called for help.
If faced any problem, please navigate to the FAQs and Troubleshooting page.
