Validator

The Validator exists so you can easily recalculate old strategies on new data. Testing your strategies in a later point in time allows you to see which ones are still profitable on the new data you didn't have before.

The Validator works only for strategies made with EA Studio.

Basic Workflow
In the Validator you can import Collections, Expert Advisors and Portfolio Experts. The strategies from the files will be loaded in the Validator's memory.

When you are done importing files you can set various requirements that the strategies have to meet. If a strategy backtest shows profit and meets your requirements it will be automatically added to the Collection.

You might already have some strategies in the Collection before using the Validator. To avoid mixing the old and the new strategies you can go and delete all strategies in the Collection beforehand. If you want to keep your old strategies you can export the Collection before deleting the strategies.

At a later point in time it might be hard to remember when you created and last backtested each strategy or collection. To know when you created/last validated your strategies you can put them in a folder. The name of the folder containing the current date. For example “Strategies EURUSD H1 1-Oct-2017”

Toolbar
Progress  - shows what percentage of the imported strategies have been calculated.

Calculated  - Number of calculated strategies so far.

Ascended  - The number of strategies that passed all validations and were sent to the Collection. (The number of strategies in the collection might not increase though. This is because the Collection accepts only unique strategies. If a strategy already exists in the Collection it won't be added twice.)

Import Strategies
You can import all kinds of files you exported from EA Studio: Collections, Expert Advisors and Portfolio Experts. The importer does not check if the strategies are unique. Specifically this means that if you import a collection of 50 strategies twice the importer will show “Loaded strategies: 100”.

Imported strategies will stay in the Validator until you use the  Remove all  button. You can leave the Validator tool and do some other work with EA Studio. When you return the strategies you imported will still be loaded in the Validator.

2. Historical Data
The strategies you import might have been created on different markets and periods. However the Validator will test all imported strategies on a single market and period. The market and the period are set here.

The  “From to “  text in the bottom of the box is useful. This shows us what is the timespan covered by the data. If the timespan is different from your expectations you might check the Data Horizon settings.

3. Validator Settings
Here we have less settings than in the Generator because we are not creating new strategies. We are only validating older strategies.

You can select what percentage of the data to be used for Out of Sample testing. Also you can specify if the strategy should pass the Acceptance Criteria to pass validation.

The Validator can run the strategies through a number of additional tools.

We can toggle the additional tools by using their checkboxes. When a tool is enabled we can also see a panel with statistical data on the right side of the screen.

When a strategy is validated goes through the enabled tools. For a strategy to be pushed to the Collection it has to pass all the enabled tools' tests first.

It is a good practice to enable Acceptance Criteria in the Validator Settings tool. This guarantees you won't be optimizing losing strategies in the first place. There is little value in optimizing losing strategies since the Optimizer can make them lose less but this won't turn them into winning strategies. Optimizing might for example only reduce the number of trades to near zero so this will minimize losses but won't make the strategies more profitable. The Acceptance Criteria (by default) require a strategy to have a statistically significant number of trades (100). Only then the strategy will pass to the next filter.

You might however disable Acceptance Criteria for different reasons. For example if there is a market where there is not enough data to meet the Acceptance Criteria for trades but you still want to create a strategy for this market.

Full Data optimization
The Optimization tool displays a generalized version of the Optimizer. This allows to apply optimization on many strategies with different indicators.

Optimize Stop Loss and Take Profit  - if enabled will optimize ST and TP only on strategies that already have ST and TP in their logic. This option won't add ST or TP rules to the strategy if it does not have them already.

It some of your strategies have ST and TP and you want to keep these values unchanged, you should leave this option disabled.

Numeric Values Range  - The Optimizer tries to find better strategies by changing the numeric indicator parameters of the primary strategy. This field lets you choose in what range the Optimizer can change these parameters. A smaller range will allow the strategies to be optimized faster but the gains might be only marginal. On the other hand a greater range allows for better optimization but this is a two-edged sword because over-optimization can lead to curve-fitting and fragile strategies. It is a good practice to always couple the optimization with one of the robustness testing tools - for example Monte Carlo.

Search Best  - The optimizer will optimize the strategy according to which parameter you selected. For example if you choose Net balance it will try to make the strategy profit more.

Out of Sample  - What amount of data to use for In Sample and what amount - for out of sample testing.

The strategies fulfill the Acceptance Criteria  - if you enable this option the optimizer won't let a strategy pass through this tool if it does not fulfill the Acceptance Criteria.

Walk Forward optimization
Walk Forward analysis (WFA) or Walk Forward optimization (WFO)  is a sequential optimization applied to an investment strategy. The name of the analysis is called “walk forward” because we have a moving window that progressively traverses the whole period of the data history with a pre-established step. Algorithmic traders apply such analysis to decrease the over-optimized parameters used in the investment strategy as we don’t want only a great looking backtest result, we want also a system that don't fail in live real money account.

Read more about it at Walk Forward

In the Reactor context you can apply this optimization to the strategy, if the WFO results (Out of Sample net) perform better than the original strategy (full data backtest) the last parameters used by the optimization will be accepted to the strategy.

Perform robustness testing
With the previous tools you could check if the strategy is still profitable and attempt to make it more profitable. The tools below have a different goal. The robustness testing tools let you analyze how trustworthy a strategy is. This step is very important.

Walk Forward validation
You can use WFO as validation tool and it returns only strategies that pass the validated segments without changing the original parameters of the strategy.

Monte Carlo Validation
The idea of the Monte Carlo tool is to introduce some “noise” into the historical data and the strategy execution. The goal is to take the strategy out of its ideal comfort zone. This is an attempt to predict if the strategy will still perform well in slightly different market conditions.

The Monte Carlo tool has detailed settings that you can set on its own page. In the Validator the settings are simplified for easier use.

Count of tests  - The Monte Carlo tool will run this number of tests. Each test includes random noise in history data and strategy execution.

Validated tests  - Select what percentage of tests that should pass the Monte Carlo validation criteria. For example if we set it to run 10 tests and specify 90% validated tests then at least 9 out of the 10 Monte Carlo tests should pass through the validation.

Multi Market Validation
The multi market tool will validate the strategy on the selected markets. The theory says that if a strategy works on more than one market then the strategy is probably not curve fitted to one specific market.

The tool will use the data from the broker you selected in step  2. Historical Data. Multi Market has also its own settings in the Multi Market section in the Editor.

Since the Multi Market tool can use only that is already uploaded it is a good idea to upload data for more than one symbol if you plan on using Multi Markets.

Add market  - lets you add a new test market.

Validated markets  - specify how many market tests should pass in order for the strategy to pass the Multi Market validation.

https://forexsb.com/wiki/eas-guide/validator