Building upon the excellent suggestion from @FXEZ, I tried to build portfolios of many different parameter sets to average out the performance of the individual settings. In the plots below, I sorted on the various backtest metrics as provided by ...

Finding The Most Robust Parameters Using Optimization

Good idea! That looks like a good way to detect plateaus and easily extends to n-dimensional parameter spaces. In the past I did some attempts to calculate the gradient to find the peaks in parameter space but I was not very successful. Your ...

Finding The Most Robust Parameters Using Optimization

Thanks again for the link. I have read the article. What it describes is a pragmatic introduction to MC, what Aronson describes in more detail in his book. However, both the article and Aronson only consider one single system at a time. How can we ...

Finding The Most Robust Parameters Using Optimization

Thanks for the link. I'll have something to read under the christmas tree. Interesting idea. That way it doesn't hurt when I accidentally pick a loosing parameter set, I choose a whole lot of them. Top 5 with highest Sharpe, plus top 5 with highest ...

Finding The Most Robust Parameters Using Optimization

Thanks for the link. Much appreciated.

Finding The Most Robust Parameters Using Optimization

Hahaha, now you sound like a fairy tale wizard speaking in riddles ;-) Not sure what to do with this advice...

Finding The Most Robust Parameters Using Optimization

Can you please elaborate on that? I am familiar with the Monte Carlo Permutation Method (MCP) as described in Aronson's book "Evidence-Based Technical Analysis" (short online extract here). He uses that method to compare against the Null Hypothesis. ...

Finding The Most Robust Parameters Using Optimization

I tried to simulate two "neutral" strategies (i.e. strategies without an edge, breaking even on average). One with 50% win rate, risking 2% equity per trade, RR 1:1, and one with 33.3% win rate, risking 2% equity per trade and RR 2:1. When ...

Finding The Most Robust Parameters Using Optimization

The example strategy used here is just simple price action (breakout strategy using a Donchian channel) without any ML algo, but in post #15 I wondered whether we can use ML to predict the forward test performance based on the backtest performance ...

Finding The Most Robust Parameters Using Optimization

For a coin flip we can use the binomial distribution to figure out what sample size we need to know with p=0.995 certainty that the coin is fair (i.e. 50% win rate). If the coin was unfair (e.g. 35% win rate), wouldn't we need more samples to know ...

Finding The Most Robust Parameters Using Optimization

Could we perhaps train a Neural Network using the backtest metrics (e.g. Sharpe, Recovery Factor, Profit Factor, etc.) as inputs to learn how to predict the performance of a strategy? NN's look like a good tool to combine several inputs into one ...

Finding The Most Robust Parameters Using Optimization

Thanks for further elaborating your ideas! Yes, we humans tend to under-estimate the raw computing power we have today. These tools can find really nice linear equity curves in completely random data. Literally! I demonstrated that once to a ...

Finding The Most Robust Parameters Using Optimization

Yes, of course! You can not use 2001 data to find the best settings for 2001. That would be cheating. However, of all the possible parameter values we tried out on the 2000 data, which are the ones that most likely give good profits in the following ...

Finding The Most Robust Parameters Using Optimization

Yes. absolutely! I fully agree with you that you should not use all 20 years of historical data in one go to optimize. Indeed, what happened in 1999 is hardly relevant for today's market. However, what happened in 1999 was relevant for the market of ...

Finding The Most Robust Parameters Using Optimization

Interesting suggestion. I could try that by adding an input parameter to the EA to only allow it to trade on a certain day-of-week and do the full optimization process 5 times. Five times more work, but interesting to try once based on your ...

Finding The Most Robust Parameters Using Optimization

How do you suggest to include the time factor? Should I search for different parameters per day of week? This is just a simple EA, but for more advanced EAs using machine learning I always avoided to use time-of-day, time-of-week or day-of-week as ...

Finding The Most Robust Parameters Using Optimization

No, I am using real ticks including spread and slippage simulation (1000 ms) in the MT5 strategy tester. This is just a simple breakout strategy using only price action. Without any machine learning involved backtesting is very fast. There are 1250 ...

Finding The Most Robust Parameters Using Optimization

Here is the data used to generate all the graphs: file

Finding The Most Robust Parameters Using Optimization

The previous version (1.90) added logging the trade execution to file (see post #101). After running the EA for weeks, these log files can become quite large. Especially on systems with limited disk space (e.g. rented VPS) this can become a problem. ...

DOSI: Daily Open Strategy Improved