Menu

Options

Bookmark Thread

First Page First Unread Last Page Last Post

Print Thread

Similar Threads

Optimization, statistics, and avoidance of curve fitting 21 replies

Curve Fitting: The Ultimate Conundrum 17 replies

EA Backtesting and Curve Fitting Concern 2 replies

why curve fitting doesn't work? 23 replies

"curve fitting" or just improving?? 5 replies

Attachments: Curve fitting vs optimization
Exit Attachments

Curve fitting vs optimization

  • Post #21
  • Quote
  • Edited 8:42pm Dec 7, 2019 5:48pm | Edited 8:42pm
Quoting HeyYou
{quote} yes everybody acts as a succesful millionaire, they come here mocking backtesting...they call us "delusional" etc. but in the end who is making money ?? ME HERE... 45 bucks! {image} Just kidding guys, JK, JK... jeeesus, all this seriuosness was killing me.
Quote
they come here mocking backtesting...they call us "delusional" etc.
Dude -- no one came here mocking backtesting. Every post I've read so far recommends backtesting. The only difference is whether to take a shotgun approach and backtest with as much data as you can get your hands on, or whether to think more deeply about what you are attempting to do and what your tools are capable of and take a more refined approach. And understanding a little bit more about how algebra and statistics work probably wouldn't hurt.

One of the most frustrating aspects of posting on FF is many people can't read -- or, they can read but their brain only absorbs what they've already decided they want to believe.

And just to make sure I hadn't missed something, I used my highly honed detective skills (along with my browser's Search feature) and searched for the word 'delusional'. Turns out only one person uses it -- the OP. So, it seems the OP has a belief that people consider backtesting 'delusional' and even though no one has suggested that, has still come to that conclusion.

Quote
Just kidding guys, JK, JK... jeeesus, all this seriuosness was killing me.
Oh, I see -- I get it now. You weren't really being serious. This was just intended to waste people's time. Another joker...

Mr. OP, I see you have 133 Ignores -- Holy Smokes, man -- even more than me. Please add me to your list...
To make it clear.

I can make a f joke if I feel so, especially on my own thread. Oh sometimes I abuse the ignore function, I'm working on that...
Quoting hanover
{quote} My 2c fwiw: Excellent posts by topherhk88 here and merlin here. Bruce Babcock, a well known futures trader during the 1980s-90s, wrote a lot of good material about optimization. I found this as a starting point; you might be able to find more of his writing using Google....

Wow, these posts from 2006-2008 were amazing h, thanks.
about the role of humans in quantitative investing and the fallibility of systems I'd suggest watching this interview


https://insights.som.yale.edu/insigh...tive-investing
Now I would like to answer to our friend who doesn't want me to make jokes.. first when I said "here" I was referring to the WHOLE forum

what about his "superior approach": 3 months is not backtesting.. it's something else.

what about the broker's data feed... that's completely irrelevant for obvious reasons: each broker has a different feed.

I'd use open prices to backtest and forget precision entry.. what ?? what ?? I'm a joker ? oh well, thanks.
Quoting HeyYou
Now I would like to answer to our friend who doesn't want me to make jokes.. first when I said "here" I was referring to the WHOLE forum what about his "superior approach": 3 months is not backtesting.. it's something else. what about the broker's data feed... that's completely irrelevant for obvious reasons: each broker has a different feed. I'd use open prices to backtest and forget precision entry.. what ?? what ?? I'm a joker ? oh well, thanks.
Which friend? robots4me? If you want him to see this, you might unignore him first although he wanted to

Anyway, I must add that some strategies I find almost impossible to back-test.. for example this "ideal entry" types (CHEETAH), when we try to find the perfect/precise ENTRY time to get max at max leverage max position... Some people were asking me if I have backtested this.. I mean, the hell

But yeah, there are some systems that could be backtested good, specially some co-relations (parameters, events, whatever)...
Can you afford to take that chance?
Quoting auricforecas
{quote} Which friend? robots4me? If you want him to see this, you might unignore him first although he wanted to Anyway, I must add that some strategies I find almost impossible to back-test.. for example this "ideal entry" types (CHEETAH), when we try to find the perfect/precise ENTRY time to get max at max leverage max position... Some people were asking me if I have backtested this.. I mean, the hell But yeah, there are some systems that could be backtested good, specially some co-relations (parameters, events, whatever)...
yeah.. that guy... what the hell, """" sniper accurate """ entries/exits on cfds ???

the pip doesn't really matter.
Quoting HeyYou
{quote} yeah.. that guy... what the hell, """" sniper accurate """ entries/exits on cfds ??? the pip doesn't really matter.
That is what we are trying to find out here
Can you afford to take that chance?
Quoting auricforecas
{quote} That is what we are trying to find out here
you are a friend A and you can say what you want, you can even call me names... but this thread is for "normal people"...

you are a CHEETAH !!
Quoting HeyYou
{quote} you are a friend A and you can say what you want, you can even call me names... but this thread is for "normal people"... you are a CHEETAH !!
Right Roger Yes, I admit "CHEETAH" are not normal people BTW I also hate when people are expecting anything "normal" out of it...
I also have or participate in "normal" threads... but I am almost disappointed that there are not much interest there

Also, on-topic... was kinda excited for some "math-stat" people to come teach us how to do it better, PipMeUp showed some promise but then he stopped... BTW I use curve and linear fitting a lot, if for nothing else... as an additional component/verification at some entries/strategies
Can you afford to take that chance?
Interesting links, especially by hanover.

From my little experience:

a) mechanical system is the way to go
b) recalibration of basic parameters of the system (e.g. period) is necessary time to time but not too much otherwise it leads to system switching
c) recalibration of SL/TP parts of the system is necessary on a regular basis (I do it weekly using percentile function)
d) quantitative analysis of the data (e.g. what were the max/min positions of the signal, what are the best hours to execute the signals, etc) are also necessary and I believe very few people actually do it. Heck I wasn't doing it for many years and now I couldn't live without my trading sheet.

Putting everything into data-perspective can also confirm whether your system really has a XY% WR. Otherwise you can't say whether it was just a part of random distribution of results or a real edge.
Quoting mades
Interesting links, especially by hanover. From my little experience: a) mechanical system is the way to go b) recalibration of basic parameters of the system (e.g. period) is necessary time to time but not too much otherwise it leads to system switching c) recalibration of SL/TP parts of the system is necessary on a regular basis (I do it weekly using percentile function) d) quantitative analysis of the data (e.g. what were the max/min positions of the signal, what are the best hours to execute the signals, etc) are also necessary and I believe...
all of these are correct!
changing the parameters because they didn't work well in the past 3 months is crazy. you need at least 10 years of data...
  • Post #34
  • Quote
  • Edited 12:09pm Dec 18, 2019 10:14am | Edited 12:09pm
Quoting PipMeUp
{quote} If you draw samples from a normal distribution, that never changes over time, how many samples do you think you would need to estimate within ±1% error margin the mean of this distribution?
Well, I do what is called Monte Carlo analysis on a Wiener process. The results are by definition normally distributed, to the extent that one can trust the function NORMSINV(rand()) in Excel 2010. I have just one parameter, volatility, which is constant, and a starting price.

I use 250 rows (each row is a new observation, the timescale matters not) and 5,000 scenarios of simulated price paths. Each time I hit recalculate (F9) (the rand() function recalculates 5000 times 250, so recalculate automatically is disabled) the averages are "slightly" different. I'd say by about a few percent, less than maybe 4 or 5% but definitely more than 1%. To tabulate this was not the purpose of the exercise.

It would be a bit tricky to think of which result would be relevant for the purpose of this thread except to say that the extremes can be very different each time I recalculate, despite any average being relatively stable within acceptable limits (by design).

Purely theoretical, so nicely behaved. Its the extremes that are closer to reality - financial markets are non-stationary, fractal, female, and everything else...(this is possibly the bit that the clever folks at Long-Term Capital Management glossed over).

Edit - but to answer your question, it would depend. If population size was 10, you would only need one sample of 10 and you would have the exact mean, regardless of the type of distribution - none of this multiple samples or margin of error statistics stuff. Maybe you may like to rephrase your question?
  • Post #35
  • Quote
  • Edited 2:21pm Dec 18, 2019 12:46pm | Edited 2:21pm
I wouldn't bother optimising beyond what is generally accepted. Really.

I have about a thousand reasons for saying that, probably most of which have been articulated by others, including the late Mr Babcock who was mentioned here and whose books I did read all those years ago. One is that despite having say a 100 trades, what if one of the settings was only responsible for one trade, say a stop that was triggered once, which influenced the entire results, and without which the results would have been forgettable? Optimising everything else may make a difference, of course. But....

you would need to go through each trade individually to actually know. And it is circular, because that one individual trade then influences the settings of every other trade. This is not robust. That is reality. My example is extreme, of course, to illustrate my point.

Lets say you have 3 parameters, A, B and C, each optimised over just 5 settings, say 20, 21, 22, 23 and 24 for A, similar for B and C. That is kinda tiny, yet there are 125 combinations (check for more parameters and multiple times as many possible settings).

Once you have results, you would have to be sure that they were all OK in the neighbourhood of the best, that it wasn't just a fluke, and that each combination in itself had sufficient trades.

Etc. None of this is new, and that is kinda just the introduction.

Anyway, should you use an indicator in a unique way (or design your own, which is kinda fun once you get the hang of it), then you will have to do some trial and error. More important is to know your indicators intimately and what market conditions they are favourable to. All markets are always favourable to some approach, and market conditions tend to persist. Until they change. I care not for opinions saying otherwise. To ignore that salient feature of price action is indicative of laziness on the part of the trader.

So, use appropriate tools/settings for the market situation as they arise (this is one feature that differentiates the technical trader from one using fundamentals, I think, but I digress). Actually, the easiest way is to make the indicator a *function* of the market and this is done all the time - for example stops a number of daily ranges away if you use stops - by all means optimise things like that should you have certain risk preferences.

Using a different approach to different market conditions is what some may say approaches curve fitting. No, it is just the same as you may wear different clothes depending on the weather. Common sense.
I have a confession, I discarded perfectly good systems bcause they didn't perform well in the last couple years.. to then realize they weren't well optimized.

don't make my mistake... spend more time optimizing lol
Here is a good place for this link:
http://grailtrading.blogspot.com/200...ent-wrong.html
  • Post #38
  • Quote
  • Edited Jun 25, 2020 10:17am Jun 24, 2020 3:20pm | Edited Jun 25, 2020 10:17am
Quoting Weyk
Here is a good place for this link: http://grailtrading.blogspot.com/200...ent-wrong.html
Thanks, I read "Starting on October 1st 2005 I aim to trade my way to £1 Million Pounds in 1000 days." Well... what is his/her starting capital ?

I think it's essential to keep expectations low. For instance 10%/year consistently for 20 years looks good to me.


The article also looks good and I agree that long term backtesting is a must in fact I'm using a system backtested over a 200+ years period
I too agree long term backtesting is pretty much helpful. It helps us to analyse our mistakes in the past trades and let us work on those mistakes. Backtesting can definitely help us to accurate results in the future. If the same strategy does not bring similar results one can switch to other strategies.
Quoting willemcurtis
I too agree long term backtesting is pretty much helpful. It helps us to analyse our mistakes in the past trades and let us work on those mistakes. Backtesting can definitely help us to accurate results in the future. If the same strategy does not bring similar results one can switch to other strategies.
yep, if the system does not work, it does not work! that's my motto!
Fact is that I have some sleeping money to invest..... and a system that worked for many years is appealing.


now, if you ask me why some systems worked in the past and why they should continue to work into the future... IDK. I just take the risk because I don't really know how to spend this money lol
0 traders viewing now