How do I know if I have an edge? A thread... ππππ I've been helping...

A thread... ππππ
I've been helping a family friend with his trading. I've given him a simple systematic strategy to trade by hand.
We can plot the distribution of historic trade returns from past trading or a backtest as a histogram.
1/n
This is useful because it gives us a hint as to what the "edge" of our strategy might be - if we could ever truly *know* such a thing.
2/n
We saw winning trades about 58% of the time but losers were bigger, on average, than winners.
(As many things that make money tend do, regrettably)
3/n
You might think of it like we're picking observations out of the bucket described by the histogram we just made.
BUT....
4/n
The "true distribution" changes with time (it is stochastic) and cannot be observed directly. We can only infer it from the past.
5/n
We might run rolling stats to try to observe the time-varying nature of things - but we'd be working with only a few samples and the variance of our trade return is large.
6/n
Now we have set reasonable expectations about the P&L distribution, my friend starts trading...
7/n
"I've learned to exit later when momentum is in my favour" etc...
This is what humans do. They look for patterns in noise.
8/n
He's just fitting stories to random data. Individual trade P&L carries no useful information.
9/n
Imagine we're building that up trade by trade, observation by observation.
How many points would you need before it had a meaningful shape?
10/n
But everything is non-stationary (it changes with time) so our observations always arrive later than we want them to, and there are never enough of them.
11/n
You get plenty of useful quick feedback on things such as market impact, but the data on "edge" takes forever to collect and stuff is constantly changing underneath you
12/n
You need a quantitative approach. You need to analyze in aggregate. You need an understanding of stats
13/n
You need to understand you can never *know* if you have an edge right now.
14/n
You never know if you have an edge right now, but when you think you do - sample from it as much as you can in the simplest, most robust way possible.
15/15
If you find a good source of edge, your primary objective should be to harness it in the highest probability way possible, as often as possible.
Which demands simplicity, usually