Arthur Benjamin: Teach Statistics Before Calculus!

I cannot emphasize enough the importance of thinking in terms of probability and how useful it is for anyone in making any kind of decisions that involve trade-offs and risks.

In this TedTalk, Arthur Benjamin provides some very good reasons why it is better to teach everyone statistics over calculus.

Personally, I find that it is never too late to learn statistics and think in terms of probability. It is not just a very good tool in making trading decisions. It is also a life skill that can transform your life and put you in better perspective of things happening around you and on you.

When I get the chance I will write about these interesting stories with the people I encountered whom I showed them how to think probabilistically. It is a life transforming experience for them. I learned a lot from sharing my thoughts with them too. It is something I always remember as it shows that a slight change of perspective in an individual can make all the difference.

[DTBpromo]

Essence of Trading: Price Goes Up And Down

iStock_000017014380XSmallBefore you spend time even thinking of the more complex price patterns or technical setups, have you even paying any attention to the most basic foundation of the price charts, like the price bars themselves?

It does not matter you use candlesticks or open-high-low-close bars (OHLC bars). The important thing is whether you know anything about your market and timeframe down to the bar to bar relationship. Many would say no. There is nothing to be ashamed of because we are not told to even look at that. Most of the materials you can find on the Internet just skim over the definition of the bars and then immediately move on to talking about the chart patterns, indicators, etc.

It is important to know the basic statistics about these bars you are using. I am not talking about fancy statistics here. Just simple ones that tells you what is more likely. Price movements, down to transaction level, in its simplest form, has 3 possible outcomes. Trade at a higher price, same price or lower price. Price goes up. Price goes down. It cannot get any simpler than this. The important thing here is to know if there exists bias in the simplest price relationships that you can exploit.

Let’s consider a simple coin-flip betting game. If you know for sure that someone is using a loaded coin that favours head over tail (e.g. 70% of the time you get head) when you flip the coin. In almost all normal circumstances, and that your intention is to win money, you would keep betting on head, right?

So if you know that the market you trade, say, on daily basis, has a 70% chance of printing $5 higher as long as it has not dropped $3 yet, would you not keep buying everyday?

Or if you know every Thursday the market you trade has a 65% chance of closing higher, shouldn’t you pay more attention to the possibility of a late day rally that can print a higher close on Thursday?

Every market and timeframe combination has its own special bar to bar statistical behaviour. It is caused by the traders who participate in that market who focus on that specific timeframe. There is nothing magical about that. Assuming these price movements all behave the same way is one of the major contributors to many chartists’ inability to interpret their charts correctly.

Many smart individuals assume that such basic properties of the price bars would be well discussed or mentioned in books if these statistics have any useful bias at all. Since they cannot find such information readily available, they assume the statistics are either not important or not good enough to be utilized profitably. They could be correct for some markets in certain timeframes, but not all markets behave like that. The several free end-of-day swing trading models I published clearly illustrate that there are useful biases hidden in plain sight all along with no publicity whatsoever.

It is not that useful (and simple) biases do not exist.

It is people not trying hard enough to discover them.

[DTBpromo]

Clarifications to Some Basic Concepts in Probability

I wrote many articles on statistics over the years (Teaching a Grade 4 on Probability, Understanding the Market in a Statistical Way, etc.) and advocate traders to pay more attention to proper application of statistics in trading and model development.

I came across this educational video by Peter Donnelly from TED that explains several basic concepts in statistics clearly.

There is no point for me to repeat his work, so I will just include the video here.

 

 

In case your browser does not support HTML5 embedded video, here is the link directly to the lesson itself,

http://ed.ted.com/lessons/peter-donnelly-shows-how-stats-fool-juries

 

Relating to the Video

The HTH vs HTT occurrence frequencies – It is one of the most overlooked and misunderstood basic pattern recurrence behaviour.

Many rookie traders and trading model designers often assume that certain common patterns (price, indicator, geometry lines) are useless because of their potential frequent recurrences. What they have overlooked, is that, first the assumed common patterns are not that common (like HTH) and second, the importance of any recurring pattern is not how common or rare it is, but the consistencies in the expected outcome following the occurrence of such pattern.

Here is a good teaser – How often does S&P make new year high after the last occurrence has led to 1% (or more) drop in price?

The 99% interpretation issue – You think your trading setup has a 80% win rate is pretty good, think again.

Most successful traders know their own performance pretty well. They are often better than computers in analyzing real-time scenarios. In their heads, they can picture the possible scenarios and reduce that to actions that improves their profitability.

Such mastery is not just a plain 80% win rate or other simple measurement can do to discover or qualify a method.

80% win rate is an overall measurement. It does not take into account the separate performance of a correctly identified setup and a falsely identified one. Often traders cannot even distinguish the two scenarios for trading setups they use all the time.

For example, a 1-2-3 sell setup can fail and results in a bull flag upside breakout pattern. 1-2-3 sell setup works best if the trader can identify the strong resistance area to key off the setup.

A good combination can be 90% of the time you identify the trading setup at a good resistance zone where 90% of those identified setups result in profits, and that in the 10% of falsely identified setups, you still edge out 50% winners. The reason why this is a good combination is that your expected performance will likely to have good consistency.

A bad combination could be 50% of the time you identify the trading setup correctly and 90% profitable on those correctly identified ones. And then in the other 50% of trading setups, 70% of the time you edge out a profit. In this case, even though the historical performance is around 80% win rate, you do not really know why your setups are producing profit. In fact, that 70% winners out of the falsely identified setups could be just random outcomes. Worst yet, traders often try to improve the win rate by adjusting the stop loss which simply curve fit the trading setup to perform better on historical data. Those extra winners in the particular 70% is likely a result of that.

The rare case independency issue – Traders often make the mistake in thinking that since a (huge) losing trade based on a particular setup is very rare, why not double down, triple down, or bet the farm on the next occurrence of the trading setup. People justifying such measures are likely in a losing streak. In such stressed situations, bad ideas often pop up and this is one of them.

Even if the huge losses are rare and likely independent, that does not change the potential of the very next trade being a loser. Betting the farm or sudden increase in bet size in general will significantly affect the expectancies with your trading method. If it is not something you have anticipated and well planned out, it is just plain stupid doing that knowing the consequence can wipe out your trading account.

Professionals in various fields often misuse statistics – In the video Peter Donnelly pointed out this very important issue that infested our society.

Doctors, lawyers, economists, security analysts, etc. are not statisticians. Yet these professionals frequently try to draw conclusions and inference from data they have compiled. Often these inference are incorrect and misleading. Unluckily there is no rule or regulation to, say, revoke their professional licenses for misuse of statistics. At the receiving end of these analysis and conclusions, it is difficult for the public to protect themselves from these misleading information.

[DTBpromo]