Ben power speaks with Henry Carstens about how you can really achieve an edge through your trading system.

At some stage in his or her career, every trader will be hit by the uncomfortable questions:  ‘Am I winging it? Do I really have an edge?’ The answer is, of course, to begin testing your system or method. But for a discretionary trader, many of whom use charts and intuition, learning how to test existing and fresh ideas can be confusing.

Henry Carstens is one person who is helping demystify the complex world of trading system testing and development. Carstens, who is based in Portland, Oregon, develops and trades systems through his company, Vertical Solutions.
He has recently begun trading other people’s money.

A protégé of trading legend Victor Niederhoffer, Carstens is developing a growing reputation as an insightful teacher of systems development and testing. His ‘Introduction to Testing Trading Ideas’, available at www.verticalsolutions.com, is recommended by trading coach Dr Brett Steenbarger.

Carstens spoke with YTE’s Ben Power about how traders can boost their trading through systems testing and development. He has helpfully provided a number of links to papers available on his site, which readers can follow up.

How did you get into trading?

I bought my first stock, Anglo Lautro, when I was 12. I bought it at 1.125, the high was nine and it was eventually nationalised by the Chilean government for no gain.

Later, I noticed that on some days the market really seemed to take off and run in one direction and I figured out how to predict when that would happen. I'd watch the market in the morning before going to work, place my trades and head off to work.

How did you get into the development of trading systems?

I found a book called ‘Trading Systems Toolkit’, by Joe Krutsinger. I got a copy of TradeStation trading software and coded up my idea – an idea I use to this day. The core of that idea is in the Trend Day or Range Day forecasts on my website at www.verticalsolutions.com/forecasts.html.

After that I worked a bit for Dr Victor Niederhoffer. I have been on my own since then. It was for Dr Niederhoffer that I built a huge array of systems based on machine learning – all of which ended up being spectacularly curve fitted. It was a great, if painful, lesson.

What was it like working for Niederhoffer?

Absolutely fascinating! His ability to generate simple, insightful ideas and to test them is unparalleled. Everybody should read his book, ‘Education of a Speculator’.

I assume that developing a trading system begins with an idea or a hypothesis about a potential edge. How do you generate ideas?

Three ways: I watch the market, I play with the data, and I try to cross-pollinate ideas from outside the world of trading into trading – for example, chess has the concept of critical points, a place where the game turns; a similar idea in trading might be support and resistance, or it might be the line where the day changes from being a range day into a trend day.

As an example, you can access ten ideas based on an article about a basketball player at www.verticalsolutions.com/notes/battier_insights.html. Still, when it comes to idea generation, I am nothing compared to Dr Niederhoffer.

Once you’ve got an edge, I assume you test it? How do you got about that? What numbers would confirm a solid edge?

I use two numbers: ‘t score’ (or t-Test) and ‘optimal f’. The ‘t score’ tells me how likely it is that the results I'm seeing are statistically significant (or how likely the test results are to have occurred by chance alone); and ‘optimal f’ tells me how much the edge can be leveraged without the risk of ruin.

So having found an edge, you verify that it is 'real' using ‘t score’, and then ‘optimal f’ tells you how tradeable it is. Generally, the more leverage that can be applied to the system the better the system is (regardless of whether that leverage is ever applied).

You can find a pretty good example of developing an edge at www.verticalsolutions. com/notes/simple_tools.html, and a tool for evaluating it as a potential trading system at www.verticalsolutions. com/tools/system_ranking.html.

What kind of investment would it require for a discretionary trader to learn software like TradeStation?

I don't know. It's such a continuous process. Maybe not so much, to use it as a very simple tool. A bit more, to use it to  create new tools.

One of the most important things TradeStation does for me is to manage historical data. That saves huge amounts of time. The second thing it does is to provide a way to run tests against that historical data to determine if an edge or idea is real and tradeable. For the discretionary trader, knowing an edge is real is almost an edge on top of an edge.

Personally, I have a mental block when it comes to statistics and hard maths. Do you need to be good at maths for systems development?

Not just for trading systems development, but to test discretionary ideas and set-ups too! You need some math because that's the language you're going to use to understand and contextualise results. The better you understand the results the more successful you'll be. It's like the Eskimos having 42 words for snow -– math helps you understand the results of strategy testing.

An example: the t score is a very simple calculation (square root of number of observations/standard deviation x average trade), but it tells you something that is of fundamental importance: are the results I'm seeing real, or did they occur by chance alone? A little bit of math provides a lot of insight.

Simplicity works best in trading systems; hard math isn't necessary. Insight, creativity and hard work are all more important than complexity.

What are some of the major pitfalls of systems testing? Is there a danger, for example, that you twist the numbers to suit your hypothesis?

Yes, curve-fitting is the great danger and the odds of curve-fitting tend to increase at least linearly with each variable added to a model. This is because, when relationships in the market change, the odds that the change will affect your system are higher because you have more variables to be affected.

Another pitfall is related to the ability to anticipate future changes in the market from historical data. Calculating and building tolerances for systems can help with this.

Perhaps the biggest pitfall is thinking that any single trading system is going to perform flawlessly. All trading systems ebb and flow as the market changes and cycles, so having multiple systems to counteract the ebb and flow is very important.

What are the benefits of automated trading systems versus discretionary trading? Can a trader use both?

Automated trading systems help organise multiple edges. An automated system never forgets and is never afraid. Still, they are rough tools. A generalised system never gets the best entry or the best exit, so there is always room for improvement. A nice combination of automated and discretionary trading might be to let an automated system fill the holes in a repertoire; i.e., the discretionary trader handles range days and the automated systems handle trend days.

Say you have found an edge and verified it through testing. How would you then trade the system? Would that be automated too?

Yes, although it's not necessary to automate it. You could have it just generate signals which you followed based on current market conditions. The advantage for the discretionary trader is there is an edge and then they can choose when to exploit it.

I have an idea that discretion may be most useful over shorter time frames (minutes and hours, but not seconds) because as the profits per trade shrink the importance of maximising entries and exits increases.

I assume changes in the market are a challenge to systems trading. How quickly does it take to lose an edge in the market? Does it mean you need to constantly build new systems?

If you build a highly curve-fit system you can lose your edge before you ever get started. The changes in the market are, to my eye, more of an ebb and flow. Diversification of the systems working within a market helps offset the ebb and flow.

I end up building new systems more because I have new ideas than because of market changes... hmmm... but maybe those new ideas are being generated by market changes... It's all incredibly circular.

In general, I think our edge, as system developers or discretionary traders, is much smaller than we tend to think.

Finally, how do you recommend the discretionary trader who wants to begin exploring automated trading go about it? Are there any courses or Internet resources that might help them along?

I wrote a paper called ‘Introduction to Testing Trading Ideas’ that will help. It can be accessed at www.verticalsolutions.com/notes/intro_to_testing.html.

Ben Power is a writer, journalist and trader. He has written on politics, economics and finance for numerous publications.

This article was originally published in the Jan/Feb 2010 issue of YourTradingEdge magazine. All rights reserved. © Copyright 2011, Your Media Edge Pty Ltd. 
To subscribe to YourTradingEdge magazine click here, or to purchase this issue as a single back issue, click here.