Ed Thorp on Statistical Arbitrage, from Wilmott.

[chin]

Facinating 3-part articles by Ed Thorp. A window to the quant trading world. Some quick numbers & observations:

1. at 26% annual return (on capital) & 25x annual turnover, his average profit margin was 1% on volumn after cost (see 3 below.) Low margin but high absolute dollar return. Kind of highly leveraged, not in the sense of borrowing, but in terms of sensitivity to modelling error.

2. position limits on the long side 2.5% and short side 1.5% of capital. Given his quoted figures as example of intended exposure, the average risk taken per position was 1% of capital, much smaller % per trade on average. (This remains me again of my friend's "algo trading fund" where they would risk up to 30-40% per position. Now that fund is gone.) It just hammer in the point - when you have a positive expectation model, bet small and bet very frequently.

3. average all-in transaction cost of $0.0074 on average share price of $36, the average transaction cost was 0.02%! Retail investors pay perhaps 0.25%, and people betting on horses pay 20% without rebate!

4. when in doubt, close position.

5. his fund represent 0.5% of NYSE total turnover in 2000, compare this to Gene's market making strategy that trades 4% recently!

6. 2/3 trades gone through one broker?! how does he stop brokers from following/frontrunning his trades? Maybe "broker" means the exchange platform, instead of an agent.

[chin]

A somewhat related reading - "Fortune's Formula: The Untold Story of the Scientific Betting System That Beat the Casinos and Wall Street" by William Poundstone.

Fortune's Formula was mainly about Shannon's Information Theory and, most interesting to me, the Kelly Criterion. Ed Thorp has been a long time advocate of the Kelly Criterion. The book also has a section on Ed Thorp's earlier stat arb ventures, and his trouble with Jay Regan in the Princeton office.

One of the many memoriable stories from this book - when Thorp object to one guy's investment strategy, his comment was that that guy was a "Martingale Man", as sort of oppose to a "Kelly Man".

[chin]

Ed Thorp has three more followup stories on his Stat Arb venture. The articles were found on his own site, and are in Word format. Once I remember by Wilmott login, I will get the better looking pdf files.

His web site is www.edwardothorp.com

[chin]

I started reading Ed Thorp's writing because of his frequent discussions of the Kelly Criterion.

Supposedly Ed Thorp popularized the use of Kelly Criterion in his Beat the Dealer book (which I have not got the chance to read.) He has since been a strong advocate on the use of Kelly Criterion in investment & gambling.

The interest in Kelly Criterion probably has been strong as ever in the age of hedge funds, quants, and the use of heavy duty math modeling in investment and gambling. In many discussions, the most popular objections to the use of Kelly Criterion are

- the difficulties to pinpoint the probability accurately in order to bet Kelly. The results from betting Kelly is very sensitive to the accuracy of the expected value, which in term depends on accurate assessment of both probability & payoff. (I once had a discussion with the author of 計得精彩 about the use of the complete formula vs the approximation form. He was very insistence that it only works with the complete formula, and published in his blog the simple simulation I sent him. I only realized after a few weeks that the approximation works equally well in real world since we will never have completely accurate probability for real world events.)

- wild rides. Kelly betting tend to produce wild swings in total asset value, even though the probability for high asset is higher in the long term. Not everyone can accept or stomach the short term wild rides. The solution is really to use small fractional Kelly, and only use it in situation where there are plenty of positive expected return opportunities. Again, bet small but bet many independent events.

The attached files are:
- the original 1956 John Kelly paper discussing the algorithm;
- Ed Thorp's two recent discussions on the algorithm, and particularly the logic behind fractional Kelly;
- Ed Thorp's reflections on building models - from Blackjack to Convertible Bond to Stat Arb.

[kido]

I started reading Ed Thorp's writing because of his frequent discussions of the Kelly Criterion.

Supposedly Ed Thorp popularized the use of Kelly Criterion in his Beat the Dealer book (which I have not got the chance to read.) He has since been a strong advocate on the use of Kelly Criterion in investment & gambling.

The interest in Kelly Criterion probably has been strong as ever in the age of hedge funds, quants, and the use of heavy duty math modeling in investment and gambling. In many discussions, the most popular objections to the use of Kelly Criterion are

- the difficulties to pinpoint the probability accurately in order to bet Kelly. The results from betting Kelly is very sensitive to the accuracy of the expected value, which in term depends on accurate assessment of both probability & payoff. (I once had a discussion with the author of 計得精彩 about the use of the complete formula vs the approximation form. He was very insistence that it only works with the complete formula, and published in his blog the simple simulation I sent him. I only realized after a few weeks that the approximation works equally well in real world since we will never have completely accurate probability for real world events.)

- wild rides. Kelly betting tend to produce wild swings in total asset value, even though the probability for high asset is higher in the long term. Not everyone can accept or stomach the short term wild rides. The solution is really to use small fractional Kelly, and only use it in situation where there are plenty of positive expected return opportunities. Again, bet small but bet many independent events.

The attached files are:
- the original 1956 John Kelly paper discussing the algorithm;
- Ed Thorp's two recent discussions on the algorithm, and particularly the logic behind fractional Kelly;
- Ed Thorp's reflections on building models - from Blackjack to Convertible Bond to Stat Arb.

These few weeks my self-learning on horse betting just started, terms like these are frequently heard: like Kelly's Criterion Fortune's formula, exponential distribution, etc. Also some psychological term like favourite longshot bias... Just coincidence or whatever, I heard the book 計得精彩 and is looking for it, (heard that the author will publish pdf verison 'sometime' later) Comments are good from all over the net (at least I saw none are bad). It might be good for me to brush up the learning by that book.

[chin]

These few weeks my self-learning on horse betting just started, terms like these are frequently heard: like Kelly's Criterion Fortune's formula, exponential distribution, etc. Also some psychological term like favourite longshot bias... Just coincidence or whatever, I heard the book 計得精彩 and is looking for it, (heard that the author will publish pdf verison 'sometime' later) Comments are good from all over the net (at least I saw none are bad). It might be good for me to brush up the learning by that book.

I have Fortune's Formula & 計得精彩. You can borrow the books if you want.

The first one is an easy light reading for fun. The actual formula is the above attached.

The second book is good only in the sense that it's the only Chinese text on serious math of racing. For you who can comfortably read English, you should just go for Efficiency of Racetrack Betting Markets. Favorite Long-shot Bias, etc... are also well discussed in the Efficiency.

[chin]

I just finished reading "When Genius Failed - the Rise and Fall of Long-term Capital Management" by Roger Lowenstein.

Assuming Lowenstein's descriptions are accurate, what a contrast in style from Ed Thorp's as described in the above attachments!

In the Genius book, the LTCM guys, especially the main trader Hilibrand, can be almost say to be a "Martingale Man" - he would redouble in the face of losses. And in many occasion, Long-term was say to be "betting the farm". This is a sharp contrast to Kelly Criterion based concept to scale back in the face of losses & diminishing edge. In the Ed Thorp articles, there were a number of occasions where Ed Thorp either turned down addition capital or reduce the capital, in order to maintain the edge.

In the Genius book, the LTCM guys repeatedly characterize their models as picking up nickels here and there (and later "picking up nickels in front of the bulldozer" when they got into merger arb.) From the description in the book, they were picking up nickels mainly in the sense that they were pickup the minor inefficiencies overlooked by others, but not really "here and there" in the sense of large number of mostly unrelated trades. So they were hoping to found a very large pile of nickels and sent in a large vaccum. Their vaccum was leverage - around 20 times in 'normal' time and up to 20 times right before the trouble. At that leverage, 3% drop in asset value would (and did) whip out the entire capital.

The Ed Thorp stat arb described in the above articles, seem to me more truely a "nickel sucking here & there" operation. See my comment 2 in the 1st post.

In the Genius book, Goldman was said to be transformed from a gentlemen investment banker to a predatory trader. It front-ran the LTCM book while discussing a rescue and "raped" LTCM, according to the LTCM partners. While reading this part, I cannot help but to compare the now famous Goldman Sach high-frequency trading - perhaps by now the best known giant "nickel sucking" operation. From the public information, this strategy is compared to front running.

If front-running is so common-place & institutionalized, I am even more curious how Ed Thorp solve this problem (see 6 in my first post.)

BTW my comment regarding Long-term is perfectly aided by hind-sight. I am sure it was not entirely fair comment if made at the time of the event. But that's the fun of commenting on other's misfortune, in hinh-sight. Right?

(For my friends who are not entirely familiar with American English, nickel = US 5 cents coin, penny = US 1 cent coint, dime = 10 cents.)

[q]

I think one of the big problems Hedge Funds have is holding onto talent, and hence the proprietary trading techniques.
Especially when most of the techniques applied for arbitrage and algorithmic trading are relatively simple (not a criticism
since we do the same).  So, the only thing competitors are missing is the twist.

[kido]

I think any 'betting' strategies will result in a stochastic wealth behaviour, given that these bets are probabilistic.  So, trying to devise any strategies/algorithms will only results in a wild ride, the only hope is that it gives you returns before the wealth diminished.

Mind my ignorance here.  I'm not really in that industry (I think it's investment or wealth management or whatever you called) so please excuse me. Would you mind telling me what's a 'Martingale', how's it opposing to a Kelly's bet? I guess a 'Martingale' is a random bet?

Yes, I want 計得精彩 desperately, Fortune's formula would be bonus.  Thank you.

[chin]

I think any 'betting' strategies will result in a stochastic wealth behaviour, given that these bets are probabilistic.  So, trying to devise any strategies/algorithms will only results in a wild ride, the only hope is that it gives you returns before the wealth diminished.

Mind my ignorance here.  I'm not really in that industry (I think it's investment or wealth management or whatever you called) so please excuse me. Would you mind telling me what's a 'Martingale', how's it opposing to a Kelly's bet? I guess a 'Martingale' is a random bet?

Yes, I want 計得精彩 desperately, Fortune's formula would be bonus.  Thank you.

Google or Wiki the term.

When a betting system ask you to increase the bet amount solely on the condition of lossing the last bet, and/or decrease to a smaller amount after a winning bet, this system is generally said to be "Martingale". And most of the Martingale systems are applied to independent event, which make it more ridiculous.

Call me to arrange a time to pick up the books.