Author Topic: Computer Robotic Wagering (CRW)  (Read 88180 times)

Offline chin

  • Global Moderator
  • *****
  • Posts: 6,663
Computer Robotic Wagering (CRW)
« on: 25 July 2009, 02:08:32 »
The term CRW was invented in the US horse racing market, describing people who use software and sophisticated algorithm to scan the pools and odds to find mispriced bets.

Less sophisticated gamblers thought CRW teams have access to odds in the normally "blind" pools like Trifecta & Superfecta, where the odds are not displayed. They often cry foul and accuse the CRW teams of "seeing other people's hand" before betting.

In reality, scan is the wrong word, because the software does not really "scan" because the pools are blind. Project and bet against the projections is more accurate description of CRW teams.

And then I found this article today. The big boys DO scan and cherry pick, perhaps with some projections to boot.

AND we are not talking about betting horses, but the largest gambling pool in the world - the stock markets.


Offline chin

  • Global Moderator
  • *****
  • Posts: 6,663
Re: Computer Robotic Wagering (CRW)
« Reply #1 on: 25 July 2009, 02:50:41 »
Before quoting the New York Times article on the stock market equivalent of CRW, I would like to relay a personal experience.

In around April 09, we received a visitor G*** who is the head of a specialized proprietary trading unit in a very large & well known investment bank. My friend brought him to our office to see our program trading or CRW. In those few hours, I found out that:

- His unit trades the bank's own capital.
- His algorithm can project share prices a few hours ahead.
- His unit made "large number of small bets" in his own words.
- His unit issue 4 or 5 million orders per day, but only a few hundred thousand orders would be matched!
- His unit accounted for 4-5% of all trading in the Japanese market!
- They are in other Asian markets as well. Tradings are all electronic, of course.
- My friend told me G's unit is among the most profitable units in his bank.


The above sounds very much like a CRW to me.

Then I see this NYT story.

Offline chin

  • Global Moderator
  • *****
  • Posts: 6,663
Re: Computer Robotic Wagering (CRW)
« Reply #2 on: 25 July 2009, 02:52:03 »
Compare the highlighted to my points above.

The strategy described in this NYT article is very similar to front running. The only difference is that Front Running is when a stock broker is cheating his own client, while this strategy is cheating(?!) everyone.

---
http://www.nytimes.com/2009/07/24/business/24trading.html?_r=1&ref=global-home

Quote
Stock Traders Find Speed Pays, in Milliseconds
By CHARLES DUHIGG
Published: July 23, 2009

It is the hot new thing on Wall Street, a way for a handful of traders to master the stock market, peek at investors’ orders and, critics say, even subtly manipulate share prices.

It is called high-frequency trading — and it is suddenly one of the most talked-about and mysterious forces in the markets.

Powerful computers, some housed right next to the machines that drive marketplaces like the New York Stock Exchange, enable high-frequency traders to transmit millions of orders at lightning speed and, their detractors contend, reap billions at everyone else’s expense.

These systems are so fast they can outsmart or outrun other investors, humans and computers alike. And after growing in the shadows for years, they are generating lots of talk.



Nearly everyone on Wall Street is wondering how hedge funds and large banks like Goldman Sachs are making so much money so soon after the financial system nearly collapsed. High-frequency trading is one answer.

And when a former Goldman Sachs programmer was accused this month of stealing secret computer codes — software that a federal prosecutor said could “manipulate markets in unfair ways” — it only added to the mystery. Goldman acknowledges that it profits from high-frequency trading, but disputes that it has an unfair advantage.

Yet high-frequency specialists clearly have an edge over typical traders, let alone ordinary investors. The Securities and Exchange Commission says it is examining certain aspects of the strategy.

“This is where all the money is getting made,” said William H. Donaldson, former chairman and chief executive of the New York Stock Exchange and today an adviser to a big hedge fund. “If an individual investor doesn’t have the means to keep up, they’re at a huge disadvantage.”

For most of Wall Street’s history, stock trading was fairly straightforward: buyers and sellers gathered on exchange floors and dickered until they struck a deal. Then, in 1998, the Securities and Exchange Commission authorized electronic exchanges to compete with marketplaces like the New York Stock Exchange. The intent was to open markets to anyone with a desktop computer and a fresh idea.

But as new marketplaces have emerged, PCs have been unable to compete with Wall Street’s computers. Powerful algorithms — “algos,” in industry parlance — execute millions of orders a second and scan dozens of public and private marketplaces simultaneously. They can spot trends before other investors can blink, changing orders and strategies within milliseconds.

High-frequency traders often confound other investors by issuing and then canceling orders almost simultaneously. Loopholes in market rules give high-speed investors an early glance at how others are trading. And their computers can essentially bully slower investors into giving up profits — and then disappear before anyone even knows theywere there.

High-frequency traders also benefit from competition among the various exchanges, which pay small fees that are often collected by the biggest and most active traders — typically a quarter of a cent per share to whoever arrives first. Those small payments, spread over millions of shares, help high-speed investors profit simply by trading enormous numbers of shares, even if they buy or sell at a modest loss.

“It’s become a technological arms race, and what separates winners and losers is how fast they can move,” said Joseph M. Mecane of NYSE Euronext, which operates the New York Stock Exchange. “Markets need liquidity, and high-frequency traders provide opportunities for other investors to buy and sell.”

The rise of high-frequency trading helps explain why activity on the nation’s stock exchanges has exploded. Average daily volume has soared by 164 percent since 2005, according to data from NYSE. Although precise figures are elusive, stock exchanges say that a handful of high-frequency traders now account for a more than half of all trades. To understand this high-speed world, consider what happened when slow-moving traders went up against high-frequency robots earlier this month, and ended up handing spoils to lightning-fast computers.

It was July 15, and Intel, the computer chip giant, had reporting robust earnings the night before. Some investors, smelling opportunity, set out to buy shares in the semiconductor company Broadcom. (Their activities were described by an investor at a major Wall Street firm who spoke on the condition of anonymity to protect his job.) The slower traders faced a quandary: If they sought to buy a large number of shares at once, they would tip their hand and risk driving up Broadcom’s price. So, as is often the case on Wall Street, they divided their orders into dozens of small batches, hoping to cover their tracks. One second after the market opened, shares of Broadcom started changing hands at $26.20.

The slower traders began issuing buy orders. But rather than being shown to all potential sellers at the same time, some of those orders were most likely routed to a collection of high-frequency traders for just 30 milliseconds — 0.03 seconds — in what are known as flash orders. While markets are supposed to ensure transparency by showing orders to everyone simultaneously, a loophole in regulations allows marketplaces like Nasdaq to show traders some orders ahead of everyone else in exchange for a fee.

In less than half a second, high-frequency traders gained a valuable insight: the hunger for Broadcom was growing. Their computers began buying up Broadcom shares and then reselling them to the slower investors at higher prices. The overall price of Broadcom began to rise.

Soon, thousands of orders began flooding the markets as high-frequency software went into high gear. Automatic programs began issuing and canceling tiny orders within milliseconds to determine how much the slower traders were willing to pay. The high-frequency computers quickly determined that some investors’ upper limit was $26.40. The price shot to $26.39, and high-frequency programs began offering to sell hundreds of thousands of shares.

The result is that the slower-moving investors paid $1.4 million for about 56,000 shares, or $7,800 more than if they had been able to move as quickly as the high-frequency traders.

Multiply such trades across thousands of stocks a day, and the profits are substantial. High-frequency traders generated about $21 billion in profits last year, the Tabb Group, a research firm, estimates.

“You want to encourage innovation, and you want to reward companies that have invested in technology and ideas that make the markets more efficient,” said Andrew M. Brooks, head of United States equity trading at T. Rowe Price, a mutual fund and investment company that often competes with and uses high-frequency techniques. “But we’re moving toward a two-tiered marketplace of the high-frequency arbitrage guys, and everyone else. People want to know they have a legitimate shot at getting a fair deal. Otherwise, the markets lose their integrity.”

Offline chin

  • Global Moderator
  • *****
  • Posts: 6,663
Re: Computer Robotic Wagering (CRW)
« Reply #3 on: 25 July 2009, 03:14:40 »
The 6th paragraph in the above article refers to the 7-Jul news below. It sounded to me started as employment dispute.

Apparently Goldman's profit making algorithm is almost raised to the level of US national security matters, while the morality of this practice may still in question. Not only Goldman was too big to fail, it has to be extremely profitable too. 官商勾結 is only natural.


http://www.nytimes.com/2009/07/07/business/07goldman.html?scp=3&sq=Sergey%20Aleynikov&st=cse

Quote
Ex-Worker Said to Steal Goldman Code

By GRAHAM BOWLEY
Published: July 6, 2009

He is no John Dillinger, no public enemy No. 1. But Sergey Aleynikov nonetheless masterminded a dazzling bank theft, the authorities say, and he did it without brandishing a gun or cracking a vault.

Instead, he cracked — or, rather, hacked — the secrets of Goldman Sachs, according to federal agents.

Until a few weeks ago, Mr. Aleynikov, 39, was a computer programmer at Goldman, whose prowess in trading has long made it the envy of Wall Street.

But over five days in early June, the authorities say, he stole proprietary, “black box” computer programs that Goldman uses to make lucrative, rapid-fire trades in the financial markets. Their value, experts say, could be incalculable.

Mr. Aleynikov, however, will not get a chance to use those secrets. He was arrested by federal agents on Friday evening, as he got off a plane at Newark Liberty International Airport. He has pleaded not guilty to charges of theft of trade secrets and transporting them abroad.

The case, as detailed in a federal complaint filed in court in the Southern District of New York, throws a spotlight on the multimillion-dollar technology that is increasingly employed by the world’s biggest banks to gain an edge in financial markets.

Goldman divulged little about the trading programs on Monday, though court documents related to the case said the code that Mr. Aleynikov was suspected of stealing allowed the bank to “engage in sophisticated high-speed and high-volume trades on various stock and commodities markets.”

The software generated “many millions of dollars of profits per year” for the bank, the documents said.

Mr. Aleynikov joined Goldman in May 2007 and was a vice president for equity strategy, but announced his resignation after little more than two years.

He was, he told Goldman, joining a new trading company, which various news reports said was in Chicago. He said he would earn triple the $400,000 salary he commanded at Goldman.

But, just before he left, according to the complaint, Mr. Aleynikov used his desktop computer at Goldman’s New York offices to upload a stream of code to a Web site hosted by a server based in Germany.

Later, he downloaded the files again to his home computer, his laptop computer and to a memory device.

He was caught when the bank noticed the surge of data leaving its servers — and despite his prowess as a highly paid programmer, his activities were recorded even though he tried to erase his programming commands because Goldman kept back-up records.

When confronted by federal investigators at the airport in Newark, Mr. Aleynikov, a naturalized American citizen who immigrated from Russia and now lives in New Jersey, insisted that he had intended to collect “open source” files on which he had worked and only later realized he had copied more files than he had intended.

On Monday, Goldman Sachs refused to comment publicly on the attempted theft. A person familiar with the bank said it had since “secured its systems.” This person, who asked not to be identified, given the confidential nature of the programs, insisted that the theft had had no effect on Goldman Sachs’s business or on that of its clients.

However, at a court appearance in Manhattan on July 4, Joseph Facciponti, the assistant United States attorney, told a federal judge that Mr. Aleynikov’s supposed theft posed a risk to United States financial markets and that other people may have had access to it, according to Bloomberg.

“The bank has raised the possibility that there is a danger that somebody who knew how to use this program could use it to manipulate markets in unfair ways,” Mr. Facciponti said in the court, according to Bloomberg. “The copy in Germany is still out there, and we at this time do not know who else has access to it.”

Bruce Schneier, the chief security technology officer for British Telecom and an expert on computer security, said this type of corporate crime — of a former employee leaving a company with data he should not have — occurred quite regularly. But he agreed that Goldman’s systems had worked well in stopping Mr. Aleynikov.

“This is an example of a system of detection and response working,” he said.

But computer experts expressed caution on the value of the code outside the bank.

Peter Niculescu, a partner at Capital Market Risk Advisors, an advisory firm specializing in risk management and capital markets, said computerized trading had become increasingly important drivers of revenue growth within banks over the last 10 years.

But he said stealing a bank’s trading code did not necessarily guarantee riches, because running it somewhere else was not easy without, for example, a bank’s databases or links to customers.

“If you have the code, but not the database then it is of limited value,” he said. “It is not easy to transfer the code and run it somewhere else.”

Mr. Schneier said, “It is certainly possible that if you knew what the big guys were doing you could anticipate it and make money.” He said that if a rival bank in the United States had been approached to buy the software, it would most likely have called the police, but a seller might have had better luck abroad.

“It is worth a lot less in the U.S. than you might think, but in countries that are more lawless it could have value,” he said.
« Last Edit: 25 July 2009, 16:47:54 by chin »

Offline chin

  • Global Moderator
  • *****
  • Posts: 6,663
Re: Computer Robotic Wagering (CRW)
« Reply #4 on: 26 July 2009, 23:32:28 »
CRW (see 1st message above), high-frequency flash trading (see 3rd message), Statistical Arbitrage (link here) - all have one thing in common.

When you have a statistically sound long term edge, bet small but bet a lot, the margin is low but return is very high.

Offline kido

  • SBC Old Boys
  • ***
  • Posts: 310
Re: Computer Robotic Wagering (CRW)
« Reply #5 on: 21 August 2009, 11:57:25 »
CRW sounds interesting.  But I'm wondering whether these opportunities really exists. The racing company will set the pools such that no one can make a fortune out of it.

--
cmchan
Hey, diddle, diddle ! The cat and the fiddle.

Offline chin

  • Global Moderator
  • *****
  • Posts: 6,663
Re: Computer Robotic Wagering (CRW)
« Reply #6 on: 21 August 2009, 12:09:34 »
CRW sounds interesting.  But I'm wondering whether these opportunities really exists. The racing company will set the pools such that no one can make a fortune out of it.

--
cmchan

CRW only works in pari mutuel pools, like HK Jockey Club, US, Japan, etc.... where you are betting against other gamblers, not the Jockey Club.

CRW does not work in place where bets are placed with book maker, like UK, because you are betting against the bookie.

Offline kido

  • SBC Old Boys
  • ***
  • Posts: 310
Re: Computer Robotic Wagering (CRW)
« Reply #7 on: 21 August 2009, 15:35:03 »
After some googling, I finally understand what you meant. 

These are quite some terms that one needs to be really 'in' this field.  I can see that there might be some cases where you gets bits of profits from the 'rounding' of odds, or from the rules that governs the min. payout.
Hey, diddle, diddle ! The cat and the fiddle.

Offline chin

  • Global Moderator
  • *****
  • Posts: 6,663
Re: Computer Robotic Wagering (CRW)
« Reply #8 on: 21 August 2009, 15:58:20 »
After some googling, I finally understand what you meant.  

These are quite some terms that one needs to be really 'in' this field.  I can see that there might be some cases where you gets bits of profits from the 'rounding' of odds, or from the rules that governs the min. payout.

There is no official definition of CRW so I don't know what can be googled. But what I had in mind in this thread is much more than profit from rounding errors. It's generally algorithms, usually very mathmetically sophisticated constructions, to arbitrage mispricing against a more accurate prediction (of price/probability/event/whatever).

People have been doing arbitraging for ages, what's new now is the application of computers to scan the real or projected mispricing in a very fast and efficient (and by some preceived as unfair) manner.

The examples I cited in this thread - CRW in racing, Stat Arbitrage & now High Frequency Trading, are all sort of CRW in my opinion. That computers running sophisticated algorithms scan the markets for mispricings. And one shared strong characteristic is the "bet small but bet many" concept.

Offline chin

  • Global Moderator
  • *****
  • Posts: 6,663
Re: Computer Robotic Wagering (CRW)
« Reply #9 on: 16 October 2009, 02:11:19 »
This subject continued to interest me in the last few months, and after reading more books, I realized that this is an entirely new area in finance that I have never heard of - market microstructure. Some of the studies in this area focus on liquidity discovery through advance mathematics, clever algorithms & superfast computers.

In one of the books I read, the author claimed that now about 50% of the trading volume of NYSE listed shares are done by about 10 firms whose name most people won't recognize & the trades were made by computer algorithms instead of human instructions.

One of these little known automated trading firms is ATD - found by a finance professor David Whitcomb. In an 2002 testimony to SEC, ATD was said to be trading 80 million shares in 300,000 trades per day (or at 267 shares per trade, that's LOTS OF SMALL ORDERS!) According to ATD web site, in 2006, they traded 6% of NYSE & NASDAQ volume. By 2007 they routinely traded 300 million shares per day! (There is no update on these figures after Mar 2007.)

The following article in today's New York Times tells the story that majority of tradings of shares listed in NYSE are not longer traded in NYSE. Instead they are traded on ECNs - private computer networks for trading.

After reading more about high speed automatic trading on these "dark pools", I realized that I may have judge too soon to said the traders are front-running. However, I am still highly suspicious of firms who provide the trading platform, provide the liquidity, AND ALSO trading on house accounts. Maybe for their trading customers who trade large blocks, the liquidity & minimized market impact out weights the harm from potential conflict of interest?!


http://www.nytimes.com/2009/10/15/business/15exchange.html?ref=global

Quote
Rivals Pose Threat to New York Stock Exchange
By GRAHAM BOWLEY
Published: October 14, 2009

For most of the 217 years since its founding under a buttonwood tree on Wall Street, the New York Stock Exchange was the high temple of American capitalism.

Behind its Greco-Roman facade, traders raised a Dante-esque din in their pursuit of the almighty dollar. Good times or bad, the daily melee on the cavernous trading floor made the Big Board the greatest marketplace for stocks in the world.

But now, even as the Dow Jones industrial average topped 10,000 for the first time since the financial crisis sent it tumbling, the exchange and its hometown face an unsettling truth: the Big Board, the symbolic heart of New York’s financial industry, is getting smaller.

Young, fast-moving rivals are splintering its public marketplace and creating private markets that, their critics say, give big banks and investment funds an edge over ordinary investors.

Some of the new trading venues — “dark pools,” the industry calls them — are all but invisible, even to regulators. These stealth markets enable sophisticated traders to buy and sell large blocks of stock in secrecy at lightning speed, a practice that has drawn scrutiny from the Securities and Exchange Commission.

These upstarts are utterly unlike the old-school Big Board, which is struggling to make its way as a for-profit corporation after centuries of ownership by its seat-holding members. Last year, its parent company, NYSE Euronext, lost $740 million.

Wall Street’s judgment has been swift and brutal. Since January 2007, the share price of NYSE Euronext has lost nearly three-quarters of its value, even though stock trading over all has soared.

While the exchange has been under assault since the beginning of the decade, its decline has accelerated in recent years as aggressive competitors have emerged. Today, 36 percent of daily trades in stocks that are listed on the New York Stock Exchange are actually executed on the exchange, down from about 75 percent nearly four years ago. The rest of are conducted elsewhere, on new electronic exchanges or through dark pools.

The old Big Board was far from perfect. Its floor brokers — who occupy a privileged, and potentially lucrative, niche between buyers and sellers — have sometimes enriched themselves at their customers’ expense.

But changes inside the exchange’s grand Main Hall are startling. For decades, the New York Exchange was the kind of place where sons followed their fathers onto the trading floor. But half of the jobs there have disappeared over the last five years. Many of the 1,200 or so remaining workers retreat quietly to their computers shortly after the opening bell clangs at 9:30 a.m.

The Big Board has been forced to close one of its five trading halls, and it has repopulated two others with business from the American Stock Exchange, which NYSE Euronext bought last year. The Main Hall — the soaring, gilded room opened in 1903 — can seem little more than a colorful backdrop for CNBC.

“It has not been pretty,” said Benn Steil of the Council on Foreign Relations in New York. “All the big established exchanges around the world have experienced the same phenomenon, but the New York Stock Exchange has taken the biggest beating.”

It is a remarkable comedown for the New York Exchange, and for New York. Once the undisputed capital of capital, the city is struggling to retain its dominance in finance as the industry globalizes. “Wall Street” seems to be no longer a place, but a vast, worldwide network of money and information.

The Big Board says that it is fighting back — and that its hybrid of computers and human traders can beat the new rivals. It slashed commissions and developed its own purely electronic exchange, Arca, in Chicago. Arca has captured about 11 percent of the market for Big Board-listed stocks. It is also winning business in areas like derivatives.

“What’s going on here is a reinvention,” said Lawrence Leibowitz, head of United States markets and global technology at NYSE Euronext. “How can you bring this institution forward into the 21st century?”

Proponents of the new exchanges and private trading systems contend that ordinary people benefit from the technologies whether they know it or not.

“Competition has benefited the average investor,” said William O’Brien, chief executive of Direct Edge, one of the new exchanges. “Their broker has so many choices available, on or off exchanges, anywhere in the world, and they can get their order executed in less than a second.”

Critics maintain that only the most sophisticated players are benefiting, able to execute their trades seconds before smaller investors and in private.

“There are tools now that certain investors have that give them an advantage over other investors,” said Joseph Saluzzi, who trades equities for institutional investors and hedge funds at his boutique brokerage, Themis Trading.

The Securities and Exchange Commission is beginning to take notice of such complaints, opening investigations into the new type of trading venues and promising action. It is worried, for example, that dark pools, with their scale unknown, could destabilize the market.

Unlike the Big Board, the new electronic exchanges are virtually unknown outside financial circles. Direct Edge, the largest, is in Jersey City. Another, the BATS Exchange, is based in Lenexa, Kan. Both are only about five years old. But each now accounts for about a 10th of daily United States stock trading.

In its fight to survive, the Big Board is building a new data center in New Jersey and another outside London. The Main Hall is being overhauled, in an attempt to lure business back to the floor. There is even a new coffee shop, Outtakes.

Even so, the world still watches — literally — what happens on the New York Stock Exchange. Twenty television networks broadcast live from the exchange, in nine languages.

But whichever way the market goes from here, many see a difficult road for the Big Board. The competition is unlikely to let up.

“There has been a sea change,” said Sang Lee of the Aite Group, a financial services consulting company. “I don’t envy what any of the exchanges have to do.”