I often refer to the data field – “big data,” “data science,” “machine learning” (ML), “AI,” and so on – as “analyzing data for fun and profit.” It’s a reminder that the concept isn’t new, and that there are practitioners who do this work even if they don’t refer to themselves as data scientists. That means that data scientists, and those who employ them, can look to those other fields for ideas.
Electronic trading – sometimes called “algorithmic trading” or just “algo trading” – is certainly worth a look. I spent part of my career in that space, and a lot of what I learned then translates very well to today’s world of ML/AI.
Over the coming posts, I’ll share some of those ideas and best practices. The core themes are: “think in terms of experiments,” “keep an eye on the money,” and “always account for your risk.”
But first, let me explain what algo trading really is, and how it’s similar to ML/AI:
A primer on algorithmic training
In the investing world, traders buy and sell shares of stocks. People who work in electronic trading develop execution algorithms 1 (“algos” for short) to place those trades. Like other forms of automation, execution algorithms have an advantage over people placing trades by hand because they:
- operate faster – they can respond to a variety of market activity in the time it takes a person to blink
- are consistent – they don’t accidentally click the wrong field, or get distracted and enter the wrong price 2
- don’t express emotion – algos can’t get nervous and buy/sell shares before the designated time
- can factor a very large amount of data into their decisions – both in terms of “number of features” and “amount of historical data used for training”
Despite those advantages, algos are still code, so they are not as adaptable as people. And when you consider that the stock market is full of surprises, that leaves room for the algos to misbehave (usually in the form of: “lose money, quickly”) due to unexpected operating conditions. This is why traders bake a lot of testing into model development, and also build plenty of monitoring around deployed algos.
What next?
To summarize: algorithmic trading involves building tools that perform automated decision-making with minimal human intervention, raking in all the risk and reward that comes with. If your company is using ML/AI to automate work and generate revenue, then this probably sounds familiar.
Over the next several posts, then, I’ll explore some of those ideas and best practices. You can improve your company’s ML/AI efforts by building on the hard-earned lessons of algo traders:
- Know your objective
- Think in terms of experiments
- Develop a solid data infrastructure
- Monitor your data feeds
- Monitor your models
- Keep an eye on your risk
- Develop controls around data
-
This is inconsistent with how I use the term “algorithm” in the ML/AI world – “data, fed to an algorithm (Naive Bayes, SGD, and so on), yields a (predictive) model” – but as that’s the terminology in the trading world, I’ll use it here. ↩︎
-
Granted, the human who developed the algorithm can make a mistake and embed that into the code. While that means the algo would be taking the incorrect action, it would still be consistent with what it was programmed to do. ↩︎