Julia and Fintech

Recently, I have been reading up on Julia, since a fellow programmer and quant told me about it, and he was very excited by it. Julia is a new open source programming language developed by computer scientists at MIT. The first compiler was released in 2012, and has been gaining adherents. Currently, it ranks #51 on the RedMonk language ratings. The key features of the language is that it is a high level dynamic code that can out perform other similar JIT (Just In Time) interpretive languages. The MIT scientists developed it for speed in numerical computing while also being easy to develop new algorithms.

Julia addresses the need to easily create new numerical algorithms, while still executing fast. Julia's creators noted, that before Julia, programmers would typically develop their algorithms in MatLab, R or Python, and then re-code the algorithms into C or FORTRAN for production speed.
Obviously, this slows the speed of developing usable new algorithms for numerical applications. In testing of 7 basic algorithms  Julia is impressively 20 times faster than Python, 100 times faster than R, 93 times faster than Matlab, and 1.5 times faster than FORTRAN.

So what does this mean for Fintech?

In the last 15 years, Capital Markets has seen explosive growth of Algorithmic Trading (aka High Frequency Trading) where in 2010,  it comprised 60 percent of all U.S. equity volume. In 2010 there was the well known Flash Crash, and algorithmic trading has retreated to about 50 percent. This form of trading depends on high speed numerical computing, and rapidly evolving quantitative finance algorithms which look at market conditions and tries to take advantage of the smallest of changes in those conditions. Could Julia, help speed the development and deployment of new algorithms that can outsmart other players in the market? It seems that it is poised to do just that.

Currently, the Julia ecosystem is still immature, but programmers are developing new Julia packages for the financial markets. Julia is also developed to be easily deployed on cloud containers, and has capabilities for distributed computing for distributed algorithms. With these features of fast algorithm development, cloud ready containers and distributed numerical computing, could Julia provide a platform for the rise of a second generation of the machine trading?

NoSQL Confession: I miss “joins”

I have been working with ElasticSearch and Couchbase this past year and a half, and I have a confession to make. Yes it has been amazing to see how to build a flexible, scalable and fast database that integrates sophisticated text searches with more traditional value matching queries. At Berkery Noyes, we have built, on NoSQL technology, an amazing tool to sift through millions of records of business intelligence including web visits, landing pages, emails, phone calls, merger and acquisition activity and even changes to company personnel. We use this tool to focus our efforts, and preliminary usage shows the search tool to be able to identify solid leads. But…

To do this, we have needed to de-normalize a large amount of data into our data documents. The NoSQL “join” features in both ElasticSearch and Couchbase are still too primitive to effectively use. I keep hitting problems like the inability to sort on “joined” documents, or severe latencies in updates to indices. In addition, there is the additional headache of insuring that de-normalized data is up to date. Oh for the old days of Primary Keys and Foreign Keys. However, for us, there is no turning back, the new power that our search app has is so useful that we deal with it. I guess I miss the old days of 2013 when we could have all our data tied up in a neat algebraic package on a SQL database. We live in a world that can be a little messy, so perhaps our data should reflect that.