I have been working with
ElasticSearch and Couchbase this past year and a half, and I have a confession
to make. Yes it has been amazing to see how to build a flexible, scalable and
fast database that integrates sophisticated text searches with more traditional
value matching queries. At Berkery Noyes, we have built, on NoSQL technology,
an amazing tool to sift through millions of records of business intelligence
including web visits, landing pages, emails, phone calls, merger and
acquisition activity and even changes to company personnel. We use this tool to
focus our efforts, and preliminary usage shows the search tool to be able to
identify solid leads. But…
To do this, we have needed to
de-normalize a large amount of data into our data documents. The NoSQL “join”
features in both ElasticSearch and Couchbase are still too primitive to
effectively use. I keep hitting problems like the inability to sort on “joined”
documents, or severe latencies in updates to indices. In addition, there is the
additional headache of insuring that de-normalized data is up to date. Oh for
the old days of Primary Keys and Foreign Keys. However, for us, there is no
turning back, the new power that our search app has is so useful that we deal with
it. I guess I miss the old days of 2013 when we could have all our data tied up
in a neat algebraic package on a SQL database. We live in a world that can be a
little messy, so perhaps our data should reflect that.
