Run don't walk - Simplicity and Speed

One of the unique features of our taxonomy is that we like to show how near items are in the taxonomy to other items classified using our taxonomy. Since we are classifying companies, it allows for us to define a "sphere of competition". Our first version of software which determined "nearness" used a walking algorithm which would walk up and down the trees to determine distances. Searches for "nearby companies" took a long time, up to a minute. Recently we changed the algorithm to use some algebra to do the walking and now searches take less than a second!. The algebraic algorithm was simpler and faster. Check it out on mandasoft.com. Use the see comparable deals link.

Taxonomy Evolution Conundrum

Our team has been developing our taxonomy for almost ten years now. Our goal is to classify businesses by looking at how they operate, who they serve, and what they do, and our focus has been on media and software businesses. Needless to say over the last ten years, there have been major changes to the media and software industries with the introduction of smart phones, tablet computers, cloud computing, SaaS, virtualization, etc. To handle this evolution of the content we are classifying, we need to make sure our framework was solid and that the taxonomy could change with abilities to add nodes, merge nodes, link nodes, and to make sure our classifications migrated with the changes. However, change is never apparent when it happens. When we saw the first business operating in Social Networking, we originally had them classified basically as forums of user generated content, as opposed to editorial content. But as the business and technology took off, and showed itself to be a new business model, we realized we had to add the term Social Networking to our taxonomy. Now our problem was that we had to go back and re-evaluate our companies that were classified as forums and see if they were really Social Networks. One way to fix this problem is to have an auto-classifier, and you set up a new set of rules to recognize Social Networking. Then  you re-run the auto-classifier on those companies. But here is the conundrum, we noticed this evolution in business models because we had human eyes seeing the trend. How can you expect an auto-classifier to see that? What are your thoughts on this problem?

Taxonomy Mapping Engine in Use

Our team is in Annual Trends Report Mode. Yesterday we published our first of seven Trends Reports tracking Mergers & Acquisitions in the Media and Software Sectors. This report is for the media space. One part of the report is to collect the deals that for this space. We use an auto-population algorithm that fills the lists many times a day. We then use an Industry Map and rules set that maps the categorized deals into a simple flat taxonomy just used for this report. We can then compare sub-segments of the Sector to see which segments are performing better or worse. Note we do this for 7 different reports each with its own Industry Map and rules set. We never have to categorize a deal more than one time. The mapping engine puts the deals into the appropriate bins for that report. Check it out here.

The Banyan Tree - a new hierarchy

Most taxonomies are set up in a hierarchical tree format. Our team consists of four distinct trees for each facet of how a business operates. Developing the taxonomy and software over the last 8 years, we found at some point that the tree structure became too strict. There were certain categories that did not want to be under just one parent category. A prime example of this situation is video game companies. These companies make software for entertainment purposes. As this industry has matured, it has become closely link with the big entertainment companies, and they employ teams of artists, writers as well as programmers. Historically, these businesses should be software, but they are also so tied closely to entertainment companies it seems odd that when searching for entertainment companies that these would not turn up in our search results. One solution is to move video game studios to under the entertainment category, but then we lose the software aspect of the business. Our team's solution was to change the nature of our trees. We now allow nodes to have multiple parents, and create what I call the "Banyan" tree, which is a tree from India that has multiple trunks to the ground. 

Looking at the Wikipedia article, we see that the banyan tree name comes from the Gujarti word for merchant, because merchant markets were often located under these great trees. It seems appropriate for a business taxonomy.

Challenges of Searching a Complex Taxonomy

As I have noted, our team has developed a very complex multi-faceted taxonomy which give a very exact categorization of any given business. The complexity we have created comes with a price. It creates challenges for users searching for businesses. A company is categorized by who their clientele is, what business needs they fulfill and by how they fill those needs, and what channels they use. A typical user does not think of businesses in such a manner. For instance, a typical search would be I would like all businesses that sell education software. In early versions of our application, users would have to savvy enough to enter in the search criteria education for the clientele and to enter software for industry (the "how a business operates"). Needless to say, users had to be trained to use the system, and hence the system was only used by our in house taxonomist who would get search requests and he would then give them results in a handy report. This was not ideal, because our taxonomist has many other things to do. The challenge for us was to make our taxonomy search as easy as Google. Nobody has to be trained to use Google, and we found that if training was involved generally that part of our application was not used unless the payback was great. In addition, since we were already fielding queries, our users found it much easier to send an email to our team, rather than run the search on their own. So how did we simplify our search. Well, in an earlier post, I mentioned we found that were phrases that users wanted in our taxonomy that would not fit in a single tree or dimension of our taxonomy. I called these phrases, "Meta-Terms". The example I gave was Trade Magazines which are B2B magazines. In our taxonomy we then map this term to magazines in our Industry tree and B2B in our Clientele tree. So the key to simplifying our search was to use "Meta-Terms" as a model for Google like search, Users can now type into a text box and it will see if it matches an existing "Meta-Term" or what I call an implicit "Meta-Term". Implicit "Meta-Terms" are created  by synthesizing synonyms from our different trees' vocabularies. There are over a million possible combinations from the trees, but some of the combinations are unlikely to exist like (e-discovery software for teenagers). So we have created a list of synthesized terms from the companies we have already categorized (numbers around 23,000). From these categorized companies and all the synonyms, we get a list of about 200,000 implicit meta-terms. Our "Google" search box then matches the user input to our list of terms and runs a query on how that meta-term maps to our multiple dimensions. This is still in testing, but shows tremendous promise.

Spheres of Competition

Last week I spoke of looking at a business taxonomy in new way. Generally, people think of taxonomies as a vocabulary with perhaps a hierarchical structure of categories and sub-categories. However when you build a multi-dimensional taxonomy as our team has, you can now start to think of it as a spatial topology. There are four trees and each one defines a dimension in our business taxonomy space. This thought is analogous to the special theory of relativity from physics where you have the x, y, z dimensions plus the time dimension. An "event" is a point in the space time continuum is defined by those four dimensions. In our business taxonomy space, a "company" is a point in the spatial topology defined by our four dimensions. If you draw a small sphere around a given company's point in our taxonomy, you will get all the competitors of that company. We have seen as you widen the sphere the outlying companies are less likely to be competitors. The key to making this work is to define the distances between points in a given dimension's tree. We generally realize that the distance between parent and child is shorter the deeper you get into the tree, and the distance between siblings is slightly more than that between parent and child. We also realize that you may define siblings where some siblings are closer in meaning than others. Our distance algorithm has to take all these things into consideration. Our work has been experimental, but has returned interesting results. We have use this in our drill-down feature on mandasoft.com.  The space defined has to be tweaked, and I may leverage algorithms similar to Einstein's general relativity where actual data defining company revenue at a point in our topology could warp the spatial distances, just like physical mass warps physical space. Any thoughts?

Why is a multi-dimensional faceted taxonomy better for Business?

As we have seen, we can more closely describe a company and how they operate by using four different hierarchical trees to categorize their clientele, their methodologies, their solutions and their channels. This faceted taxonomy definitely gives us a better view of any particular business, but at what cost.  Our team has definitely found using this taxonomy has been challenging to categorize a given company. The taxonomist must research thoroughly a given business, and then be able to abstract that understanding into the four different dimensions. Another challenge we found in our first search tool, was that the user using the search tool needed to understand how to abstract the kinds of businesses they were looking for into the four different dimensions. These two issues makes us wonder whether it is worth it. (Currently, we are working on software algorithms to make the search more user friendly and to make suggestions from web scraping to help the categorization.) 


We find it is worth it. The reason why is that, unlike NAICS codes, our taxonomy allows us to dig into micro-market views. NAICS is good for broad markets, but not for close views of a given market segment. Looking back to the 2004 Presidential Election, we see that President Bush's team was able to efficiently direct resources by using Microtargeting. The impetus of our taxonomy is Mergers & Acquisitions, and an important part of that process is valuation. To find the potential target company of an acquisition, investment bankers search for recent comparable deals. A task that is impossible using NAICS codes.  With our taxonomy, we can do this. The reason is because we have moved away from the traditional way of viewing "Business" which is in terms of a vocabulary.  Instead, we now can view "Business" as a multi-dimensional space where we can define "Spheres of Competition" to determine "comparables". Stay tuned for more.

Requirements for a better Business Taxonomy Part 2

Following my previous post, we see that a business can be classified in a parent child hierarchical taxonomy, but sometimes one could create a sub-category which is really expressing a not a sub-type of the parent category, but rather a different aspect of the business. As in healthcare software is not really a sub-category of software. Healthcare defines the customer base or subject matter of the software. A true sub-type of software would be infrastructure software or business application software. An improved business taxonomy would then categorize a company in multiple ways or dimensions. For instance you could have, a dimension to describe the clientele or market that. So our healthcare software company would have its clientele be set to healthcare. Another dimension would describe how the company solves the business problems in the case of our  healthcare software the company would be categorized as software. In my next post we will discuss other possible dimensions for a business taxonomy.