Coining Phrases - Reverse Auto-Classification

Recently, I have been talking with some software folks who have created parsers for auto-classification. These engines will take a text or website, scan the contents and see how it matches to a given taxonomy. These parsers use a variety of techniques including statistics, semantics, word location to see how a given document matches a taxonomy. These are very impressive, and I hope to use one of these for our taxonomy to classify our content. One of the unique traits of our taxonomy is that is multidimensional, and we use those dimensions to show the different ways that businesses operate in our economy. We have been able to leverage this to find buyers, sellers and comparables for Mergers and Acquisitions. One of the problems with this approach is that users who are searching our database need to have a fairly in depth knowledge of the taxonomy in order to find what they need. To solve this problem, we decided we needed to create meta-terms or coined phrases that would represent different search criteria to apply to our taxonomy. Instead of, going from text to taxonomy we are going from taxonomy to text. We also use all the synonyms for all the nodes in our taxonomy. So instead of having users select their criteria from a series of drop-down boxes, they can type in a Google-like text box and the software will auto-fill matches against the table of coined phrases generated from our taxonomy.

I have a live sample on our MandAsoft site here: http://mandasoft.com/segments/searchlob.aspx. A good example is accountant software which is a coined phrase that will search for software for accountants. Here are the results for that search http://mandasoft.com/segmentview.aspx?SearchID=LOB99.
Tell me what you think.

Challenges of Searching a Complex Taxonomy

As I have noted, our team has developed a very complex multi-faceted taxonomy which give a very exact categorization of any given business. The complexity we have created comes with a price. It creates challenges for users searching for businesses. A company is categorized by who their clientele is, what business needs they fulfill and by how they fill those needs, and what channels they use. A typical user does not think of businesses in such a manner. For instance, a typical search would be I would like all businesses that sell education software. In early versions of our application, users would have to savvy enough to enter in the search criteria education for the clientele and to enter software for industry (the "how a business operates"). Needless to say, users had to be trained to use the system, and hence the system was only used by our in house taxonomist who would get search requests and he would then give them results in a handy report. This was not ideal, because our taxonomist has many other things to do. The challenge for us was to make our taxonomy search as easy as Google. Nobody has to be trained to use Google, and we found that if training was involved generally that part of our application was not used unless the payback was great. In addition, since we were already fielding queries, our users found it much easier to send an email to our team, rather than run the search on their own. So how did we simplify our search. Well, in an earlier post, I mentioned we found that were phrases that users wanted in our taxonomy that would not fit in a single tree or dimension of our taxonomy. I called these phrases, "Meta-Terms". The example I gave was Trade Magazines which are B2B magazines. In our taxonomy we then map this term to magazines in our Industry tree and B2B in our Clientele tree. So the key to simplifying our search was to use "Meta-Terms" as a model for Google like search, Users can now type into a text box and it will see if it matches an existing "Meta-Term" or what I call an implicit "Meta-Term". Implicit "Meta-Terms" are created  by synthesizing synonyms from our different trees' vocabularies. There are over a million possible combinations from the trees, but some of the combinations are unlikely to exist like (e-discovery software for teenagers). So we have created a list of synthesized terms from the companies we have already categorized (numbers around 23,000). From these categorized companies and all the synonyms, we get a list of about 200,000 implicit meta-terms. Our "Google" search box then matches the user input to our list of terms and runs a query on how that meta-term maps to our multiple dimensions. This is still in testing, but shows tremendous promise.

Mapping a taxonomy to a taxonomy

In my last post, I talked about "meta-terms" which mapped a commonly used expressions to nodes in multiple trees. This concept could be taken much further. When our team built our four dimensional taxonomy, our goal was to be able to classify any business, and to find similarities between companies even though traditionally they  would be considered to be operating in different arenas. My favorite example is to look at Intuit which creates financial software for the consumer, and compare it to H.R. Block which provides a financial services for consumers. In the tax arena, they both provide help to people doing their taxes, and compete directly. Our taxonomy categorizes Intuit as a consumer software company for taxes, while H.R. Block is a consumer service company for taxes. As you see these companies overlap on what they do, and who they do it for, but not on how they do it. Interestingly enough, Intuit started offering a professional help service and H.R. Block started offer a software package.

What this brings up is that our taxonomy is complicated. Our team produces reports on Merger and Acquisition activity in a variety of segments (http://mandasoft.com), and each of these business segments like to break down using their own taxonomies specific to their domain. How do we reconcile the need for a taxonomy with nodes that can be used cross multiple domains, while needing to have easy to understand domain specific terms in a given domain?  The way I like to see this problem is that we have a vocabulary that works great when looking at the business world at the 50,000 foot level, but when we get down into trenches, the terms start to look vague and confusing at the lower altitudes. The way we solved this was by building a system to create 50 ft level simple taxonomies for specific domains (e.g. healthcare media and software). We then categorize each business using the 50,000 foot level taxonomy, and we then have rules sets that map from 50,000 ft level taxonomy to the 50 ft level taxonomy. The utility is especially noted when we create multiple domains with their own rules sets  (e.g. healthcare media and software, and  Cloud Computing) and a business which may reside in both domains, only needs to be categorized once at the 50,000 ft level taxonomy. We can create as many domains as we need and not have to reclassify companies as our domain views evolve!

"Meta-terms" in a multi-faceted taxonomy

Most taxonomies have synonyms for their nodes. A good example may be if you have a node for hospitals. Hospitals also could be known as Medical Centers, Clinics, Surgery, Health Service, etc. These are the synonyms typical of any given taxonomy. In the multi-faceted taxonomy, our team uses we have such synonyms, but sometimes we find certain key industry terms that actually span across multiple dimensions. For instance, Trade Books, Trade Magazines, HIMS (Healthcare Information and Management Systems) are terms that can be like a synonym except, they point to nodes in multiple trees. Trade Books are books for consumers. In our taxonomy we then map this term to books in our Industry tree and consumer in our Clientele tree.  Trade Magazines are B2B magazines. In our taxonomy we then map this term to magazines in our Industry tree and B2B in our Clientele tree. We call these special terms "Meta-terms", and they act as little mini-maps in our taxonomy, and expands our vocabulary like synonyms do. Later I will describe how we can build more complicated maps and rule sets to build mini-taxonomies that can use new sets of key words to define  new ontologies which set on top of our main taxonomy.