Coining Phrases - Reverse Auto-Classification

Recently, I have been talking with some software folks who have created parsers for auto-classification. These engines will take a text or website, scan the contents and see how it matches to a given taxonomy. These parsers use a variety of techniques including statistics, semantics, word location to see how a given document matches a taxonomy. These are very impressive, and I hope to use one of these for our taxonomy to classify our content. One of the unique traits of our taxonomy is that is multidimensional, and we use those dimensions to show the different ways that businesses operate in our economy. We have been able to leverage this to find buyers, sellers and comparables for Mergers and Acquisitions. One of the problems with this approach is that users who are searching our database need to have a fairly in depth knowledge of the taxonomy in order to find what they need. To solve this problem, we decided we needed to create meta-terms or coined phrases that would represent different search criteria to apply to our taxonomy. Instead of, going from text to taxonomy we are going from taxonomy to text. We also use all the synonyms for all the nodes in our taxonomy. So instead of having users select their criteria from a series of drop-down boxes, they can type in a Google-like text box and the software will auto-fill matches against the table of coined phrases generated from our taxonomy.

I have a live sample on our MandAsoft site here: http://mandasoft.com/segments/searchlob.aspx. A good example is accountant software which is a coined phrase that will search for software for accountants. Here are the results for that search http://mandasoft.com/segmentview.aspx?SearchID=LOB99.
Tell me what you think.

True relationship between parent and child

I recently had to make a change to our taxonomy search. Our previous search would match a selected node and all its children. However, we found a need to search and just match a node and not any of its child nodes. I have seen this feature in many other systems, but we never found the need to implement it in our system till recently. This highlights a deeper question of what do we expect the relationship between parent and child nodes. When we first developed our taxonomy, we had long heated discussions on this issue for our industry tree. Let's take an example of a well known software company, Oracle. Under our software category we have operating systems, business applications, desktop applications, database systems, email systems, graphics applications, etc. Oracle is a big software company that makes all kinds of software, but does not create all the types of software under our software category, but they do most. One of our team members suggested that if a company is categorized as something then it must do all the subcategories. His suggestion was to categorize multiple times for each type of product it does exactly. However, it starts get ugly when a company like Oracle is operating in most of the subcategories but not all. Our general consensus was to categorize a company in the parent category if it covers a good portion of the subcategories, but not all. If the company covers a few of the subcategories, then we will have multiple categorizations. The real key is consistency. We take the approach (and near cliche) that the parent is something different than the sum of its children.

Challenges of Searching a Complex Taxonomy

As I have noted, our team has developed a very complex multi-faceted taxonomy which give a very exact categorization of any given business. The complexity we have created comes with a price. It creates challenges for users searching for businesses. A company is categorized by who their clientele is, what business needs they fulfill and by how they fill those needs, and what channels they use. A typical user does not think of businesses in such a manner. For instance, a typical search would be I would like all businesses that sell education software. In early versions of our application, users would have to savvy enough to enter in the search criteria education for the clientele and to enter software for industry (the "how a business operates"). Needless to say, users had to be trained to use the system, and hence the system was only used by our in house taxonomist who would get search requests and he would then give them results in a handy report. This was not ideal, because our taxonomist has many other things to do. The challenge for us was to make our taxonomy search as easy as Google. Nobody has to be trained to use Google, and we found that if training was involved generally that part of our application was not used unless the payback was great. In addition, since we were already fielding queries, our users found it much easier to send an email to our team, rather than run the search on their own. So how did we simplify our search. Well, in an earlier post, I mentioned we found that were phrases that users wanted in our taxonomy that would not fit in a single tree or dimension of our taxonomy. I called these phrases, "Meta-Terms". The example I gave was Trade Magazines which are B2B magazines. In our taxonomy we then map this term to magazines in our Industry tree and B2B in our Clientele tree. So the key to simplifying our search was to use "Meta-Terms" as a model for Google like search, Users can now type into a text box and it will see if it matches an existing "Meta-Term" or what I call an implicit "Meta-Term". Implicit "Meta-Terms" are created  by synthesizing synonyms from our different trees' vocabularies. There are over a million possible combinations from the trees, but some of the combinations are unlikely to exist like (e-discovery software for teenagers). So we have created a list of synthesized terms from the companies we have already categorized (numbers around 23,000). From these categorized companies and all the synonyms, we get a list of about 200,000 implicit meta-terms. Our "Google" search box then matches the user input to our list of terms and runs a query on how that meta-term maps to our multiple dimensions. This is still in testing, but shows tremendous promise.

Requirements for a better Business Taxonomy Part 3

Having discussed that we can have multiple dimensions for a detailed Business Taxonomy, lets see what dimensions we might want to have. The first two dimensions we discussed about described 1) Who is the company's clientele (for media we should look at the audience) 2)How the company services their clientele. I suggest we also give a dimension for 3)What business need the company accomplishes for their clientele. For instance, our hypothetical healthcare software could accomplish a particular process. A big new push in healthcare is Electronic Medical Records (EMR). If we have this third dimension, we can now classify a Healthcare Consulting company specializing in EMR. Now, if we search for businesses providing EMR solutions, we will get results for any company who are working in that space weather they are software or a consultant. Most taxonomies that "solve" this problem by searching on a mix of keywords and their tree structure. This third dimension gives a way to tie in companies that are working on related subjects but using different methodologies. Look at HR Block and Intuit. One is a service company and one is a software company, but both provide tax solutions. We will look at more dimensions in the next post.