Hi Chris - This paper on custom entities may be of interest:  http://www.sas.com/en_us/whitepapers/discovering-what-you-want-107347.html It's a way to include pre-defined entities into a discovery analysis with SAS Text Miner. You may also be interested in the text profile node in SAS Text Miner, used to associated descriptive terms with different levels of a dependent (target) variable - including time. Hope this helps, ... View more

I'd suggest a SAS database, (but am kind of biased 😉 Because SAS can read virtually any data store, you aren't limited by the software. Of course, a SAS format is often preferred (did I mention I'm kind of biased ...) ... View more

Hi Priya - If you're looking to learn the software and about the topic I'd suggest you see of SAS Analytics U: http://www.sas.com/en_us/learn/analytics-u.html  can help. It's free! If that doesn't meet your needs, perhaps the online courses? Courses and e-Learning from the experts at SAS | SAS Traininghttps://support.sas.com/training/ ... View more

Great to hear that you enjoyed the conference. We have completed the web page update. Please see: SAS Contextual Analysis | SAS for more information regarding the product. ... View more

Hi Abhijit, In the current configuration, there aren't specific dependencies - but there are definite advantages of using the two products together. For example, entities can be created in SAS Enterprise Content Categorization and used as custom entities in SAS Text Miner. I think of this as automated discovery that also includes things that you want to find (i.e. custom entities, like product codes for example).  For those who don't have the full capabilities of SAS Enterprise Content Categorization, SAS offers an add-on to Text Miner = SAS Concept Creation for SAS Text Miner (specifically for the purpose of creating custom entities to include in text mining discovery). Here is a recent SGF paper on that topic:http://support.sas.com/resources/papers/proceedings13/100-2013.pdf  Of course, SAS Enterprise Content Categorization does alot more than just this add-on. In SAS Text Miner, you can automatically discover Boolean rules to create an initial content categorization rule set  - to further refine in SAS Enterprise Content Categorization Studio (some highlights of this capbilities, new to SAS Text Miner 12.1 can be seen in the short product demo video from: Text Mining, SAS Text Miner | SAS Linguistic rule development in Enterprise Content Categorization is not done in batch - it is user-specified (and thus the benefit of an initial rule set automatically generated from Text Miner). That being said, SAS Enterprise Content Categorization has some methods for automated taxonomy generation, like from Wikipedia and DBPedia.  Scoring with models developed from any of the SAS Text Analytics products can be done in batch.  The format of the models/rules resulting from SAS Enterprise Content Categorization are in XML, for scoring using the Enterprise Content Categorization Server. That being said, once document are scored, this is new metadata, can be used to enhance existing enterprise search and content management systems, and can be used as new variables for further exploration, reporting and analysis. Sincerely, frm ... View more

Hi Ken - The data doesn't need to be different between SAS Text Miner and SAS Content Categorization - both analyze electronic free format text, from a variety of sources and formats.  The difference is that Text Miner examines a collection, analyzing it to find topics, clusters, associations, themes and rules buried amongst the documents.  SAS Content Categorization on the other hand, examines each document at a time, applying linguistic rules and taxonomies to classify, extract terms, phrases, facts, etc. Both methods score documents. ... View more

Hi, Have you had a chance to look at this material? SAS Content Categorization and Add-Ons ... View more

Hi Ken and Julia - Text mining is a discovery technology, working across a collection of documents and applying machine learning, NLP, statistics, etc.. to identify - in this case, predictive rules.  And while you can output those rules in a format for Content Categorization (as a discovered starter rule set - using the Text Miner left hand navigation option), refinement and additions are currently built from the Categorization technology (just as Julia describes). Categorization is focused on advanced linguisitc rule definition and management. And the Categorization Server is currently used to apply those rules to incoming, individual documents. ... View more

The 'UNLESS' and 'NOT' rule operators (in Content Categorization and in Sentiment Analysis) are available to limt matches. ... View more

Yes Aurthur - absolutely. Just to add a bit, document conversion is included in all the SAS Text Analytics products - to translate from Word, .pdf, etc... to a normalized .txt format.  The Text Import node in Text Miner will crawl web pages, and retrieve documents from any site defined. For Categorization and Sentiment Analysis, web sites and file systems can be crawled and the information retrieved using SAS Informaiton Retrieval Studio - which for crawling is included with Sentiment Analysis, and is an add-on component of the Categorization bundle. Not knowing what type of data you are dealing with, have you seen this: White Paper | Sifting Through the Noise of Social Media | SAS It provides some data cleansing/quality tips for social media data. ... View more

Hi, Perhaps knowing that standardized training and validation data sets stratified by the target were used across the model test suite mean that all of the models for each package were evaluated using the same data. And as cited: model quality was assessed through common model quality measures (Han & Kamber 2006), ie. •             cumulative lift in the first decile,  •             percentage of correctly classified events (often called event precision), and  •             overall percentage of correct classification  Since the analysis used historical data, the event value for the target is known. You need historical data with known values to do predictive modeling. The predictions from each model were then compared on the KNOWN common validation data to evaluate model quality using the statistics above. As a side note - oftentimes there is an improvement in predictive model performance with the inclusion of variables derived from text data.  At last years Analytics 2012 event, United Health Group - indicated that they generally found that predictive models improved significantly when variables from text data were added to the algorithms - citing for example that the missclassification rate (from 30% to 10%). ... View more

SAS is described as one of the Megavendors in this Sept. 2012 report - along with IBM and SAP.  The report outlines the product capabilities, as a checklist as it does with all the vendors, and describes the focus areas (in addition to analysis of text data) as integration with structured data and event stream processing for use in predictive analysis and data management. It mentions that there are also a number of industry specific solutions that utilize text analytics (as well as domain expertise, reports, etc..).  A more recent report that you may be interested in - and the first comparative text analytics report is the Hurwitz Victory Index - the full report is available from www.hurwitz.com, and the SAS excerpt is free from the SAS site listing: Analyst Viewpoints by Technology | SAS ... View more