In a recent engagement with a Fortune 50 client which I had blogged about here earlier, I find there to be considerable debate within the IT department on the nature of the role that Domain Subject Matter Experts will need to play in the design of Advanced Analytics Solutions. Some argue that the role of the Domain Expert needs to be extensive in all stages of the design and implementation process. Others,myself included, argue that the engagement of the Domain Expert while extensive, will need to be more iterative , increasing in scope as domain experts learn to think more like data scientists.
A lot of the information in organizations today is stored in free form text and documents both within transactional systems and outside formal system boundaries in the form of Word documents, Excel Sheets, SharePoint Sites, etc usually managed by the Line of Business. Needless to say, the domain expertise that experienced knowledgable employees bring to the table are critical in interpreting this information, which in turn, enables day to day operational decision making. That being said, there are several drawbacks on relying too much on the business domain expert. Most business users almost exclusively depend on Excel for their Data Analysis. Jeremy Howard of Kaggle.com in his talk on "Get the Picture: Gaining Insights with Data Visualization" explains how "Excel is very easy to make errors in" which can lead to some very wrong conclusions. Also in my experience, domain experts tend to come in rightly or wrongly with preconceived conclusions and are more likely to fit the data with what they believe ,rather than the other way around, allowing the data to guide them to the correct conclusions.
I agree with the assertion that Domain Subject Matter Experts need to play a big role in the design of Advanced Analytics solutions. Practical common sense, good judgement and an understanding of the real business issues are relatively uncommon and hard to acquire skills and we need domain experts for that. However, their inputs in the design and implementation needs to be of an iterative nature increasing in scope as the analytical model matures and/or evolves. Some of the tasks in advanced analytics design that a domain expert is best suited to perform are things like creating organization specific dictionaries, defining classification categories, creating an effective Training Set for the learning algorithms,helping understand the desired business outcomes at a more granular level amongst others.
As advanced analytics techniques and technology become more mainstream, training business users and data scientists to think more like each other, may be the real challenge. Those business experts and data scientists who learn these cross functional skills will be at a distinct advantage to their peers who do not and will be able to make a much bigger impact on business drivers and decisions within their organizations.