Summary: If you have your finger on the company’s financial pulse you need to start thinking about these risks and benefits posed by Big Data.
Big data is the latest buzz phrase in business but what does it mean and why is knowledge of Big Data so important for the office of the CFO, Auditors, and financial professionals of all levels? The notion of Big Data is not entirely new. After all, CFOs are accustomed to dealing with mounting volumes of information. But when does simply a lot of information become Big Data? For the most part, the finance function has not had to deal with the issues of Big Data. The volumes in most core financial applications are large but certainly not in the realms of terabytes, petabytes, or even greater. Big data takes “large” to an entirely new level.
The statistics surrounding Big Data are truly breathtaking. According to Gartner, the volume of worldwide information is growing annually at a minimum rate of 59 percent annually—or to put it another way, all of the world’s data in existence today will have doubled in less than two years. But Big Data is characterized by much more than just volume. The so-called three “Vs” of Big Data neatly sum up its characteristics: Volume, Velocity, and Variety.
Volume: In the data that concerns financial professionals the primary focus remains the transactional data from traditional finance, payroll, inventory, order, and fulfillment systems. But wait. The economy is growing fairly slowly so why should data volumes be increasing? First of all our data capture systems are becoming more sophisticated so we capture more finely grained detail about the business. Second, if you are faced with mergers you know that even after the differences in procedures and reconciliations are ironed out, there is by definition more to be measured than there was before.
Variety: However, a great deal of the explanation lies not in an increase in transaction volumes but in a broadening of data sets (collecting more analysis about current data) plus the collection of entirely novel types of data. For example, in the accounting arena Solvency II requires insurers to hold information about counterparties and IFRS demands more-segmental analysis. Furthermore, environmental and sustainability reporting has forced some organizations to collect entirely new information, such as electricity meter readings and CO2 emissions. Hence, organizations are grappling with variety as well as volume.
Added to the mix is the rampant growth of unstructured data such as commentary and other text-based information in social media, blogs, and websites. The proliferation of mobile devices has added a new dimension, exacerbating the growth of Big Data as organizations and individuals find themselves able to interact with each other as well as corporate systems anytime and anywhere.
Velocity: But what of velocity? Uncertain times create an insatiable appetite for information. Nervous regulators want to see information more frequently and management teams want to reforecast more often. So the speed with which information is demanded, delivered, and consumed is accelerating. So how do financial professionals rise to the challenge of Big Data?
Not All Data is Big Data: Despite these definitions, if you are dealing with Gigabytes or even Terabytes of data you are probably not yet in the league of Big Data. Conversations about Big Data are actually about Big Data Engineering, a group of new and revolutionary data capture, storage, and retrieval technologies that have only been commercialized since about 2008. The more accurate question is do you need to utilize these new technologies to avoid risk and adequately control your company’s finances?
At the core of these new technologies are data bases that replace traditional relational data base storage systems (RDBMS) with newly structured tag-and-value stores. The most common of these emerging technologies is open-source Hadoop which has many commercial variants including Mongo, Cloudera, SAP Hanna, HortonWorks, Netezza and proprietary offerings by the likes of IBM and Oracle. Hadoop (or the Hadoop File System – HDFS) along with ‘map-reduce’ (its native query procedure) runs on inexpensive commodity servers that are organized in massive parallel processing systems (MPP) and are almost an order-of-magnitude less expensive to establish and operate than the traditional relational database systems that have served us so well since at least the 1980s.
This technology is specifically designed to handle not only traditional data but also the unstructured and semi-structured data that are both the problem and the promise of Big Data.
Incremental Approach: For most companies this does not mean abandoning current investments in systems and software. Technology providers are already actively providing systems with architectures that blend the two types of storage and retrieval so that analysis can be conducted on data stored in both types, and frequently that data can be aggregated to be even more informative than in the past.
What Needs Your Attention Today: In the world of financial risk and performance management Big Data offers many new types of benefit but there are two specifically that many CFOs and auditors are adopting first.
Social Media Risk: There is wide agreement that social media and the intentional or unintentional leakage of company information needs to be a part of all new financial controls both for compliance and reputational risk. However that information exists only as free-form text (unstructured data) both inside the company (emails, text messages, documents, etc.) and outside the company (Facebook, Twitter, Yelp, and all the other rapidly growing web sites and communications tools). Any of these may contain risk-related information about your company.
New Big Data technologies allow us for the first time to capture, store, retrieve, and analyze this text data in ways that was never before possible in our relational databases. Many CFOs and auditors are adding to their controls and tests by causing this form of Big Data to be captured and blended into the current information technology architecture for just this purpose.
The End of Sampling: Testing financial processes through sampling has been so central to financial controls for so long that the mere suggestion that we replace it seems impossible. However, the real reasons for sampling were technology and human time limitations. We could not examine every single transaction for fraud, or ever single T&E Report for compliance with policy. New Big Data technology however now allows two things, extremely rapid retrieval of data, and through the distributed processing of massive parallel processing systems the ability to do exactly that, examine not just some but every instance of a particular type of transaction.
There are many other applications of Big Data both to traditional financial and risk controls, and to the new more proactive role for CFOs in company performance management that we’ll discuss in future posts. But let this be a wake-up call to start this conversation within your company. The benefits of Big Data are many, and the risks of ignoring it are equally as large.
Bill Vorhies, President & COO – Data-Magnum - © 2013, all rights reserved.
About the author: Bill Vorhies is President & COO of Data-Magnum and has practiced as a data scientist and commercial predictive modeler since 2001. He can be reached at:
original blog at: