Search engines are now an integral part of people’s everyday lives. We are used to having access to information at the click of a button. However we rarely think how much work goes into this ability to search for information. Search engine software has become extremely advanced in recent years, now using complex algorithms to provide the most relevant information with predictive search and search suggestion capabilities. Many engines can do this in real-time, processing millions of…

Today the world generates vast quantities of data each day that can be used to enhance the quality of living of virtually anyone in the world. Information is power but also a tool for supporting development, knowledge sharing and social initiatives. Tracking natural disasters, crowdsourcing rainfall data and mapping out the night’s sky are amongst a diverse range of…

Stricter rules are being imposed on the financial sector in a variety of areas following the financial crisis. Regulators are more than ever involved in scrutinising Know Your Customer (KYC) processes and are large fines for non-compliant firms. Anti-money laundering (AML) practices have become deeper and more advanced to counter the financing of terrorism, corruption and crime. EMIR and Dodd Frank legislations were also put into place with the aim of improving transparency and reducing…

With the rapid growth of data and the shift towards rapid development solutions much data is being stored in NoSQL stores such as Hadoop and MongoDB. The infrastructure built upon relational databases that have been used for decades cannot keep up with the volume and scope of data being captured. Further to this SQL is also a really good invention and method for extracting and analysing data that is very widely used.  In short it will not be replaced by hierarchical query techniques…

Traditional approaches to enterprise reporting, analysis and Business Intelligence such as Data Warehousing, upfront modelling and ETL have given way to new, more agile tools and ideas. Within this landscape Data Preparation tools have become very popular for good reason.  Data preparation has traditionally been a very manual task and consumed the bulk of most data project’s time.  Profiling data, standardising it and transforming it has traditionally been very manual and error…