How enterprises can accelerate the creation of new data quality solutions while aligning with business goals.
Article originally published on TDWI.org by Philip Russom, TDWI Research
Data quality (DQ) has always been a moving target, because enterprise data represents real-world entities (such as customers, products, partners, and employees) that naturally evolve over time. As if that weren’t challenging enough, data quality professionals are under renewed pressure to identify and provide quality improvements for new sources and types of data, as organizations deploy new applications, implement new customer or partner channels, explore big data, and tap into new sources (such as machine data and social media).
To keep pace with accelerating demands for data quality solutions, many data quality teams and tools have embraced practices drawn from agile development methods. The agile method for software development has been in use for over ten years, and its tenets are summarized in the Manifesto for Agile Software Development (http://agilemanifesto.org). Agile methods originally focused on the development of hand-coded procedural logic for operational and transactional applications. Agile data quality is where agile methods are applied to data quality projects and solutions.
Agile data quality typically has four goals:
These goals are achieved by adopting a lean team structure. Agile teams are usually led by two people who represent business and technical constituencies, respectively. The two leaders communicate directly to cut bureaucratic red tape, which in turn both speeds development and assures that technical work aligns with business needs. With agile DQ, team leadership commonly consists of two positions:
Note that agile DQ maintains the established practices of data stewardship, which focus on rapidly identifying data issues and addressing them, but adapts it to be even more responsive. For example, agile DQ requires more regular and direct collaboration between the steward and technical lead than data stewardship does. In addition, the steward and technical lead have even more independence and authority for prioritizing business needs relating to DQ and defining the details of DQ solutions.
Here are some recommendations for successful practices in agile data quality:
To learn more, read the new TDWI Checklist Report, Agile Data Quality, available here.