When do BI projects that require new data we know anecdotally that once we have acquired the required data from the System of Records the majority of our time and risk is applying code that transforms and/or augments data.
In my terminology I call this “doing bad things to data”. A better term is a Business Rule.
Typically the Extract Load and Transform (ELT) code that is written in the data platform is made up of multiple Business Rules. Key Performance Indicators (KPI’s) are also comprised of multiple Business Rules.
Some examples of Business Rules in the data space may include:
- Filtering or joining data to give it context
For example identifying Customer records
- Augmenting Data
For example flagging a customer as a “Gold” customer