Gathering Data Requirements

by | 6.1.3 - Data Requirements, Concepts, Underway

As a scrum master
I want I want to understand how to make retrospectives fun
So that I can keep the sprint team engaged

The Data Requirements Gap

There are two typical challenges when we gather data requirements.  First, we capture and document the requirements in a way the person capturing them understands.  Second, we explore and model the required data structures solely based on the system of record data structures or the users reporting requirements.

Document Hand Over

Let’s look at the way we capture data requirements. We will typically capture our data requirements in a traditional application-centric requirements document which is used to document the requirements for a report or a dashboard.  This is will typically have a list of fields required, plus a specification of layout and a bunch of functional and non-functional requirements.

Once completed the Business Analyst (BA) will hand this document over to a data designer or modeller who then has to understand what the fields mean and where to find them.

Of course, the document typically holds minimal data context and the data modeller will either have a guess or engage with the key users to ask them more questions to discover the context.

This results in a gap between what the key users asked for, what the BA documented and finally what the data modeller designed for.

Data Driven or Report Driven Modelling Approach

When we first started delivering data using a Data Warehouse approach, we often took a data-driven method.  We would spend a long time looking at the data in the Systems of Record and modelling how we could land and store this data to make it easier to use in the future.   The problem with this approach was it took a long time as we often spent time understanding and modelling data that a user may never consume, just in case it was needed in the future.

Then we moved to a report driven method.  We would document the content required by the users and then acquire and model the data to meet only these reporting needs.  This was often faster to deliver, but the problem was we often ended up with a large number of disparate reporting data sets that overlapped and were hard to manage.

Both of these methods lead to a gap in expectation of the time it took to deliver data and content and the ability to quickly add or change this data in the future.

Business Event Driven Approach

Business Events are events that take place in the course of the normal operation of a business and that reoccur as business processes are executed.

What is BEAM✲?

BEAM✲ stands for Business Event Analysis & Modelling, and it’s a methodology for gathering business requirements for Agile Data Warehouses and building those warehouses. It was developed by Lawrence Corr(@LawrenceCorr) and Jim Stagnitto (@JimStag), and published in their book Agile Data Warehouse Design: Collaborative Dimensional Modeling, from Whiteboard to Star Schema (Amazon, eBook).

Why BEAM✲?

There are three parts to the BEAM method that provides the Corr of its value:

  1. Business Event Driven
    You identify data requirements based on Business Events.
  2. Bridging the language gap
    The data requirements are documented in a langauge and a template that can be understood by business users and developers alike.
  3. Modelstorming
    You gather these requirements using a modelstorming approach that is interactive and fun.

Business Events

One of the benefits of documenting data requirements as Business Events is that an organisations Business Events will not change often.  In fact, they tend to markedly change only when the organisation’s core business model changes.

For example, a company that makes Ice Cream will still be making Ice Cream even when they introduce new Ice Cream products, open’s in new markets or targets new types of customers.  However the Systems of Record that they use to capture the data and the types of analytics and reporting content they need will change over time.  If the organisation does indeed change the core of what it does, for example, starts delivering health care, then it becomes obvious that a major change has happened then the impact to the data requirements will be major as well.

Bridging the Language Gap

We will typically ask the key business users BI specific questions that we understand.  For example, what are the key dimensions of the business data or what is the “grain” of the data?  We understand these because we have worked with them for years, our business users have not and this leads to either the business users or the developers having to translate.

Good requirements need both developers and business users to speak a common language and part of BEAM’s magic is that it gives business users and developers this common language. For example, the business event has a Customer who Purchases Product. Business users understand this concept, there are two business aspects; Customer and Product and the interaction between them. Developers can also understand this.

This enables us to ask the question “what events happen in your business?” which is a much easier way to start gathering requirements than “what entities do you want to report on?” Following up with “what other events happen around the event we just discussed?” is better than “does this FACT table share any dimensions with other FACT tables?”  Rather than relying on the BA or Developer to translate the data requirements, the BA can facilitate the agreement about the answers to the questions using by BEAM method.

BEAM✲ relies on 7 Ws (or really 5 W’s and a couple of H’s); who, what, when, where, why, how and how many.

These simple questions enable you to gather all the data details you need to model and build. You take a business event like Customer Purchases Product and ask these questions to understand the aspects which are important to that process.

  • Who? – In this case, we have Customer and probably an Employee who are involved in this event;
  • What? – The Product is one of our whats, it’s the item that’s involved;
  • When? – The date/time the purchase happened;
  • Where? – The store the purchase occurred in.
  • Why? – This can be interpreted in different way’s, for example, it was a hot day;
  • How? – The key piece of data that tells me this actually happened, for example, OrderID or Purchase Receipt;
  • How Many? – These are the values that you need to know. e.g. How many products were purchased and what were their costs.

Using this methodology, you can understand the data behind every business event. It gives you a structured set of questions to document the requirements you need to model your data warehouse.  Even better Lawerence Corr has developed a set of Open Source Excel templates that allow you to document these data requirements in an easy and consistent way.  You can download the BEAM templates from here.

Modelstorming

Modelstorming = Data Modelling + Brain Storming.   Modelstorming is how you gather business requirements using the BEAM✲ method.

??? whiteboard

?? collaborate by visualising

??? populate by examples

Other requirement articles.

Information Products – Gather Size and Prioritise

  As a Developer I want to understand how to run the Information Products workshops So that I can deliver reqiurements quickly Information Products As outlined in the Information Product post there are four major stages where the Information Product template is...

AgileBI Requirements Workflow

 As a BI PractitionerI want to understand what order to gatther requirementsSo that I can gather them efficientlyWhen coaching or mentoring a new AgileBI team on the AgileBI Way one of the first things they ask is what is the standard order in which to gather...

Stakeholder Onion

As a product owner I want I want to understand how to identify my stakeholders So that I know who can tell me what to do and who to tell what we are doing Other Blogs from this...

Wireframing to confirm requirements

As a developer I want I want to understand how to use wireframing to confirm requirements So that I can build the right thing Wireframing is a technique we use to confirm two things: How does the user wish to see the content presented? Have we identified...

Business Rule Templates

When do BI projects that require new data we know anecdotally that once we have acquired the required data from the System of Records the majority of our time and risk is applying code that transforms and/or augments data. In my terminology I call this "doing bad...

The death of the 100 page BI Strategy

As a Stakeholder or Product Owner
I want to understand where the BI strategy fits into the AgileBI process
So that I know when I need to have it created

Requirements

As a Developer
I want to understand how to gather requirements
So that I can actually use them to deliver something of value

BEAM meets Data Vault and wham bam thank you ma’am 

As a Developer I want to understand who I can leverage BEAM and Data Vault together So that I can develop faster and safer. Whether you are pipelining [[link]] your Agile delivery or you are managing to deliver a thin slice [[link]] every sprint iteration both should...

Information Products

As a Developer
I want to understand how to capture content reqiurements quickly
So that I can estimate their size and obtain a priority for the order to deliver