Gathering Data Requirements
I want I want to understand how to make retrospectives fun
So that I can keep the sprint team engaged
The Data Requirements Gap
There are two typical challenges when we gather data requirements. First, we capture and document the requirements in a way the person capturing them understands. Second, we explore and model the required data structures solely based on the system of record data structures or the users reporting requirements.
Document Hand Over
Let’s look at the way we capture data requirements. We will typically capture our data requirements in a traditional application-centric requirements document which is used to document the requirements for a report or a dashboard. This is will typically have a list of fields required, plus a specification of layout and a bunch of functional and non-functional requirements.
Once completed the Business Analyst (BA) will hand this document over to a data designer or modeller who then has to understand what the fields mean and where to find them.
Of course, the document typically holds minimal data context and the data modeller will either have a guess or engage with the key users to ask them more questions to discover the context.
This results in a gap between what the key users asked for, what the BA documented and finally what the data modeller designed for.
Data Driven or Report Driven Modelling Approach
When we first started delivering data using a Data Warehouse approach, we often took a data-driven method. We would spend a long time looking at the data in the Systems of Record and modelling how we could land and store this data to make it easier to use in the future. The problem with this approach was it took a long time as we often spent time understanding and modelling data that a user may never consume, just in case it was needed in the future.
Then we moved to a report driven method. We would document the content required by the users and then acquire and model the data to meet only these reporting needs. This was often faster to deliver, but the problem was we often ended up with a large number of disparate reporting data sets that overlapped and were hard to manage.
Both of these methods lead to a gap in expectation of the time it took to deliver data and content and the ability to quickly add or change this data in the future.
Event Driven Approach
Business Events are events that take place in the course of the normal operation of a business and that reoccur as business processes are executed.
What is BEAM✲?
BEAM✲ stands for Business Event Analysis & Modelling, and it’s a methodology for gathering business requirements for Agile Data Warehouses and building those warehouses. It was developed by Lawrence Corr(@LawrenceCorr) and Jim Stagnitto (@JimStag), and published in their book Agile Data Warehouse Design: Collaborative Dimensional Modeling, from Whiteboard to Star Schema (Amazon, eBook).
There are three parts to the BEAM method that provides the Corr of its value:
- Business Event Driven
You identify data requirements based on Business Events.
- Bridging the language gap
The data requirements are documented in a
langaugeand a template that can be understood by business users and developers alike. Modelstorming
You gather these requirements using a
modelstormingapproach that is interactive and fun.
One of the benefits of documenting data requirements as Business Events is that an organisations Business Events will not change often. In fact, they tend to markedly change only when the organisation’s core business model changes.
For example, a company that makes Ice Cream will still be making Ice Cream even when they introduce new Ice Cream products, open’s in new markets or targets new types of customers. However the Systems of Record that they use to capture the data and the types of analytics and reporting content they need will change over time. If the organisation does indeed change the core of what it does, for example, starts delivering health care, then it becomes obvious that a major change has happened then the impact to the data requirements will be major as well.
Bridging the Language Gap
We will typically ask the key business users BI specific questions that we understand. For example, what are the key dimensions of the business data or what is the “grain” of the data? We understand these because we have worked with them for years, our business users have not and this leads to either the business users or the developers having to translate.
Good requirements need both developers and business users to speak a common language and part of BEAM’s magic is that it gives business users and developers this common language. For example, the business event has a Customer who Purchases a Product. Business users understand this concept, there are two business aspects; Customer and Product and the interaction between them. Developers can also understand this.
This enables us to ask the question “what events happen in your business?” which is a much easier way to start gathering requirements than “what entities do you want to report on?” Following up
BEAM✲ relies on 7 Ws (or really 5 W’s and a couple of H’s); who, what, when, where, why, how and how many.
These simple questions enable you to gather all the data details you need to model and build. You take a business event like Customer Purchases Product and ask these questions to understand the aspects which are important to that process.
- Who? – In this case, we have Customer and probably an Employee who are involved in this event;
- What? – The Product is one of our whats, it’s the item that’s involved;
- When? – The date/time the purchase happened;
- Where? – The store the purchase occurred in.
- Why? – This can be interpreted in different way’s, for example, it was a hot day;
- How? – The key piece of data that tells me this actually happened, for example, OrderID or Purchase Receipt;
- How Many? – These are the values that you need to know. e.g. How many products were purchased and what were their costs.
Using this methodology, you can understand the data behind every business event. It gives you a structured set of questions to document the requirements you need to model your data warehouse. Even better Lawerence Corr has developed a set of Open Source Excel templates that allow you to document these data requirements in an easy and consistent way. You can download the BEAM templates from here.
?? collaborate by visualising
??? populate by examples
As a developer I want I want to understand how to use wireframing to confirm requirements So that I can build the right thing Wireframing is a technique we use to confirm two things: How does the user wish to see the content presented? Have we identified...
When do BI projects that require new data we know anecdotally that once we have acquired the required data from the System of Records the majority of our time and risk is applying code that transforms and/or augments data. In my terminology I call this "doing bad...
As a Stakeholder or Product Owner
I want to understand where the BI strategy fits into the AgileBI process
So that I know when I need to have it created
As a Developer
I want to understand how to gather requirements
So that I can actually use them to deliver something of value
As a Developer I want to understand who I can leverage BEAM and Data Vault together So that I can develop faster and safer. Whether you are pipelining [[link]] your Agile delivery or you are managing to deliver a thin slice [[link]] every sprint iteration both should...
As a Developer
I want to understand how to capture content reqiurements quickly
So that I can estimate their size and obtain a priority for the order to deliver
As a Developer I want to understand how to document business rules So that we can increase the speed of delivery Another of the more difficult capabilities in data warehousing is the definition and...
One of the more challenging areas in Business Intellegence and Data Warehousing is understanding how to apply tests to data that has been transformed, to prove we have delivered a correct result. It can be as simple as proving that we are presenting the right revenue,...
With the death of the 100 page Business Intellgience strategy how do you get your stakeholders to come on the journey and agree to invest in the time for your AgileBI to create new data and content for them. With a waterfall project the process is well know, you do...
As a Developer I want to understand when and how to understand the source data So that I can be sure I can build the things in the users stories and as an input to the estimate of the effort required Often the majority of the AgileBI team are technically...
As a BI practitioner, I want to understand how to set expectations and limit the scope of a set of iterations, So that I have some boundaries which means we will not have endless iterations or additions to requirements, without agreeing these changes upfront. [[...
[[why you need to do it]] [[stakeholder onion]] [[surprise stakeholders and their new requirements]] [[identifying who should be keep informed]]