Enabling self-service BI with Modern Data Architecture

Businesses are evolving towards Self-Service Business Intelligence (SSBI) environments. But what does it exactly mean? Why do traditional BI-environments need to be reviewed and adapted to accommodate these requirements? In this short blog post we outline some of our key learnings.

The move to self-service

Today business managers like to create their own data products. A popular example is the creation of new dashboards using drag-and-drop interfaces. Tools like Microsoft Power BI, Tableau and Qlik Sense can create dynamic and interactive dashboards quickly. Some business users and managers are more ‘data literate’ than others, and for them creating the right reports comes naturally. But it can go wrong when reports are based on low quality or incomplete data models. In an exploratory phase it is fine to experiment with dashboards. But for business critical dashboards data quality is key.

Balancing self-service and managed data products

Not all areas are suited for SSBI, some of them require a standardized format. IT plays an important role in developing professional reports and dashboards while maintaining the infrastructure above which SSBI solutions operate. It’s recommended that IT creates reports that are widely used across the organization to maintain standardization.
For the self-service part, IT can focus on delivering standardised datasets, which business users can then use to build their own reports and dashboards, or even analytical models.

The traditional BI environment
Traditional BI environments make data available through data warehouses and data marts. These databases are filled by ETL-processes (Extract, Transform, Load). Data is extracted from the different source systems. Through a series of steps it is transformed to conform with a predesigned model. Finally it is loaded into that model. Building a data warehouse or data mart is project-based and time-consuming. Tomorrow the business will already need access to new information. So the big issue is that these BI environments are not agile enough to adapt.

The modern BI environment
In a modern BI environment, both structured and unstructured data is supported (and everything in between!). Typically the data warehouse is complemented with a data lake approach. A Data Lake is used to store data “as-is”, without enforcing a structure upfront. Data lakes are more flexible in ingesting data, and refining it in different ‘zones’.
Modern architectures also support a combination of batch data, and real-time data. The end user gets a seamless experience with up-to-date data, not having to wait for overnight processing to get the latest view.

Investing in data governance
In a SSBI environment it is critical to create a strong data governance framework. This framework should include master data management, metadata management, and data quality management. SSBI solutions demand a higher level of engagement and ownership from the business as well. To create a SSBI environment, it’s important for business and IT to be very well aligned with each other.
Business groups access and analyse specific data through SSBI solutions. It is the job of IT to ensure that data is accurate, secure, and up-to-date. They need to check data quality and security on a daily basis.
This is where a Data Steward enters the picture. He or she is the person who knows where different responsibilities lie, and what data means to the business. The Data Steward acts as a liaison between IT and business departments of an organization. A data steward has to collaborate with data architects, BI developers and ETL-designers. When a business manager creates a valid reporting model, the data steward can promote this model as a reliable source. Other users can then reuse this model. In the long run this approach lowers the chances of misinterpreted data, and different departments comparing apples with oranges.

Key Requirements for producing, managing and governing data

  • Create a strategy for information management
  • Create the right organisational structure to produce and govern data
  • Nominate, standardise and define data to be managed / governed
  • Create the right processes to produce, manage and govern data
  • Define policies and policy scope to manage / govern specific data
  • Follow a methodology to get your data under control
  • Use technology to implement policies and processes to manage and govern data
  • Produce and publish trusted data and services for others to consume

Build a data strategy to become “Data Driven” with information producers & consumers

Need to make use of:

  • A business glossary and information catalog
  • Re-useable services to manage and process data
  • Collaboration to manage, process and rate data
  • Role-based data management tools aimed at IT and Business

Enabling self-service with agile delivery
Changing to an Agile Business Intelligence environment will automatically drive its users to adopt Self-Service BI. Using Agile methodology, a product is delivered in shorter development cycles with multiple iterations. Each iteration creates a working software version and can be deployed to production. In an Agile development environment, IT and business work together to refine the business needs during each iteration. This increases user adoption by focusing on the evolving needs of the business user, leading to a better user experience & engagement with the end-user.

LoQutus Self-Service Analytics & Insights
We help businesses kickstart their analytics & insights journey. LoQutus methodology stands for ‘Architect, Assist, Assure’ for a reason: Our experts help you to identify the key business challenges. We help you to prepare the data, and prototype the solutions you need. This can be a report, a dashboard, a data story, a machine learning algorithm, … anything really! Afterwards we guide your organisation in implementing the chosen solutions and embedding them in your current IT landscape. Last but not least our team helps create a self-service culture through training and coaching of your business users.

LoQutus can help your organisation to:

  • Identify and locate the relevant data sources in your organisation.
  • Combine data from the different sources into a central model where up-to-date data resides in a format easily accessible for analysis.
  • Quickly create a first iteration for dashboards based on our experience
  • Assess and implement machine learning algorithms to add intelligence
  • Discuss with stakeholders to iteratively improve and fine-tune the solution
  • Organise trainings and coaching for end users
  • Set up the required governance for self-service

If you would like to hear more about how to make self-service analytics and insights work in your organisation, let us know!

Pieter-Jan Serlet

Data Engineer @LoQutus