Traditionally, IT experts created data assets for business users upon request. Of course, people still went rogue, capturing, analyzing, and visualizing data in Excel “spreadmarts” outside of IT, but their potential for damage was limited.
Today, as next-generation business intelligence (BI) tools become increasingly powerful and self-service enabled, and as global privacy laws and regulatory requirements increase, businesses without strong data management and governance programs face much greater risk.
What is data democratization, and how can your business ensure that self-service data asset development doesn’t trigger chaotic — and costly — consequences? Data management best practices can help you:
- Keep up with the pace of information needs outside of IT without spawning ungoverned “Shadow IT” practices
- Manage existing Shadow IT practices, particularly if your organization adds substantially more powerful BI tools to the mix
- Develop a more open data culture while also valuing privacy, security, and good governance
The solution lies in finding the right balance between increasing demands for data governance and the rapidly escalating need for data access.
What causes shadow IT — and why it can be dangerous
Growing demands for data-driven insights accelerate the demand for knowledge workers to get information when they need it and in a format they can use. As these requests for data insights balloon, IT departments quickly get backlogged.
To solve the problem, businesses sometimes turn to self-service data tools, particularly in the BI space. These tools reduce repetitive demands on IT time while enabling users to personalize how they access and view data in their own channels. Tools like Tableau and Alteryx provide rich data visualization, which further speeds time to insight.
Learn more about business intelligence (BI) options >>
While data preparation used to require highly technical skills and toolsets to extract, transform, and load information and generate reporting, data democratization puts significantly more power in the hands of average business users.
Business users can now do work that the savviest Excel-wielding shadow IT veteran never dreamed of. Flattening XML, geo-spatial custom polygon prep, blending, cleansing, creating predictive models, regressions, neural networks, and Naïve Bayes classifiers can be created and used without any traditional IT development knowledge.
But data democratization has a dark side. Businesses can get into trouble when data democratization isn’t closely paired with data governance and management. Without a carefully cultivated data culture that understands data governance and management, shadow IT’s ballistic upgrade puts businesses at risk.
Data management best practices for risk mitigation
As data democratization becomes more of a reality in your organization, data management migrates from your IT and security teams to every business unit. Implementing data management across the business requires clear communication and leadership commitment.
Audit your information ecosystem
First, take stock of your current state in terms of data intake, preparation, access, and use. Take a fair and honest look at the data management practices and acknowledge where pockets of shadow IT exist.
While Excel is obviously ubiquitous, understanding who has licenses for some of the newer tools, like Alteryx, may be a good place to start. When pockets are identified, ask some fundamental questions, like:
- What information is the business craving?
- Which tools or solutions have they tried?
- How are these tools being used?
- Is this the best tool or multi-tool solution for the job?
- Is there any overlap or duplication of assets across the business?
- What assets have they developed that could benefit a larger group or even the enterprise?
Shift your data management mindset
Then, resist the temptation to scold. The historical data management mindset toward those who created these one-off information stores needs to be turned on its head to focus on problems and solutions rather than reprimands. In light of more useful one-off data storage, you may find it hard to rationalize all of your current IT-generated assets. The cost to maintain them, particularly if they’re not actually being used, makes them liabilities, not assets.
Taking the time to define what the business needs, and then collaborating on the process, information requirements, tooling, and, potentially, infrastructure and architecture solutions that would best meet and scale to fit those requirements is a far healthier approach. Then, your company not only creates a self-service machine that can keep pace with demand, but also goes a long way toward building a healthy data culture.
How to build a strong data culture >>
Get clear on good governance
The term governance gets thrown around a lot, but does your organization have a clear idea of what you mean by it when it comes to your data? It’s not enough for IT to have documented policies and controls. A mature governance program must be seated in the business.
Once again, effective processes begin with business requirements. While IT may bear responsibility for implementing the actual controls to provide row-level security or perspectives, the business must provide definitions quality rules, lineage, and information to inform and support governed access. In this sense, IT becomes the stewards responsible for ensuring those business-driven governance requirements are met.
As your organization progresses toward data democratization, keep the following best practices in mind:
- Establish processes and workflows to bring democratized data and data assets under governance efficiently
- Co-create governance rules and standards with business units, and be sure they are communicated clearly to all data users
- Maintain governance requirements, quality rules, and access architectures that make data and data assets suitable and consumable by others within the organization
How data governance fits into strategic data management >>
Build a bridge between democracy and governance
Although bringing the creation and persistence of data assets into the controlled IT fold is critical for good governance, allowing the business to quickly and freely blend, experiment, and discover the most effective fit-for-purpose data sets for their information needs takes the burden off of IT to try to figure out what the business needs.
How do mature data organizations bridge the gap between democratized data and good governance? Workflows.
Workflows bring democratized asset development and IT-implemented controls together. A strong data workflow, including how requests are processed, prioritized, reviewed, and either approved or rejected, is the critical gatekeeper that prevents democratization from turning into chaos.
Your workflow should address:
- Data submission: Workflow is the process established by which data assets are submitted for enterprise or departmental consideration as governed assets and persisted according to IT’s self-service standards. Identifying the roles, process (inputs, outputs, gates), and relevant governance structure is fundamental to get a meaningful workflow in place.
- Data request backlog: Not every one-off dataset is an asset – the validity of the data produced must be verified to examine the lineage of the data and any transformation logic (e.g., joins, calculations) that was used in its creation.
- Data scoring: The usefulness of the data must be scored or assessed in some objective way to determine if it should be published and to whom.
- Data access and security: The workflow process should also address access and security requirements.
By streamlining the information demand management process and making it more efficient, your IT team can shift focus to providing higher-value data and information for the business, while potentially driving down cost by retiring the production of lower-value reports or marts.
Learn more about how to manage data as an asset >>
Manage change well
Shadow IT is called that for a reason. To get those datasets and those who create them to willingly step into the light is a culture shift that requires effective change management and clear communication.
Creating an environment that encourages the creation of self-service and democratized data asset development by the business is important, but, when unchecked, can result in the proliferation of potentially redundant or conflicting data sources, none of which are under IT’s purview.
Responsible development and management of all data assets within the organization requires balance, oversight, and commitment to change.
Democratizing data holds huge potential for your business when it’s well managed and governed.
Not sure where your company stands? Maybe a quick assessment could help. Our team of data experts can help you get clarity with a customized consultation, workshop, or audit designed to fit your needs.
