Data Warehouses: Do you need to take better care of yours?
Data warehouses. Remember them? They were big in the 1980s. Literally. Big machines (some even needed workers on rolling folding ladders to service them), big space, and all in all a big investment. However, these early data warehouses were revolutionary and innovative in their own clunky way. They changed how we use data and do business, and they were the precursor to today‘s business intelligence ecosystem.
Unlike many things from the 1980s (hello parachute pants), data warehouses are still around. More refined, faster, leaner, they’ve become so integral to business operations that we can forget they’re there. Layered under the guise of “big data“ and/or business intelligence, these data storage areas supply the reports, statements, and analytics decision-makers need to keep a business competitive.
So, great! Data warehouses, now called enterprise data warehouses (EDWs), are still here. Why should you care?
Because EDWs and how you use them can greatly impact your firm’s infrastructure and operations.
They’re critical to successful upgrades, integrations, implementations, and customizations. How they’re designed, developed, and maintained can affect your firm’s day-to-day operations and its long-term success.
Data Warehouses: Where They Started
Data warehouses were constructed to aggregate large amounts of data into a single repository from which reports and analytics could be generated. Before data warehouses, there were no efficient ways to do this type of data mining. Even so, building these repositories was a huge investment in time, dollars, and resources for many companies which delivered highly-inflexible, virtually unchangeable configurations for static reporting purposes. Since data warehouses contained highly valuable information, costly investments, and limited functionality became the norm. It was better than nothing.
Next Step: Data Federation Technology
With the introduction of Big Data (a term coined in 2005), organizations needed an alternative solution to these slow, rigid data warehouses. They needed the flexibility to extract the information they wanted when they needed it and in a useful format. Sound familiar?
This is where data federation technology comes in. This iteration of data warehousing provided stand-alone systems that served as universal repositories from which data could be referenced. Agnostic to location, format, and structure, these systems eliminated the need to invest in data association, normalization, transformation, and modeling. Furthermore, with the ability to tap into existing data warehouses, these solutions not only improved operations by delivering insight instantaneously but they also leveraged existing systems to extend the investment previously made. This made data warehousing a viable tool and brought it forward into the 21st Century. The model is still used today.
Where We Are Today
By now, you’ve deduced that every organization – your firm included! – has its own data warehouse. Hence the “enterprise” in EDW. Technology developments over the last 15 years make it possible for EDWs to be as common as staplers in any organization. Whether it’s onsite, offsite, in the cloud, or some hybrid configuration, storing precious data in one accessible environment is now the norm.
Why You Should Care
Now for the part you’ve been waiting for: why data warehousing needs to be on your radar.
Your firm’s data and EDW are fluid. Not only does the information in the EDW change, but the technology around it changes. This fluidity makes data extractions and transformations a necessity. Data structures change; database technologies advance; extraction languages (such as “R”), applications (e.g., 3E, ProLaw, Enterprise, Rippe, etc.), storage techniques (SSD, SAN, NAS, etc.), and disparate data sources multiply; etcetera. The list goes on. To put your head in the sand and ignore/avoid this dynamic landscape not only jeopardizes the integrity of your data, but it also puts your firm in peril. You open the door to security breaches, viruses, sluggish performance, and plain old inefficiencies.
Does this mean you need to become a data warehouse expert? No. It does, however, mean you need to know what to look for in technology vendors, especially those performing data extractions and transformations. Viable extractors and managed services teams keep a constantly growing test case library to address the many different applications and scenarios used by clients. They’re constantly developing qualified staff as well as keeping up with the latest trends and industry application data structures. They have established processes for documenting their work and communicating with your staff. In other words, your vendor needs to be as dynamic as your data warehousing environment.
Conclusion
Twenty years ago, on-demand business data was a novelty. Now, it’s a given. This means that data warehouses aren’t going anywhere. If anything, they’re becoming more sophisticated and more ingrained in our business culture. Going forward, companies that understand and tend to their EDWs will be on the cutting edge.
Need more information on data warehousing and EDWs? Interested in having our professionals review your EDW? Our Data Extraction and Managed Services teams are happy to talk to you. Contact them today!