Data Visualization: Now you see!

One way to comprehend and analyze the exponential volume of data is through the techniques like Data Mining. Until recently, one could only view the same data in varying formats that were static and non-visual. With the evolution of Data Visualization (DV) technology, data analysts can extend their intuitive abilities to visually interpret and actively interact with data by utilizing a user interface that exploits natural abilities in visual perception. DV tools enable non-technical users to understand data trends at both a high level, without getting their hands dirty with the actual numbers, and a detailed level through the capability of drilling down to specific information. While most business analytic tools offer only static tabular/pivot reports or at most some types of charts with very little scope of human interaction, Data Visualization offers interactive graphical capabilities that range from simple scatter plots to complex multi-dimensional representations that can be sliced and diced in multiple ways in real-time. Data visualization is essentially business intelligence taken a step further by making that information more logical and easier to understand through visualization techniques. Current data visualization leaders like Advanced Visual Systems Inc. (AVS), Antarctica Systems Visual Net and Spotfire DecisionSite offer further insight into data by demonstrating relationships that occur through multi-dimensional attributes and statistical patterns. This is an evolving field that shows much promise in the coming years in the enterprise analytics arena.

2004 Research Paper on Strategic Information Systems; R.H.Smith School of Business; Flanagan, Gera, Graves, Hersi, Patil, Yarnot

Demand for Business Intelligence Expertise

The number of consulting firms offering BI and analytics solutions seems to be growing exponentially. Front runners are the major systems integrators like IBM BCS, CGI-AMS and Deloitte. One glance at their hiring postings in the recent months shows an explosion in the requirement of talent specializing in the analytics space. Professionals need to be careful about their choice of the technology to follow as their core competency. BI is the hot "thing" in today's world of exploding data content, however it is a constantly evolving industry in stages of inevitable consolidation. Business Objects' acquisition of Crystal, Cognos acquiring Frango are the recent examples of the consolidation underway in the industry. From my past experience, I think it would not be too far before the major players like Microsoft extend their offering in the analytics space through acquisitions. Oracle also might make a move soon, a move that is probably delayed with the PeopleSoft focus. The demand for talent in analytics is high, and hopefully will continue riding the wave of technology spending. To be on the safe side though, professional will find it advantageous to supplement their tool expertise with solid programming (XML, Java) to make sure that a sudden glut in the tools market does not leave them with knowledge that is no longer wanted in the market.

Data Warehousing Architecture Alternatives

Forrester Research: Technology research and advice.: "Data warehousing architecture is needed to accommodate dynamic business requirements that cannot be anticipated. Without data warehousing architecture, organizational form and IT function operate at cross purposes. Meanwhile, lack of fit between organizational form and architecture has resulted in the religious wars for which data warehousing has become famous. These are avoidable by aligning form and structure. Firms that are highly centralized in geography and governance should pursue centralized data warehouse architecture to reap the greatest operational efficiencies and business benefits. Firms that are highly decentralized will prefer a distributed architecture; those with a mixed organizational pattern should implement a federated one."

Grading BI Reporting And Analysis Solutions

Forrester Research: Technology research and advice.: "Reporting is fundamental to all companies. Most companies strive to standardize on a single reporting and analysis platform to deliver analytic, business, and enterprise reporting, but it is a reality for very few. There is no one single reporting and analysis platform that can deliver every feature and function needed. The direction that companies should take is to adopt a standard that can support at least two of the three different reporting types and fill in gaps with emerging XML and Web services capabilities, advanced visualization, and process definition whenever possible."

Don't forget your model, stupid!

Companies spend hours and hours (and of course, tons of money) on developing their dream enterprise portal with slick dashboards and interfaces for data analysis. What probably gets ignored often is the supporting data model itself.

I have seen companies that build phenomenal transactional systems that crunch a million transactions on a regular basis. Translating this transactional data to support reporting is another matter. The same transactional structures are often carried over into the reporting world (with a bit of denormalization here, and a bit of pivots there), but essentially the same transactional structure to support the reporting applications.

And therein lies the problem. You cannot expect Business Objects or Cognos or any other tool out there to scale unless you have a sound reporting data model to support them. It does not make good business and project management sense to embed the code and transformational logic into the reporting tool, when the ETL should be the source and focus of all major transformations.

I am not sure how much this lethargy to convert a transactional model to a denormalized reporting model exists out there; I will not be surprised if it is very prevalent.