Over the years, I have built large data warehouses, small data marts and consistently written about the need for structured methodologies in building these information repositories. But more so than ever before, I have been recognizing that there is a tremendous need for exploratory or transient analysis.
Exploratory analysis is different from structured reports or OLAP tools that you may utilize against a large data warehouse; it is a precursor to building a data mart. Many times you may find that you have an idea of how different information sources can be brought together to build a powerful analytical solution, but you would rather test the value of that hypothesis before investing in a larger data mart effort.
Other times the value of analysis may be temporal and you don’t want to invest several months’ effort to build a data mart that may be useless afterward. Examples of these temporal analyses are widespread in industries from pharmaceuticals to retail. Specific use cases may be to support a new product introduction, to support a bargaining event, or experiment with a new type of analytical model.
In order to support this new analytical paradigm, a new tool set is required. In the past, these sorts of analysis would be supported with something like Microsoft Access because no other “lightweight” platform was