Tim Peterson's BI Performance Blog: Using Local Cubes

Personal Data Marts

Tue, 20 Nov 2007 02:05:46 GMT

I have just written a new paper called -

Supplement Your Microsoft Business Intelligence Strategy with the Fast Performance and Excellent ROI of Personal Data Marts

It's available at http://www.cubeslice.com/personaldatamarts.htm

Here's a summary of my paper. You should read the whole thing, but if you don't have the time, here's what you're missing:

1. OLAP should be fast. OLAP is at its best OLAP browsing results are returned in less than one second, and very rarely in more than five seconds.

2. Personal data marts are an effective, but under-used strategy for delivering fast OLAP.

3. A personal data mart is a collection of local cube files customized for the needs of an individual user.

4. Personal data marts would be used more often if people knew about their benefits and had a convenient way to create local cube files.

5. Here's a list of specific situations where personal data marts can greatly improve cube browsing speed:

Cubes have one or more large, flat dimensions
Cubes use complex calculated members
Individual users are using a small portion of the cube
Users want to see an odd subset of the cube
Cube browsing speed is usually adequate, but is slow for some users and at some times.

6. There are other strategies which may also take care of these performance issues:

Add MOLAP aggregations
Set attribute relationships in hierarchies
Add partitions
Optimize calcualted members
Use more powerful hardware
Teach users to avoid problem browsing areas
Simplify the cube

7. Here are the most important issues in calculating ROI for a personal data mart:

If users need OLAP cubes, they need fast browsing OLAP cubes
There may be a variety of ways to improve performance, some cheap, some expensive
There are times when personal data marts provide the best performance for a resonable cost

8. Conclusion - The goal of using personal data marts is to make OLAP fast – convenient, easy, and effective.

Fast cubes make for happy users. If there are happy users, there will be more users. And with fast cubes, each of those users will be able to find more insights to improve the organization.

And here are some quotes, supporting the need to be concerned about OLAP performance:

From Nigel Pendse, The OLAP Report
"Slow query performance has been consistently the most serious product-related reported problem, and for the last few years it has been the single most often complained of problem."

From Elizabeth Vitt, Microsoft SQL Server 2005 Analysis Services Performance Guide
"Query performance directly impacts the quality of the end user experience. As such, it is the primary benchmark used to evaluate the success of an OLAP implementation.”

From Gabhan Berry, Build Better Cubes: Real-Life Advice on Building Analysis Services Cubes
"Every business intelligence solution will have its problems with data scalability. It’s inevitable. Almost always, the source data will increase in size over time. What you have today may perform adequately but this may not be true next month or next year. As technology and hardware has improved, this problem has been alleviated but not solved; the limits have simply been moved, not extinguished. Building a cube that uses all data for all time, and where the data increases over time, is a recipe for a cube that will eventually be too slow to use.”

Hubs, Spokes, and the Personal Data Mart

Tue, 16 Oct 2007 17:34:24 GMT

Rick Sherman has an article in the October 2007 issue of DMReview which ties in with the topic I wrote about yesterday. Sherman argues in favor of the traditional view of having a data warehouse as the hub and multiple data marts as spokes coming from that hub. He argues that this structure provides the most efficient way to provide data for each business user.

You can read his article at: http://www.dmreview.com/article_sub.cfm?articleId=1093535

Sherman says the following:

"The data marts franchised from the DW start the process of packaging data for business consumption. But why end there? Why not extend this approach to have the data marts become hubs for creating OLAP cubes or submarts for providing performance management, reporting and business analytics?"

This is the perspective that we have been trying to encourage in our Business Intelligence practice. From our perspective, the ideal structure for a Microsoft BI project is as follows:

1. Use SQL Server Integration Services to bring data into a data warehouse from a variety of source systems, which may be both inside and outside the organization.

2. Build one or more star schemas from the data in that data warehouse, so that the data is efficiently organized for multidimensional analysis.

3. Create Analysis Server cubes from the star schemas.

4. Create sets of local cube files for individual users - local cubes which become personal data marts, optimized with the exact data that each individual user wants to see.

Personal data marts can have much quicker querying time, because they only have data needed for a particular set of queries. If you create a local cube that has sales for one sales representative out of a thousand and data for the most recent month instead of the past five years, you can give a user the ability to experience almost instantaneous response as they browse their cubes. There are times when the personal data mart is inadequate and users will want to browse a larger set of data. But for day-to-day use, personal data marts provide the fastest possible querying - which is a tremendous benefit to the users.

Microsoft Office PerformancePoint Server 2007 - Going Offline with Local Cubes

Sat, 06 Oct 2007 21:19:21 GMT

Microsoft has recently announced the release of the Office PerformancePoint Server 2007 - "an integrated performance management application that allows business decision makers to be in control."

PerformancePoint Server is a next generation Business Intelligence tool, allowing users at multiple levels of an organization to be involved with planning, monitoring, and analyzing events.

From my perspective, one of the most interesting features of PerformancePoint Server is that it uses local cube files. Users can be given permission to do work on their assignments while they are off-line. Later, when they come back on-line, their data is synced with the Server. This off-line capability uses local cube files to store multidimensional data locally.

In my opinion, local cube files are an extremely under-utilized tool in the Microsoft Business Intelligence world. One reason for this lack of use is the difficulty in creating them. The most common way of creating local cube files, using the Create Global Cube command, has many shortcomings:

1. It does not provide enough options for limiting the objects included in the local cube.

2. It does not allow the user to remove unused members from the attributes of the dimensions. Removing unused members can reduce the size of a local cube from 50-90% and sometimes over 99%. This makes a tremendous difference in local cube creation time.

3. In some situations, the data in the local cube does not match the Analysis Server cube.

4. In some situations, the local cube creation fails.

These limitations are overcome in CubeSlice by using ASSL (Analysis Services Scripting Language) to create local cubes.

I think if local cubes were made more available, they would be used in all kinds of applications - giving users the ability to work in a disconnected environment, giving users subsets of OLAP data needed for particular purposes, giving users fast browsing cubes containing the data that is most important to them.

Hopefully, the PerformancePoint Server will lead the way toward a broader use of local cubes!