Consider this work nightmare: A staff member notices that the response time of your data warehouse is sluggish ... and it's getting worse. Then, the phones start to ring, with users asking why their queries are taking so long to run. You know what has happened - somewhere, someone created an ad hoc query that's caught in a runaway loop.
"That's what scares administrators the most," says Philip Russom, senior manager of TDWI Research. "An ad hoc query with a loop that never completes will bring a data warehouse to its knees. Administrators see the offloading of potentially problematic workloads as very positive."
That, in a nutshell, is why data warehouse appliances have become so popular in recent years. "We're seeing the trend toward appliances because companies need access to insights in operational data as soon as possible, which means near real-time," says Ram "Krish" Krishnan, CEO of analyst firm Sixth Sense Advisers, who also has a blog on the BeyeNETWORK. "That's driving the industry to start looking at black-box solutions, such as appliances, to provide that kind of availability."
To understand how such an appliance works, consider the highway. Vehicles of all sizes fill the lanes, with trucks slowing the progress of cars and motorcycles. Then, think about moving the trucks into their own special lane — that is, offloading expansive queries to a data warehousing appliance. The queries that are like trucks, requiring a great deal of space, or processing power, get moved to their own lane so they don't impede the smaller vehicles. When you move each vehicle size out of the other's way, they both move more quickly.
Several converging trends are also making appliances more popular. First, there is the widespread availability of Linux, the open-source variant of Unix. "Most companies customize the file system of the operating system to manipulate and speed up the movement of data," says Krish.
In addition, as processes become more digitized and integrated, companies are storing increasing amounts of data. "A few years ago, data marts never held more than a half terabyte of data," says Russom. "Now, they're commonly twice to 10 times that size. This huge explosion of data is driving users toward appliances."
Companies are also looking for more insights into their customer base. "They want to segment their customers by demographics, by what they're spending, by what they're buying," Russom says. "By doing that, if you're lucky, you can have the data to determine what customers and products are most profitable. But those large multidimensional queries demand a lot of processing power, so it makes sense to put them off on the appliance."
 |
 |
|
Next Article » Tackling the Challenge of Real-Time Data Flows |
