Join Newsletter
Trusted Business Advisors, Expert Technology Analysts

Taneja Blog

Taneja Blog / Data Center Systems

Visualizing (and Optimizing) Cluster Performance

Clusters are the scale-out way to go in today's data center. Why not try to architect an infrastructure that can grow linearly in capacity and/or performance? Well, one problem is that operations can get quite complex especially when you start mixing workloads and tenants on the same cluster. In vanilla big data solutions everyone can compete, and not always fairly, for the same resources. This is a growing problem in production environments where big data apps are starting to underpin key business-impacting processes.

Pepperdata was formed to help deliver consistent big data application performance. They lay in their solution into Big Data clusters (i.e. YARN/Hadoop, Spark...) and then can dynamically optimize runtime operations - tuning and tweaking at sub-second intervals to help guarantee required QoS to key workloads.

Now first, Pepperdata had to dig in and produce a fine grained understanding of who is doing what, when and where on these big data clusters. Many folks first really appreciate just the visibility they now can get of their cluster utilization - then they start to realize the dynamic optimization they get which can greatly improve performance and reduce total CAPEX (which can be large on those big clusters). This week, Pepperdata has grown another related capability by demand - support for chargeback/showback on those big shared clusters.

Pepperdata's value proposition is easy to justify against the infrastructure savings, but the opportunity to do more with existing cluster infrastructure, and provide consistent performance for business processes that depend on it, really makes it a must-have - a checkbox requirement for Hadoop clusters.

I think Cluster Operations Management (and all the sub-capabilities that includes) is going to be the next hot area of focus for many IT shops. Look for our upcoming Taneja Group report on cluster operations optimization featuring Pepperdata, and let us know of your trials and tribulations (and sucesses) in managing large big data clusters in production

Bookmark and Share
  • Premiered: 09/29/15
  • Author: Mike Matchett
Topic(s): Pepperdata Big Data Cluster Management Chargeback Hadoop Spark Performance Optimization


There are no comments to display. Scroll down to leave your own!


Leave a Comment

You must be logged in to comment. Click here to log in or register if you don't have an account.