Machine Learning for Resource Management

The practice of collecting big performance data has changed how infrastructure providers model and manage the system in the past decade. There is a methodology shift from domain-knowledge based models, e.g., queuing and simulation, to data-driven models, e.g., machine learning. I will present such a game change for resource management from workload characterization, dependability prediction to sprinting policy, with examples from IBM datacenters. I will conclude the talk with future directions of performance models and challenging resource management problems in machine learning clusters.