The Aggressive Oversubscribing Scheduling for Interactive Jobs on a Supercomputing System

As interactive usages of supercomputing systems become popular, especially in the AI and machine learning (ML) fields, the systems are expected to provide resources in real time. As interactive jobs have different features from traditional batch jobs, the systems should be designed to accept both types of jobs efficiently. This paper shows that the aggressive oversubscribing scheduling, in which multiple jobs share computational resources regardless of job types, can effectively process hybrid jobs. This paper investigates behaviors of the real interactive jobs with fluctuating CPU utilization. And a simulation method is described, which combines existing workload trace data and data on CPU utilization. Through the evaluation, we demonstrate oversubscribing scheduling achieves a short response time for interactive jobs. Also our solution eliminates the necessity of configuring dedicated queues for job types and achieves robustness towards the change of demand of interactive jobs.