KubeCon + CloudNativeCon China 2018

Benchmarking is an essential part in machine learning research and productization, that provides useful performance information from the perspectives of both models and systems. While Kubernetes and Kubeflow give us a great platform for ML workloads to run on, they do not automatically provide a straightforward way to perform benchmark tasks, especially for complex ML workloads based on distributed jobs. In this talk we present Kubebench, an open sourced benchmarking tool based on Kubeflow, that helps us better understand the performance signature of our ML workloads on Kubernetes through automated and consistent benchmarks. We also show how we can leverage other benchmarking efforts from academia and industry like MLPerf and Dawnbench.