Héron: Taming Tail Latencies in Key-Value Stores Under Heterogeneous Workloads

Avoiding latency variability in distributed storage systems is challenging. Even in well-provisioned systems, factors such as the contention on shared resources or the unbalanced load between servers affect the latencies of requests and in particular the tail (95th and 99th percentile) of their distribution. One effective counter measure for reducing tail latency in key- value stores is to provide efficient replica selection algorithms. However, existing solutions are based on the assumption that all requests have almost the same execution time. This is not true for real workloads. This mismatch leads to increased latencies for requests with short execution time that get scheduled behind requests with large execution times. We propose He ́ron, a replica selection algorithm that supports workloads with heterogeneous request execution times. We evaluate He ́ron in a cluster of machines using a synthetic dataset inspired from the Facebook dataset as well as two real datasets from Flickr and WikiMedia. Our results show that He ́ron outperforms state-of-the-art algorithms by reducing both median and tail latency by up to 41%.