Skip to main content
  • Conferences
  • Students
Sign in
Gold Sponsor
Gold Sponsor
Gold Sponsor
Silver Sponsor
Bronze Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Media Sponsor
Industry Partner

USENIX ATC '15 button

Get more
Help Promote graphics!


  •  Twitter
  •  Facebook
  •  LinkedIn
  •  Google+
  •  YouTube
Tweets by @usenix
  • Event Code of Conduct
  • Conference Network Policy
  • Statement on Environmental Responsibility Policy
Tweet

connect with us

Authors: 

Francis Deslauriers, Peter McCormick, George Amvrosiadis, Ashvin Goel, and Angela Demke Brown, University of Toronto

Abstract: 

Cluster computing frameworks such as Apache Hadoop and Apache Spark are commonly used to analyze large data sets. The analysis often involves running multiple, similar queries on the same data sets. This data reuse should improve query performance, but we find that these frameworks schedule query tasks independently of each other and are thus unable to exploit the data sharing across these tasks. We present Quartet, a system that leverages information on cached data to schedule together tasks that share data. Our preliminary results are promising, showing that Quartet can increase the cache hit rate of Hadoop and Spark jobs by up to 54%. Our results suggest a shift in the way we think about job and task scheduling today, as Quartet is expected to perform better as more jobs are dispatched on the same data.

Francis Deslauriers, University of Toronto

Peter McCormick, University of Toronto

George Amvrosiadis, University of Toronto

Ashvin Goel, University of Toronto

Angela Demke Brown, University of Toronto

Open Access Media

USENIX is committed to Open Access to the research presented at our events. Papers and proceedings are freely available to everyone once the event begins. Any video, audio, and/or slides that are posted after the event are also free and open to everyone. Support USENIX and our commitment to Open Access.

BibTeX
@inproceedings {196370,
author = {Francis Deslauriers and Peter McCormick and George Amvrosiadis and Ashvin Goel and Angela Demke Brown},
title = {Quartet: Harmonizing Task Scheduling and Caching for Cluster Computing},
booktitle = {8th USENIX Workshop on Hot Topics in Storage and File Systems (HotStorage 16)},
year = {2016},
address = {Denver, CO},
url = {https://www.usenix.org/conference/hotstorage16/workshop-program/presentation/deslauriers},
publisher = {USENIX Association},
month = jun
}
Download
Deslauriers PDF
View the slides
  • Log in or register to post comments
  • Privacy Policy
  • Contact Us

© USENIX
EIN 13-3055038