One-size-fits-all solutions have not worked well in storage systems. This is true in the enterprise where noSQL, Map-Reduce and column-stores have added value to traditional database workloads. This is also true outside the enterprise. A recent paper [7] illustrated that even the single-desktop store is a rich mixture of file systems, databases and key-value stores. Yet, in research one-size-fits-all solutions are always tempting and point-optimizations emerge, with the current theme du jour being key-value stores [8].

Workloads naturally change their requirements over time (e.g., from update-intensive to query-intensive). This paper proposes research around a multi-structured storage architecture. Such architecture is composed of many lightweight data structures such as BTrees, key- value stores, graph stores and chunk stores. The call for modular storage and systems is not dissimilar to the Exokernel [4] or Anvil [10] approaches. The key difference that this paper argues about is that we want these data structures to co-exist in the same system. The system should then automatically use the right one at the right workload phase. To enable this technically, we propose to leverage the existing N-way redundancy in the data center and have each of N replicas embody a different data structure.

Storage infrastructure in large-scale cloud data center environments must support applications with diverse, time-varying data access patterns while observing the quality of service. Deeper storage hierarchies induced by solid state and rotating media are enabling new storage management tradeoffs that do not apply uniformly to all application phases at all times. To meet service level requirements in such heterogeneous application phases, storage management needs to be phase-aware and adaptive, i.e., to identify specific storage access patterns of applications as they occur and customize their handling accordingly.

This paper presents LoadIQ, a novel, versatile, adaptive, application phase detector for networked (file and block) storage systems. In a live deployment, LoadIQ analyzes traces and emits phase labels learnt on the fly by using Support Vector Machines(SVM), a state of the art classifier. Such labels could be used to generate alerts or to trigger phase-specific system tuning. Our results show that LoadIQ is able to identify workload phases (such as in TPC-DS) with accuracy > 93%.

Disk contention is a fact of life in modern data centers, with multiple applications sharing the storage resources of a single physical machine. Log-structured storage designs are ideally suited for such high-contention settings, but historically have suffered from performance problems due to cleaning overheads. In this paper, we introduce Gecko, a novel design for storage arrays where a single log structure is distributed across a chain of drives, physically separating the tail of the log (where writes occur) from its body1. This design provides the benefits of logging – fast, sequential writes for any number of contending applications – while eliminating the disruptive effect of log cleaning activity on application I/O.

Multi-tiered storage systems using tiers of SSD and traditional hard disk is one of the fastest growing trends in the storage industry. Although using multiple tiers provides a flexible trade-off in terms of IOPS performance and storage capacity, we believe that providing performance isolation and QoS guarantees among various clients, gets significantly more challenging in such environments. Existing solutions focus mainly on either disk-based or SSD-based storage backends. In particular, the notion of IO cost that is used by existing solutions gets very hard to estimate or use.

In this paper, we first argue that providing QoS in multi-tiered systems is quite challenging and existing solutions aren’t good enough for such cases. To handle their drawbacks, we use a model of storage QoS called as reward scheduling and a corresponding algorithm, which favors the clients whose IOs are less costly on the back-end storage array for reasons such as better locality, read-mostly sequentiality, smaller working set as compared to SSD allocation etc. This allows for higher efficiency of the underlying system while providing desirable performance isolation. These results are validated using a simulation-based modeling of a multi-tiered storage system. We make a case that QoS in multi-tiered storage is an open problem and hope to encourage future research in this area.

In this work, we propose a new batching scheme called temporal merge, which dispatches discontiguous block requests using a single I/O operation. It overcomes the disadvantages of narrow block interface and enables an OS to exploit peak throughput of a storage device for small random requests as well as a single large request. Temporal merge significantly enhances device and channel utilization regardless of access sequentiality of a workload, which has not been achievable by traditional schemes.

We extended the block I/O interface of a DRAM-based SSD in cooperation with its vendor, and implemented temporal merge into I/O subsystem in Linux 2.6.32. The experimental results show that under multi-threaded random access workload, the proposed solution can achieve 87%∼100% of peak throughput of the SSD. We expect that the new temporal merge interface will lead to better design of future host controller interfaces such as NVMHCI for next-generation storage devices.

Exploiting internal parallelism over hundreds NAND flash memory is becoming a key design issue in high-speed Solid State Disks (SSDs). In this work, we simulated a cycle-accurate SSD platform with twenty four page allocation strategies, geared toward exploiting both system-level parallelism and flash-level parallelism with a variety of design parameters. Our extensive experimental analysis reveals that 1) the previously-proposed channel-and-way striping based page allocation scheme is not the best from a performance perspective, 2) As opposed to the current perception that system and flash-level concurrency mechanisms are largely orthogonal, flash-level parallelism are interfered by the system-level concurrency mechanism employed, and 3) With most of the current parallel data access methods, internal resources are significantly under-utilized. Finally, we present several optimization points to achieve maximum internal parallelism.

For backup storage, increasing compression allows users to protect more data without increasing their costs or storage footprint. Though removing duplicate regions (deduplication) and traditional compression have become widespread, further compression is attainable. We demonstrate how to efficiently add delta compression to deduplicated storage to compress similar (non-duplicate) regions. A challenge when adding delta compression is the large number of data regions to be indexed. We observed that stream-informed locality is effective for delta compression, so an index for delta compression is unnecessary, and we built the first storage system prototype to combine delta compression and deduplication with this technology. Beyond demonstrating extra compression benefits between 1.4-3.5X, we also investigate throughput and data integrity challenges that arise.

A “home” sharing environment consists of the data sharing relationships between family members, friends, and acquaintances. We argue that this environment, far from being simple, has sharing and trust relationships as complex as any general-purpose network.

Such environments need strong access control and privacy guarantees. We show that avoiding information leakage requires both to be integrated directly into (rather than layered on top of) replication protocols, and propose a system structure that meets these guarantees.

The TokuFS file system outperforms write-optimized file systems by an order of magnitude on microdata write workloads, and outperforms read-optimized file systems by an order of magnitude on read workloads. Microdata write workloads include creating and destroying many small files, performing small unaligned writes within large files, and updating metadata. TokuFS is implemented using Fractal Tree indexes, which are primarily used in databases. TokuFS employs block-level compression to reduce its disk usage.