RFS: An LSM-Tree-Based File System for Enhanced Microdata Performance

Lixin WANG; Yutong LU; Wei ZHANG; Yan LEI

doi:10.1587/transinf.2015EDP7470

RFS: An LSM-Tree-Based File System for Enhanced Microdata Performance

Lixin WANG, Yutong LU, Wei ZHANG, Yan LEI

Full Text Views

0

Share
Cite this

Summary :

File system workloads are increasing write-heavy. The growing capacity of RAM in modern nodes allows many reads to be satisfied from memory while writes must be persisted to disk. Today's sophisticated local file systems like Ext4, XFS and Btrfs optimize for reads but suffer from workloads dominated by microdata (including metadata and tiny files). In this paper we present an LSM-tree-based file system, RFS, which aims to take advantages of the write optimization of LSM-tree to provide enhanced microdata performance, while offering matching performance for large files. RFS incrementally partitions the namespace into several metadata columns on a per-directory basis, preserving disk locality for directories and reducing the write amplification of LSM-trees. A write-ordered log-structured layout is used to store small files efficiently, rather than embedding the contents of small files into inodes. We also propose an optimization of global bloom filters for efficient point lookups. Experiments show our library version of RFS can handle microwrite-intensive workloads 2-10 times faster than existing solutions such as Ext4, Btrfs and XFS.

Publication: IEICE TRANSACTIONS on Information Vol.E99-D No.12 pp.3035-3046

Publication Date: 2016/12/01

Publicized: 2016/09/06

Online ISSN: 1745-1361

DOI: 10.1587/transinf.2015EDP7470

Type of Manuscript: PAPER

Category: Fundamentals of Information Systems

Authors

Lixin WANG
  National University of Defense Technology (NUDT)
Yutong LU
  National University of Defense Technology (NUDT)
Wei ZHANG
  National University of Defense Technology (NUDT)
Yan LEI
  National University of Defense Technology (NUDT)

Keyword

local file systems, log-structured merge tree, microdata, namespace partitioning, write-ordered write-optimized store

Cite this

Copy

Lixin WANG, Yutong LU, Wei ZHANG, Yan LEI, "RFS: An LSM-Tree-Based File System for Enhanced Microdata Performance" in IEICE TRANSACTIONS on Information, vol. E99-D, no. 12, pp. 3035-3046, December 2016, doi: 10.1587/transinf.2015EDP7470.
Abstract: File system workloads are increasing write-heavy. The growing capacity of RAM in modern nodes allows many reads to be satisfied from memory while writes must be persisted to disk. Today's sophisticated local file systems like Ext4, XFS and Btrfs optimize for reads but suffer from workloads dominated by microdata (including metadata and tiny files). In this paper we present an LSM-tree-based file system, RFS, which aims to take advantages of the write optimization of LSM-tree to provide enhanced microdata performance, while offering matching performance for large files. RFS incrementally partitions the namespace into several metadata columns on a per-directory basis, preserving disk locality for directories and reducing the write amplification of LSM-trees. A write-ordered log-structured layout is used to store small files efficiently, rather than embedding the contents of small files into inodes. We also propose an optimization of global bloom filters for efficient point lookups. Experiments show our library version of RFS can handle microwrite-intensive workloads 2-10 times faster than existing solutions such as Ext4, Btrfs and XFS.
URL: https://globals.ieice.org/en_transactions/information/10.1587/transinf.2015EDP7470/_p

Copy

@ARTICLE{e99-d_12_3035,
author={Lixin WANG, Yutong LU, Wei ZHANG, Yan LEI, },
journal={IEICE TRANSACTIONS on Information},
title={RFS: An LSM-Tree-Based File System for Enhanced Microdata Performance},
year={2016},
volume={E99-D},
number={12},
pages={3035-3046},
abstract={File system workloads are increasing write-heavy. The growing capacity of RAM in modern nodes allows many reads to be satisfied from memory while writes must be persisted to disk. Today's sophisticated local file systems like Ext4, XFS and Btrfs optimize for reads but suffer from workloads dominated by microdata (including metadata and tiny files). In this paper we present an LSM-tree-based file system, RFS, which aims to take advantages of the write optimization of LSM-tree to provide enhanced microdata performance, while offering matching performance for large files. RFS incrementally partitions the namespace into several metadata columns on a per-directory basis, preserving disk locality for directories and reducing the write amplification of LSM-trees. A write-ordered log-structured layout is used to store small files efficiently, rather than embedding the contents of small files into inodes. We also propose an optimization of global bloom filters for efficient point lookups. Experiments show our library version of RFS can handle microwrite-intensive workloads 2-10 times faster than existing solutions such as Ext4, Btrfs and XFS.},
keywords={},
doi={10.1587/transinf.2015EDP7470},
ISSN={1745-1361},
month={December},}

Copy

TY - JOUR
TI - RFS: An LSM-Tree-Based File System for Enhanced Microdata Performance
T2 - IEICE TRANSACTIONS on Information
SP - 3035
EP - 3046
AU - Lixin WANG
AU - Yutong LU
AU - Wei ZHANG
AU - Yan LEI
PY - 2016
DO - 10.1587/transinf.2015EDP7470
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E99-D
IS - 12
JA - IEICE TRANSACTIONS on Information
Y1 - December 2016
AB - File system workloads are increasing write-heavy. The growing capacity of RAM in modern nodes allows many reads to be satisfied from memory while writes must be persisted to disk. Today's sophisticated local file systems like Ext4, XFS and Btrfs optimize for reads but suffer from workloads dominated by microdata (including metadata and tiny files). In this paper we present an LSM-tree-based file system, RFS, which aims to take advantages of the write optimization of LSM-tree to provide enhanced microdata performance, while offering matching performance for large files. RFS incrementally partitions the namespace into several metadata columns on a per-directory basis, preserving disk locality for directories and reducing the write amplification of LSM-trees. A write-ordered log-structured layout is used to store small files efficiently, rather than embedding the contents of small files into inodes. We also propose an optimization of global bloom filters for efficient point lookups. Experiments show our library version of RFS can handle microwrite-intensive workloads 2-10 times faster than existing solutions such as Ext4, Btrfs and XFS.
ER -