Max: A Multicore-Accelerated File System for Flash Storage

Max is a file system designed for modern multicore CPUs and high-performance SSDs. This is the source code repository of Max.

Please read the paper "Max: A Multicore-Accelerated File System for Flash Storage" of USENIX ATC'21 for more details.

If you use this work, please cite:

@inproceedings {273904,
author = {Xiaojian Liao and Youyou Lu and Erci Xu and Jiwu Shu},
title = {Max: A Multicore-Accelerated File System for Flash Storage},
booktitle = {2021 {USENIX} Annual Technical Conference ({USENIX} {ATC} 21)},
year = {2021},
isbn = {978-1-939133-23-6},
pages = {877--891},
url = {https://www.usenix.org/conference/atc21/presentation/liao},
publisher = {{USENIX} Association},
month = jul,
}

WARNING: The code can be only used for academic research. Please do not use it for production.

Contact information: liaoxiaojian@tsinghua.edu.cn

Downloading the source code

apt install git-lfs
git lfs clone git@github.com:thustorage/max.git

Building Max (be root user)

1. Build the Linux kernel

apt install -y flex bison make gcc libssl-dev bc libelf-dev
tar -xf linux-kernel.tar.gz  
cd linux-kernel
cp /boot/config-`uname -r` .config
make olddefconfig
make -j24 && make modules_install -j24 && make install

2. Update the grub configuration and reboot into the new kernel.
3. Build and install Max

cd Max
make -j24
insmod max.ko

4. Build the disk format tool

cd disk-tools
apt install pkg-config autoconf libtool libselinux1-dev
./autogen.sh
./configure
make -j24

Running Max

1. Format the drive

disk-tools/mkfs/mkfs.f2fs -N 8 /dev/nvme0n1

Note: the number 8 means the maximum number of mlogs. You can replace it with a proper number lower than 64. 2. Mount the drive

mount -t max -o imds=72,mlog=8 /dev/nvme0n1 /mnt/test

Note: the number of 72 means the number of file cell groups. the number of 8 means the number of mlogs.

Now, the Max file system is mounted at /mnt/test, storing its data on /dev/nvme0n1.

Testing Max with FxMark

1. Build FxMark

cd fxmark
make -j24

2. Build Exim

cd vbench/exim
make -j24

3. Build RocksDB

cd vbench/rocksdb
make -j24

4. Setup the test. Open the file bin/run-fxmark.py.

• Setup the tested devices in lines 30-33. An example:

HDDDEV = "/dev/sda1"
SSDDEV = "/dev/sdb1"
NVMEDEV = "/dev/nvme0n1"
LOOPDEV = "/dev/loop0"

• Setup the tested media and file systems in lines 61-75. An example:

self.MEDIA_TYPES = ["ssd","hdd","nvme",]
self.FS_TYPES = ["ext4","xfs","f2fs","max",]   

• Setup the workloads in lines 78-127. An example:

self.BENCH_TYPES = ["DWOL","DWAL","MWCL","MWUL","filebench_varmail","rocksdb_overwrite"]

• Setup the test in lines 632-635. An example:

run_config = [(Runner.CORE_FINE_GRAIN, PerfMon.LEVEL_LOW,("nvme","max","DWOL","72","bufferedio"), "", "")]

Note: the entry "nvme" specifies the medium. The entry "max" specifies the file system. The entry "DWOL" (data overwrite) specifies the workload. The entry "72" specifies the number of cores. You can set the entries to "*" to enable all media, file systems, workloads and numbers of cores. 5. Run the tests.

bin/run-fxmark.py