MUSE: a high-performance metadata storage engine for cloud storage system. In the cloud storage systems, the accesses of massive small files metadata will generate a large number of random disk I/O requests, which will become the performance bottleneck of the entire storage system. In this paper, metadata unit storage engine (MUSE), a kind of metadata storage engine for cloud storage system, is proposed to support massive small files storage with high performance. Firstly, LevelDB, a high speed key-value storage engine based on LSM-tree, is used as underlying physical storage module. Secondly, LevelDB is enhanced by introducing multiple buffer tables and multiple compaction threads, which take full advantages of memory and multi-core processor. Thirdly, a new metadata accesses scheduling mechanism on multiple I/O channels is proposed. Channel is an independent data storage pipe formed by binding the independent thread to the independent physical disk. In this way, the access operations are isolated between channels, and then the aggregation of multiple channels can provide high concurrency random I/O. In addition, MUSE proposes two namespace management strategies: split-path mapping strategy and absolute path mapping strategy, aimed to make trade-off according to different application scenarios by users. Benchmarks show that MUSE can support the massive small files storage scene and outperform other metadata storage systems.
References in zbMATH (referenced in 1 article )
Showing result 1 of 1.
- Duan, Hancong; Xiang, Xiaoke; Lu, Pengcheng: MUSE: a high-performance metadata storage engine for cloud storage system (2016)