Design of persistent embedded main memory databases on non-volatile memory

With the increasing complication and variation of embedded applications in recent decades, both industry and academia have been proposing to use main memory databases to meet the ever-growing demand of high-performance data processing in embedded systems. Nevertheless, the existing main memory databases all rely on the secondary storage, such as magnetic disks and SSD, to maintain the real database persistently. Moreover, they have large overhead in synchronizing the data between memory and storage via slow I/O operations. Even though they are deployed in the emerging Non-Volatile Memory (NVM), the existing main memory databases are unable to get rid of disks for their temporary in-memory data structures cannot survive system reboot. In solving the persistency problem of main memory databases, this paper proposes to manage the NVM by the database itself that is independent from the memory management system. The information of NVM and the metadata of the database are persistently fixed on the NVM. The proposed ideas are implemented in an open-sourced main memory database, Redis. The experimental results show that compared with the existing persistency scheme of Redis AOF, the proposed approach improves the speeds of starting database, closing database, set operations, and delete operations by 2400 times, 5 times, 58 times, and 34 times, respectively. © Copyright 2016, Institute of Software, the Chinese Academy of Sciences. All rights reserved.