The operating system is the foundation and core support technology of modern computing platforms, responsible for managing hardware resources, controlling the operation of programs, improving the human-machine interface and providing support for application software. Its connotation and extension are constantly expanding with the development of applications and hardware. The scientific aspects of operating systems fall into two categories: The first is the efficient abstraction and management of physical resources; the second is the provision of an efficient operating environment for applications. In the last decade and the period ahead, the specific connotation of the scientific problem is how to provide efficient abstraction and management of physical resources such as heterogeneous cores and data centers, to create efficient operating environments to support application scenarios such as cloud computing, big data and the Internet of Things. Because of the importance of the operating system, the security capability of the operating system is critical to the security of the entire system. The security of the operating system is the security pillar in mobile platforms or cloud platforms. Similarly, in many emerging scenarios, such as the industrial internet, smart networked cars and serverless computing, computer systems' security is related to data and property security, and possibly production and life safety. Therefore, the need to enhance the security capability of the operating system against software and hardware attacks remains urgent in the face of various security threats and multi-dimensional security vulnerabilities. It requires the design of system software to take into account chip TEE security, virtualization security, system kernel security, and application system security. This paper presents our team's work on the innovation and application of operating system security from the above aspects. Specifically, in the aspect of TEE on-chip, we propose Penglai that can offer trusted execution environments for securitysensitive computation. Penglai is built over the emerging RISC-V architecture and its core feature is scalability in both memory capacity and performance. In the aspect of virtualization security, we design CloudVisor which introduces a new system architecture for the virtualization software stack. CloudVisor can offer the abstraction of secure VM against curious or malicious hypervisor and cloud providers. In the aspect of kernel security, we build ChCore which uses the microkernel design and introduces some new mechanisms for reliability. As a microkernel OS, ChCore can greatly mitigate the consequences of security vulnerabilities and it can also work as an TEE OS to cooperate with the TEE hardware technologies. In the aspect of application system security, we propose PiXiu which can provide security guarantee for distributed applications. PiXiu targets on securing sensitive computation, especially for the distributed computation, and it assumes a powerful attack model. Overall, all of the above-introduced research adopt the hardware-software co-designs, and they can fuse with each other to offer a full-stack security system. Meanwhile, the paper will also provide an overview of the representative academic work in each aspect, which includes the comparison of different technical contributations.
