Cryptographic properties of nested functions and algebraic immunity of the Boolean function in Hitag2 stream cipher

Hitag2 is a widely applied lightweight stream cipher with a traditional structure containing linear shift feedback and nonlinear filtering. It uses a Boolean function of 20 variables as its nonlinear filter. For easy implementation, this function is constructed by a two-layer composition of one 5-variable Boolean function and five 4-variable Boolean functions. In this paper, the concept of nested function is extracted from the construction of the two-layer Boolean function in Hitag2. Then we study some general properties of nested functions, such as balancedness, algebraic degree, Walsh spectra and algebraic immunity. We prove that the Walsh spectra of a nested function can be split into a product of the Walsh spectra of its subfunctions and generating function when the subfunctions are all balanced. Moreover, two upper bounds on algebraic immunity of nested functions are proposed. By using a hybrid approach of logical reasoning and computer computation, we obtain the precise value of the algebraic immunity of the filter function used in Hitag2, which is equal to 6.