Optimality conditions for degenerate extremum problems with equality constraints

In this paper we consider an optimization problem with equality constraints given in operator formas F(x) = 0, where F : X --> Y is an operator between Banach spaces. The paper addresses the case when the equality constraints are not regular in the sense that the Frechet derivative F'(x*) is not onto. In the first part of the paper, we pursue an approach based on the construction of p-regularity. For p-regular constrained optimization problems, we formulate necessary conditions for optimality and derive sufficient conditions for optimality. In the second part of the paper, we consider a generalization of the concept of p-regularity and derive generalized necessary conditions for optimality for an optimization problem that is neither regular nor p-regular. For this problem, we show that the tangent cone to a level surface of F can consist of rays (rather than lines). This is in contrast to the regular and the p-regular cases, for which the tangent cone is always "two-sided." We state that if the gradient of the generalized p-regular problem is nonzero, it can belong to an open set, despite the fact that all constructions are usually closed. Both p-regular and generalized conditions for optimality reduce to classical conditions for regular cases, but they give new and nontrivial conditions for nonregular cases. The presented results can be considered as a part of the p-regularity theory.