SYMMETRY AND NON-EXISTENCE OF SOLUTIONS FOR A NONLINEAR SYSTEM INVOLVING THE FRACTIONAL LAPLACIAN

In this paper, we consider the following system of pseudo-differential nonlinear equations in R-n { (-Delta)(alpha/2)u(i)(x) = fi(u(1)(x), ..., u(m)(x)), i = 1, ..., m, (1) u(i) >= 0, i = 1, ..., m, where alpha is any real number between 0 and 2. We obtain radial symmetry in the critical case and non-existence in the subcritical case for positive solutions. To this end, we first establish the equivalence between ( I) and the corresponding integral system {u(i)(x) = integral(Rn) c(n)/vertical bar x-y vertical bar(n-alpha) fi(u(1)(y), ..., u(m)(y)), i = 1, ..., m, u(i)(x) >= 0, i = 1, ..., m. A new idea is introduced in the proof, which may hopefully be applied to many other problems. Combining this equivalence with the existing results on the integral system, we obtained much more general results on the qualitative properties of the solutions for (1).