Invariants of symplectic and orthogonal groups acting on GL(n, C)-modules

Let GL(n) = GL(n, C) denote the complex general linear group and let G subset of GL(n) be one of the classical complex subgroups O(n) , SO(n) , and Sp(2k) (in the case n = 2k). We take a finite dimensional polynomial GL(n) - module W and consider the symmetric algebra S(W). Extending previous results for G = SL(n) , we develop a method for determining the Hilbert series H(S(W)(G),t) of the algebra of invariants S(W)(G). Our method is based on simple algebraic computations and can be easily realized using popular software packages. Then we give many explicit examples for computing H(S(W)(G),t). As an application, we consider the question of regularity of the algebra S (w)(O(n)). For n = 2 and n = 3 we give a complete list of modules W , so that if S(W)(O(n)) is regular then W is in this list. As a further application, we extend our method to compute also the Hilbert series of the algebras of invariants Lambda((SV)-V-2)(G) and Lambda(Lambda V-2)(G), where V = C-n denotes the standard GL(n) -module.