Pairing symmetries in a Hubbard model on an anisotropic triangular lattice

To consider the paring symmetry formed in organic compounds kappa-(BEDT-TTF)(2)X, we study the half-filled-band Hubbard model on an anisotropic triangular lattice (t in two bond directions and t' in the other), using an optimization VMC method. As trial states, we adopt a coexisting state of an antiferromagnetic (AF) order and the d(x2-y2)-wave RVB gap, in addition to the d + id- and d + d-wave gap states. In these states, we take account of the effect of band (or Fermi surface) renormalization. Magnetic Mott transitions occur, and a regime of robust superconductivity could not be found, in contrast with our previous study. In the insulating regime, the coexisting state in which an AF order prevails is always the lowest-energy state up to remarkably large t'/t (less than or similar to 1.3), whereas a d(xy)-wave RVB state becomes predominant when t'/t exceeds this value. In the insulating regime, the effective Fermi surface, determined by the renormalized value t'/t, is markedly renormalized into different directions according to t'/t; for t'/t 1.3, it approaches that of the square lattice (t'/t = 0), whereas for t'/t greater than or similar to 1.3, it becomes almost one-dimensional (t'/t >> 1). (C) 2007 Elsevier B.V. All rights reserved.