This study deals with the characterization of 5-hydroxytryptamine (5-HT, serotonin) receptors positively linked to adenylyl cyclase in membranes from pig brain caudate. 5-HT and related agonists induced a concentration-dependent stimulation of adenylyl cyclase activity in pig caudate membranes, with the following rank order of potency (mean pEC(50) values): 5-HT (7.1) greater than or equal to 5-methoxytryptamine (6.9) > 5-carboxamidotryptamine (5.6) > sumatriptan (< 5). Maximal stimulation by 5-HT averaged 35 pmol cyclic AMP/min/mg protein over a basal activity of 159 pmol cyclic AMP/min/mg protein. 5-Methoxytryptamine and 5-carboxamidotryptamine had similar efficacies to that of 5-HT, whereas sumatriptan was about half efficacious. Other compounds known as agonists at some 5-HT receptors were weakly potent (mean PEC(50) values < 5). They include the 5-HT1A receptor agonist, 8-hydroxy-2-(di-n-propylamino)tetralin hydrobromide (8-OH-DPAT), the 5-HT4 receptor agonist, renzapride and the 5-HT2 receptor agonist, (1-(2,5-dimethoxy-4-iodophenyl)-2 aminopropane) (DOI). In antagonist studies, methiothepin (0.1 and 1 mu mol/l) shifted the 5-HT curve to the right with no depression of the E(max), yielding pK(B) values of 7.4-8.0. Clozapine (1 mu mol/l) also produced surmountable antagonism of 5-HT-induced effects (pK(B) 6.9). Ketanserin (10 mu mol/l) weakly antagonized 5-HT (pK(B) 5.0). The 5-HT4 receptor antagonists, tropisetron (ICS 205-930) and SDZ 205-557 (2-methoxy-4-amino-5-chloro-benzoic acid 2-(diethylamino) ethyl ester), each at 1 mu mol/l, did not significantly alter the concentration-response curve of 5-HT. The present receptor shares some characteristics of the recently cloned 5-HT6 receptor (Monsma et al. (1993) Mol Pharmacol 43:320-327): similar pharmacological profile, location (striatum) and ability to stimulate adenylyl cyclase. It may thus represent the functional 5-HT6 receptor in its natural environment.