Research has shown that there are at least two kinds of visual selective attention: location based and object based. In the present study, we sought to determine the locus of spatially invariant object-based selection using a dual-task paradigm. In four experiments, observers performed an attention task (object feature report or visual search) with a concurrent memory task (object memory or spatial memory). Object memory was interfered with more by a concurrent object-based attention task than by a concurrent location-based attention task. However, this interference pattern was reversed for spatial memory, with greater interference by a location-based attention task than by an object-based attention task. These findings suggest that object-based attention and location-based attention are functionally dissociable and that some forms of object-based selection operate within visual short-term memory.