Background: Anxiety disorder (AD) and substance use disorder (SUD) highly co-occur with bipolar disorder (BD). AD and/or SUD co-occurrence is associated with poorer clinical outcomes in BD. However, respective associations between AD and/or SUD diagnoses and BD outcomes require clarification. Baseline data from the Systematic Treatment Enhancement Program for Bipolar Disorder (STEP-BD) were therefore utilized to investigate independent and simultaneous contributions of ADs and SUDs on clinical variables in BD. Methods: Two latent factors, "pathological anxiety" and "substance use problems," were derived from presence/absence of lifetime AD and SUD diagnoses. Latent dimensions' associations with clinical variables, obtained from the Affective Disorders Evaluation, Mini-International Neuropsychiatric Interview and Range of Impaired Functioning, were estimated via structural equation modeling (SEM). Results: Modeled independently, pathological anxiety and substance use problems were significantly associated with several variables. Yet when modeled simultaneously, pathological anxiety's associations with functional impairment, past-year rapid cycling, and past-year %time spent anxious and depressed remained while most variables' associations with substance use problems became non-significant. The only significant latent-factor interaction evidenced was for age of BD onset. Limitations: Analyses were limited to lifetime diagnoses and causality may not be inferred given cross-sectional data. Conclusions: ADs and SUDs impact on BD was mostly additive rather than synergistic. Findings highlight the potentially understated importance of treating inter-episodic anxiety in BD as it may exacerbate mood symptoms, increasing functional impairment and risk for subsequent mood episodes.