Simple Summary Breast sarcomas, phyllodes tumors, and desmoid tumors are rare and unique entities whose epidemiologic, molecular (especially genomic), clinical, prognostic and predictive, and therapeutic landscapes are poorly characterized. Despite their rarity, the potential aggressiveness and significant functional impact of these entities make it important to characterize them thoroughly. The challenges of mapping the genomic landscape of breast sarcomas for the first time and updating the existing literature in terms of risk, prognostic and predictive factors, and novel treatment options motivated us to pursue our goal of producing a high-quality review focused on this group of neoplasms. This article intends to be a new source of data for researchers seeking new targets to address therapeutically and a tool that can be used by clinicians to guide their decisions when treating patients with breast sarcomas, phyllodes tumors, and desmoid tumors. Breast sarcomas (BSs), phyllodes tumors (PTs), and desmoid tumors (DTs) are rare entities that arise from connective tissue. BSs can be classified as either primary or secondary, whether they develop de novo or after radiation exposure or lymphedema. PIK3CA seems to play an important common role in different BS. Malignant PTs show similar behavior to BSs, while DTs are locally aggressive but rarely metastasize. BSs usually present as unilateral, painless, rapidly growing masses with rare nodal involvement. The diagnosis should be based on magnetic resonance imaging and a core needle biopsy. Staging should comprise a chest computed tomography (CT) scan (except for benign PT and DT), while abdominal and pelvic CT scans and bone scans should be added in certain subtypes. The mainstay of treatment for localized BS is surgery, with margin goals that vary according to subtype. Radiotherapy and chemotherapy can be used as neoadjuvant or adjuvant approaches, but their use in these settings is not standard. Advanced BS should be treated with systemic therapy, consistent with recommendations for advanced soft tissue sarcomas of other topographies. Given the rarity and heterogeneity of these entities, multidisciplinary and multi-institutional collaboration and treatment at reference centers are critical.