Stromal immune cells constitute the tumor microenvironment. These immune cell subsets include myeloid cells, the so-called tumor-associated myeloid cells (TAMCs), which are of two types: tumor-associated macrophages (TAMs) and myeloid-derived suppressor cells (MDSCs). Breast tumors, particularly those in human epidermal growth factor receptor 2 (HER-2)-positive breast cancer and triple-negative breast cancer, are solid tumors containing immune cell stroma. TAMCs drive breast cancer progression via immune mediated, nonimmune-mediated, and metabolic interactions, thus serving as a potential therapeutic target for breast cancer. TAMC-associated breast cancer treatment approaches potentially involve the inhibition of TAM recruitment, modulation of TAM polarization/differentiation, reduction of TAM products, elimination of MDSCs, and reduction of MDSC products. Furthermore, TAMCs can enhance or restore immune responses during cancer immunotherapy. This review describes the role of TAMs and MDSCs in breast cancer and elucidates the clinical implications of TAMs and MDSCs as potential targets for breast cancer treatment.