In social species like the feral horse (Equus caballus), changes in individual behavior are likely to affect associated animals. On Shackleford Banks, North Carolina, USA, mares treated with the contraceptive agent porcine zona pellucida (PZP) demonstrate decreased fidelity to their band stallions. Here, we assess the effects of such decreased mare fidelity on male behavior and address potential interactions with habitat visibility, a component of the environment shown to significantly affect feral horse behavior. We compared the frequency and escalation of male-male contests, rates of aggressive and reproductive behaviors directed toward females, and the percentage of time spent vigilant among males experiencing varying levels of mare group changing behavior. We found that regardless of habitat visibility, males experiencing more female group changes engaged in contests at a higher rate (P = 0.003) and escalation (P = 0.029) and spent more time vigilant (P = 0.014) than males experiencing fewer group changes. However, while visibility had a positive effect on aggression directed by stallions toward mares (P = 0.013), female group changing behavior did not influence male-female aggressive or reproductive behaviors (P > 0.1), showing that decreases in mare fidelity altered male-male but not male-female interactions. These results have important implications for feral horse management; PZP-contracepted mares demonstrating prolonged decreases in stallion fidelity may have a disproportionate effect on male behavior. Moreover, our results shed light on the relative influences of female behavior and environmental factors like habitat visibility on male behavior. Such findings can ultimately improve our understanding of how the social and physical environments interact to shape male-male and male-female interactions.