I mean, I consented to being in a monogamous relationship which was agreed upon by said partner presumably before (at least without my knowledge of) their dalliances. Yeah it is kind of a consent issue, consent isn’t only applicable to sexual relations despite that being the most common context the word is said in.

Sex without consent is rape, cheating isn’t rape, but it is non-consensual polyamory for the cheated upon. There can be more than one bad thing, rape isn’t it.