Your intuition is on the right track, but it works “the other way around”.

RGB are additive primary colours, because the colour you see when you look at something that emits light is the actual colour of the light. And so when you mix two coloured lights, the colours add up (additive colours). And adding every colour gives you white.

CMY are instead subtractive colour, because when you look at something that does not emit light, the colour you see is just the light that bounces off of it, while some colours get absorbed. So when you mix paints, the resulting paint absorbs more colours, and you only see what’s left, so the colours subtract down (subtractive colours). And subtracting everything gives you black.

P.s. mathematically, any three independent colours could be used as primary. Independent means that you can’t get any of the three by mixing the other two (i.e. blue, red, and purple are not independent). But those two triplets are the most obvious choices. You might recall that as a kid, they taught you that primary colours were Red, Blue and Yellow instead of CMY, and yet mixing worked fine.