Friends talking about my brother inappropriately?

My little brother is a swimmer for his high school and me and my mom go to most of his meets. Its hard not to notice this, his team wears grey speedos and you can totally see their penises. On Friday, I brought two of my girlfriends to his meet. I thought it would be fun but the entire time they were commenting on my brother and his penis. They kept talking about how big it was and asking me what it looked like without the speedo. My mom thought it was funny but I got offended and mad at them like the whole rest of the night. I don't like thinking about my baby brother's penis and I certainly don't want my friends talking about it. Did I overreact? Why didn't my mom care?