Are speedos really that bad?

So, I have worn boardshorts since high school. Recently I started wearing speedos to the gym pool for lap swimming and I LOVE them! I was thinking of switching to them for my regular swimwear. But, all I hear from other people is negativity about them. Are they really that bad? If I were to switch to speedos would it really be that bad?