Cold water swimming while pregnant

Is it safe to swim in cold water while pregnant? I haven't asked my doctor yet, but I'm afraid their default answer will always be "no" when they aren't sure. I live in San Diego and I don't plan on swimming at all if it gets below 58-59. Once it's in the low 60s, I would consider wearing a short wetsuit. I wouldn't normally wear a wetsuit at all but I'm willing to make some small concessions to protect the baby from the cold. So it's not like I'm swimming in extreme temperatures...but low to mid 60s would be considered cold by some people. Also, I don't push myself much now that I'm pregnant. So I probably won't be swimming more than 20-30 minutes at a time, and even less than that if I feel cold or tired.