Thinking of starting to swim again

Hello all...I'm a soon-to-be college grad in southern New England, and I'm looking to start swimming again. I swam for my YMCA team in middle school, and for the Y and my high school team in high school. As my user name suggests, I was also a competitive rower, and being New England-level in rowing and State-level in swimming meant that crew was my focus when considering my collegiate sports options. I rowed for a full year at my D-I university, but discovered that competing at that level took away from academics too much, as well as put too much wear and tear on my body. I was not a fast enough swimmer in high school to swim for my university, so that was not an option once I stopped rowing. The last three years I have exercised somewhat sporadically, and gained perhaps 20 pounds from my rowing competition weight (which itself is around 15 pounds heavier than I *should* be). Let's just say that I am not terribly pleased with my physique at the moment! This is how swimming came into my mind - I never had problems losing fat/toning up when I was in-season for swimming (always gained fat and muscle when rowing for some reason). The dreaded post-workout carbohydrate cravings were never really an issue - apparently I am lucky in this regard. I also actually enjoyed workouts. I found the website for my state, and there is a Masters team at a college about 5 minutes from my future place of work. I need to join a gym, and they have a nice pool - I had meets there in high school. The idea of competing again is also very appealing since I enjoyed it so much when I was younger. Swimming is something I can see myself doing when I'm 80, and I just don't see that with any other sport or fitness activity. The way I see it, I have two options: Workout on my own for a couple months to get back into shape, or workout with the local Masters team in the slow lane. I used to be a distance swimmer, but I haven't trained in 4 years and doubt I could even swim an easy 500 straight through. Also, I could stand to lose 30-40 pounds - it may sound silly but I don't even go to the beach looking that bad in a swimsuit. On the other hand, coaching never hurt anyone! Being the slowpoke could help me stay motivated to get back to where I was 4 years ago. So, for those who have come back to swimming after a period away, what worked better for you?