How fast should I build up yardage?

I'm in my third week back after taking a few years "off" (and gaining about 20 lbs), and don't want to burn out, but I want to keep adding yards! My first week back, I swam 4 times, and did 1000 yds, 1200, 1500, and 1250. I was exhausted by Friday! My second week I only got in the water twice :P but felt much stronger. I did 2000, and 1500 yds. This week yesterday, I only eeked out 1300 yrds b/c it snowed and I had to shovel the driveway before heading to the pool, so my time was cut short. It's only taking me about 35 to 40 min's to do these workouts, but I just joined USMS and want to start using the awesome workouts on the forums (usually under 4000yds). Should I just dive right into the full yardage, or what? :confused: (if it matters, I also lift full-body ciruit M,W,F at lunch, and do SS cardio T,Th also at lunch.) Thanks!!