Swim vs gym - striking a balance

I've (M55) started adding gym twice a week (evenings) to my regular early morning swim sessions. I do a session based loosely around the jason lezak video (www.bodybuilding.com/.../weight-training-for-sprint-swimmers.html) with some core work and pressups. The full session including stretching takes about one hour twenty minutes. I'm usually knackered after it. I'm still pretty tired the following day and DOMS kicking in on the second day, so consequently my pool work isn't at the same level it would normally be, think less of USRPT and more long distance plod at an easy pace. So, my question is how do people strike a balance? I imagine swimming is the best exercise for swimming, so am I being counterproductive by doing gym work, if my swim training suffers? I realise I'm not 16 anymore, but I was never very strong, so I feel that the extra strength from gym work should be beneficial. I'm a middle distance freestyler if that make any difference. My morning swim sessions are around 90 mins. Thanks.