fat loss vs muscle gain

First off I'm new here and this is my first post so if this question has been answered on a different post, sorry. I come from a bodybuilding background and recently started swimming and liked it so much that for the summer I'm forgoing lifting for swimming. I've been swiming four times a week now for about four months and I've lost about 8 lbs - as far as I can tell it's all body fat. My question is, can I build muscle mass by just swimming or will I just be reducing body fat? Also is it possible to burn muscle? In other words could some of my weight loss be muscle loss? The reason I ask is that a couple of my friends have told me that my back is getting broader - I haven't really noticed.