Balancing Life, Work, and Swimming

I have been struggling lately finding balance in my life. For short period of time, I was able to focus most of my time on my family, but I wasn't doing enough for myself and I was not happy. When I decided to focus more on myself and swim more often, I wasn't very happy with how much time I was spending away from my family. The past few months, work interrupted my swimming schedule and other life happenings required most of my focus; now my swimming has been much less than what I want it to be. I want to find a happy balance between all three. I know that compared to some of you that I am still young, but I worry that my life is slipping away and I am not enjoying it as much as I can. I am curious as to how to find that magical balance and be happy with all three, or if I am looking for something that does not exist. Thanks.