Do other swimmers worry about shoulders outside of pool?

I am always worrying about tearing something in my shoulders when doing work outside of pool. Yesterday I was turning over my garden and a few times I moved my arm the wrong way and I felt a pain and thought I was going to tear something. It has come up other times as well. Is it normal?