Newbie Lap Counting Question

Hello All, So I've gotten back into swimming as part of my fitness regime and had a question many might think is basic, but I'm interested in your thoughts. I usually do longer swims as opposed to interval training. My go to swim is 2000yds which is usually around 40min, give or take. 2 weeks ago I clocked 39:25 which was great - during the next swim a couple of days later I thought "I might even beat that" only to finish with a 46:20. I was really surprised (and bummed). The next swim was 40:35. So I was thinking... I may have lost count and added a couple of laps to the longer time or dropped a couple on the shorter. I had a lap counter watch which got moisture inside and stopped working a while ago, and frankly was kind of a pain to hit the little button at the top of each lap - I found it distracting. My question: What do you do to make sure you're accurately counting laps on longer swims? I'm going to move to 2500yds soon and fear that may be more of a count problem. Thanks in advance for your time and any help you can offer. Cheers, Chris