I m nervous about going white water rafting this weekend.

I am 22 and just learned to swim recently. A year ago i couldn't swim very well, however today, I can swim freestyle and back stroke in the deep end one full lap (usually complete 1 25 m lap, take a breather, and do another one and repeat the process for a while), I am getting more and more comfortable treading water, and my *** stroke is coming along as well. Obviously I am no expert swimmer, and I am trying to increase my endurance and stamina, and my technique so far gets me through the water, but could definitely be improved upon to make me an efficient swimmer. So anyways, I am going white water rafting with some friends this upcoming weekend, and I am kind of nervous. The rapids we are doing go up to class IV, and apparently the raft guides let people out of the raft to swim for a good portion of the trip. They are going to give us pfds. I have actually been rafting once, last year before I learned to properly swim ( class 1-3 so pretty mellow). my friends told me to come out of the boat and swim around with them, so i fell for peer pressure jumped out, floated and panicked. I tried moving my arms and legs but couldn't move at all, one of my friends had to drag me back to the boat, where i stayed for the rest of the trip, feeling super embarrassed and awkward. Thats actually the main reason i forced myself to take swimming lessons. However I feel like I am going to panic all over again. Any ideas to help me keep calm? Is there a certain way to swim with a life jacket on? the only flotation devices i have trained with are kick boards and flotation barbells, and both make it harder for me to swim, I swim much faster and more efficiently without any flotation support, and I feel like it might be harder for me to swim with the life vest on. Also the water in a river moves around a lot more than a swimming pool, which probably means you have to put in a lot more power behind your strokes, correct?