Swim shorts could work, but you could possibly stiff get chaffing using them as the bottom of the legs of some swim shorts are loose fitting. Regarding chaffing, most important to stay dry----moisture/rain/dampness (even from sweat) causes it to set in. If it starts, treat immediately. Elsewise it will just get worse and worse like a blister. Prevention is the best policy.
To prevent chaffing, you need a snug fit, something that will cling to skin and not jump around and cause friction between the fabric and the skin.
Suggest biker shorts, spandex type shorts, or cut off long john bottoms, with shorts over them to be a better option.
Might want to consider long pants or at least high gaiters (if using shorts) to keep the ticks, poison ivy, bees, wasps, and other things off of you a bit.
See you out there. :cheers
Maintain