I’m sorry but I fail to see how this can happen.
Assuming you’re not wearing the down vest in direct contact with your skin and exercise with it (which would be super weird by the way), how’s the sweat going to soak through all the clothing layers you’d normally wear under it like t-shirt, shirt or sweater, and then also soak the lining and make it all the way to substantially soak the feathers to the point they get sticky and clump up?
This makes little sense to me and it makes more sense that the moisture absorbed by the down would come from environmental exposure, rather that your own body.
But by all means, wash it as needed.
Edit: typo.
