belfastboy nailed it, wear whatever the hell you want. I will keep a diplomatic silence on the pork pie.I suggest wear whatever the hell you want except perhaps a Pork Pie.....as no one except Van Morrison looks good in Pork Pie and even he doesn't wear them anymore. My perspective is that unless you have a head shape that is an extreme outlier you can wear whatever you like, whatever you feel comfortable wearing and with whatever outfit. It is all a matter of style and confidence. I have a collection of somewhere north of 60 hats.......all colours, multiple styles and wear the hat I feel like wearing that day with whatever I feel like wearing. I have no firm rules of which hat with which outfits.......I just wear the hat. So if it 'breaks' any fashion norms I can live with that.
The two photos that you have posted are great images, so where's your problem? Your hat size might limit you but there's always another supplier somewhere that can accommodate you.
Welcome to The Lounge.