You've figured out what you like to wear, and you wear it - simple as that. And that's how it should be done. As far as the casual/formal argument, someone already pointed out that the older generation would wear bib overalls and fedoras. It was a common thing when I was growing up to wear newer bibs and a fedora with either a clean white shirt or a "suit coat" (often an older one, because that's all they had) to church. As a matter of fact a lot of us still do that at our annual church "Heritage Day" picnic. I think where you grew up has a lot to do with your later perceptions of style.