In short, if you sweat you die. Layer using multiple loosely fitting garments of synthetic, breathable insulation and keep a windproof/waterproof shell handy. Practice thermal management religiously and you'll live. Fashion has no place in the wilderness when your life is at stake. Unless you place ultimate importance in leaving a well dressed corpse for rescuers to find, that is.
Most natural materials are wildly inappropriate for true cold weather use. You may have heard "cotton kills". This is very true, because cotton retains moisture (from your sweat or the atmosphere). Wet and cold is a perfect recipe for hypothermia. Synthetics such as polyester will wick moisture away from your skin and also have the benefit of insulating even when wet. Only a select few natural materials (goose down, wool) are appropriate for cold weather for this reason.
Multiple layers allow you to practice good thermal management. If you feel yourself starting to sweat, take off layers immediately. Vents in both jackets and pants are mandatory because they permit usage of these items while allowing water vapor to escape. Pants or jackets without vents should be stricken from your shopping list.
Your windproof/waterproof layer stays in your backpack until you encounter rain, high wind or if you have to stop for an extended period of time (e.g. it's your turn to belay your climbing partner). Despite what they advertise, Gore-Tex does NOT breathe well. Thus, it stays in the pack until you need it.
Layers must be loose, not tight. This permits airflow, so your perspiration dries faster and does not soak through your insulating layers. A base layer can be skin-tight if you so desire, but everything else needs to be loose to permit airflow.
Layering also applies to your hands. Frostbite will attack your smaller extremities first. Big puffy insulated gloves are useless in these situations, because your sweating hands will soak the insulation -- and they take forever to dry. Better to buy a nylon shell glove and at least two pair of fleece liners. The nylon shell can be used to fend off severe wind or if you are climbing on ice to keep water out. In normal conditions you can wear the liners without the shell so your hands breathe and do not sweat. If you are forced to use the shell and liners and your sweat soaks the liners, just swap them out for a dry pair of liners.
Make sure you protect your feet as well. Toes are extremities that are easily lost to frostbite. Mountaineering boots are usually plastic with a removable insulating bootie (again, a layer) with enough room in the boot so that you can wear polypropylene liner socks to wick away sweat. They're also stiff enough so you can use crampons without getting blisters. Wear a boot that is too tight and you'll restrict bloodflow to your toes, which means you'll get colder faster.
You lose a lot of heat through your head. Again, this means a breathable hat (fleece or wool) is required, and one that covers your ears is mandatory. If you get caught in the rain or if high winds are present, you break out your windproof/waterproof parka which has a hood that keeps your head dry.
I also recommend carrying a breathable fleece neck gaiter so you can protect your neck and cheeks. I will take a windproof balaclava if high winds are anticipated; it stays in the pack until needed. Windproof neck gaiters are not recommended during strenuous activity because if you've got it up over your mouth then it funnels your exhaled breath up onto your sunglasses or ski goggles. When you are on a mountain surrounded by snow, you do not want to have your eye protection fog up on you because you'll end up ditching your glasses. Snow blindness is very painful and you should be wearing eye protection to fend off falling ice or rocks as you climb. Ski goggles will protect your eyes from extreme wind but they stay in the pack until needed.
Fully buttoned up, you might look like my colleague here ... this was taken during a winter assault on Mt. Washington a few years ago:
(the next photo is after the whiteout lifted so you can actually see the rest of Tuckerman Ravine behind him)
Last, did you know you lose a lot of water through your breath in cold weather? Your respiratory system warms and moisturizes the air as it gets taken into your lungs and you exhale. Each time you take a breath, you are losing water. Stay hydrated and you'll be better off. Make sure you keep your internal furnace fueled by snacking on energy bars as well. Your car won't run without gas; the same goes for you!
Sorry for the long winded post. I'm still learning things myself, even after all this time in the woods. There are too many times I've seen a lot of instances of people using/wearing the wrong gear and for the wrong reasons. Stay dry and you'll be fine!