Gloves/Mittens
For winter riding, heavy gloves are a must, but sometimes even gloves
might not work against the cold. If you get glove liners, they do work
to some extent, but I find that the fingers might still get cold.
The best thing you can get are the cycling mittens. These are different than normal mittens in that they are split between the middle finger and the ring finger (first image). This is so you can handle the brakes while gripping with the handlebar with the other two fingers. Mittens work a lot better because the fingers are more grouped together instead of individually wrapped as in a glove. So it's more likely to retain the heat a lot better.
Feet
There's two types of protection for the feet. There is the external protection and then there's the internal protection. Booties are the best external protection for the feet. Most of the cycling shoes are breathable since that is what they do in warmer days. But in colder days, the cold air penetrates the shoes and freezes your feet. Neoprene booties cover the entire shoe (except for the cleat) so that you get a good measure of protection from the elements.
Winter cycling socks are quite thick and can help your feet internally. I have the Pearl Izumi thermal wool socks that help tremendously while cycling. I did try them for running, but my feet got overly hot and sweaty; they are designed for protection at higher speeds, like cycling! So these socks are used for cycling only.
Head and Neck
Lastly, but most importantly, the head and the neck! It is SO important to buy a balaclava that encompasses both the head and the neck. You really don't want anything exposed here, so you need a balaclava long enough to stick into the collar of your jacket so that nothing is exposed! Again, the Pearl Izumi balaclava that I own does the job.
If it's particularly freezing, I do stick a running cap over the balaclava for more protection, although you'll need to adjust your helmet to make everything fit.
And if it's really REALLY cold, then you need to cover your eyes. I find that a cheap pair of ski goggles will do fine here.
Bottom line is that as long as you are totally sealed against the elements, leaving no inch of skin exposed, you can definitely get outside and ride. Of course, if there is ice or snow present, then if you only have a road bike, then you're pretty much out of luck. If you do have a mountain bike, however, with knobby tires, you can venture out. You still have to be careful with ice, but those bikes can handle the snow fantastically!
The best thing you can get are the cycling mittens. These are different than normal mittens in that they are split between the middle finger and the ring finger (first image). This is so you can handle the brakes while gripping with the handlebar with the other two fingers. Mittens work a lot better because the fingers are more grouped together instead of individually wrapped as in a glove. So it's more likely to retain the heat a lot better.
Feet
There's two types of protection for the feet. There is the external protection and then there's the internal protection. Booties are the best external protection for the feet. Most of the cycling shoes are breathable since that is what they do in warmer days. But in colder days, the cold air penetrates the shoes and freezes your feet. Neoprene booties cover the entire shoe (except for the cleat) so that you get a good measure of protection from the elements.
Winter cycling socks are quite thick and can help your feet internally. I have the Pearl Izumi thermal wool socks that help tremendously while cycling. I did try them for running, but my feet got overly hot and sweaty; they are designed for protection at higher speeds, like cycling! So these socks are used for cycling only.
Head and Neck
Lastly, but most importantly, the head and the neck! It is SO important to buy a balaclava that encompasses both the head and the neck. You really don't want anything exposed here, so you need a balaclava long enough to stick into the collar of your jacket so that nothing is exposed! Again, the Pearl Izumi balaclava that I own does the job.
If it's particularly freezing, I do stick a running cap over the balaclava for more protection, although you'll need to adjust your helmet to make everything fit.
And if it's really REALLY cold, then you need to cover your eyes. I find that a cheap pair of ski goggles will do fine here.
Bottom line is that as long as you are totally sealed against the elements, leaving no inch of skin exposed, you can definitely get outside and ride. Of course, if there is ice or snow present, then if you only have a road bike, then you're pretty much out of luck. If you do have a mountain bike, however, with knobby tires, you can venture out. You still have to be careful with ice, but those bikes can handle the snow fantastically!