My desktop pc bristles with USB 2.0 ports, front and back.

I have a wide assortment of items (accessories, perpherials, etc.) which plug in to these USB ports - recording mic, headset, Skype camera, printers (2), memory card reader, flash memory sticks, digital camera download cord, to name some.

I have never heard or read any discussion regarding "when" to plug or unplug these devices from my pc tower.  To clarify the question:
1) should the computer be shut down before plugging or unplugging?
2) can the computer be safely left on during plugging or unplugging? (Harm not likely to either the pc or the external device?)
3) does it matter at all?
4) is there a difference if the device is being connected for the first time (driver to be loaded) of if the device is already "known" by the pc?

Seems a simple question, but with so MANY cords at the ready I began to scratch my head.     Opinions welcome. The best way (in my case) to use usb devices is to plug only when you use them. The exceptions are printers, keyboard, mouse, webcam which are by designed usb interface devices that can be left alone plug either your system is turned on/off. While usb storages such as flash or portable drives can also be left unplugged during shutdown and powering up your PC, in my case, I've practiced removing them and using the safely remove hardware method. While usb devices are plug and play, a NEW device plugged in WOULD require a generic windows usb controller driver for the system to recognize and be able communicate with the new device and work properly. Windows would install the compatible device driver only once. Generally, it's a user preference on how you want these usb devices clogged your ports and desk. All USB devices can, by definition, be unplugged or plugged in without physical damage to either the device, port, or machine.

However, If the device is being used at the time, you'll have problems later. For example you shouldn't yank out a Thumb Drive while reading or writing files to it.

Personally I never use the "safely remove Hardware" option for Flash Drives, and I've not had problems, though of course past performance is no guarantee of future results, and many others have had problems.

Using the "Safely remove" option is ideal for storage devices, not because it requires a SPECIAL removal step, or even benefits from it, but the fact that it will only succeed if nothing is CURRENTLY using the device. Since if the device is in use when you unplug it you could lose data.

As for driver software, jason2074 covered that thoroughly.