ahh... did a couple hours reading last night. Read through both threads totalling 9 pages of junk plus every single link there which was like 15 or so. Also read a bunch of reports and studies on hologen vs HID and other sites dealing with lighting.
First off, there's a couple reasons that HID's cause more glare or seem to, there are a few more but they weren't as important.
1. The higher heat intensity, which gives them thier white or blue colour, gives off a shorter wavelength than the yellowish lights. This is not as easy on the eyes and causes more strain and a "sun" effect. Also the higher heat intensity gives off far more lumens than a halogen at the same wattage making them brighter.
2. The light pattern on HID's is a crecent shape which has a distinct contrast between lit and unlit area. This causes a great deal more glare when on turns and hills to other drivers, causing a "flashing" effect as the other drivers come in and out of the lights beam. Hologens have a cylindrical effect where there is a gradual increase of intensity from the far outside of the beam pattern to the center. This makes for a gradual increase in intensity, therefore no "flashing" as oncoming traffic will gradually come into the beam.
This in addition to poor quality installs from shops/self install kits, poorly adjusted headlights, and high wattage bulb installs in hologen casings are causing a problem for other drivers, not to mention the more lumens per wattage of HID's vs halogens.
Also in comparisons with oem HID's vs oem halogens the average lighting distance between them is about the same. A high quality set of halogens will actually outperform HID's in depth. The cylindrical light shape of the hologens can shine further down a road. These characteristics will also limit flashing/blinding as they shine the majority of the light down the road and graduate towards the sides. To have a HID shine at that same distance of a halogen, or try to improve on a stock halogen system, would be blinding oncomming traffic for sure, which is what we all see.
Generally what makes HID's "seem" to light better is that they have a wider beam pattern, the bluish/white light seems brighter, and higher heat intesity and lumens at given wattage. Most all the comparisons and positive comments on HID's and the like were because of the wider pattern. This gives the false impression of being a better lighting system. Seeing these areas of the roads only provides a better driving light for under 30mph, fog lights give the same impression by lighting the road nearby and to the sides. This is actually a negative aspect of the HID's. Although providing a safer driving light under 30mph this causes a more dangerous light at higher speeds, not only for other drivers but yourself. Lighting the unnecessary areas of the road causes more light to enter your eyes that is not needed. This causes your pupils to be more dilated than with standard halogens causing your view distance to actually be LESS than with a hologen that lights at the SAME distance, not to mention the blueish/white light causing more dilation.
I'd post more but I ramble too much as it is, lol.