In the US, people often mistakenly blame the conspicuous goldenrod flower for allergies. Since this plant is entomophilous (its pollen is dispersed by animals), its heavy, sticky pollen does not become independently airborne. Most late summer and fall pollen allergies are probably caused by ragweed, a widespread anemophilous plant.
Arizona was once regarded as a haven for people with pollen allergies, although several ragweed species grow in the desert. However, as suburbs grew and people began establishing irrigated lawns and gardens, more irritating species of ragweed gained a foothold and Arizona lost its claim of freedom from hay fever.
Anemophilous spring blooming plants such as oak, birch, hickory, pecan, and early summer grasses may also induce pollen allergies. Most cultivated plants with showy flowers are entomophilous and do not cause pollen allergies.
Larger pollen grains are more likely to be transported by pollinators, whereas smaller pollen is more likely to be wind-dispersed.
This page lists pollen size by plant family.
Source:
Wikipedia and MNN.com