This is an old topic I'm ressurecting, but I just talked to my allergy specialist Doctor about this today and thought I'd share his words.
First he strongly advised against eating raw pollen. He quickly found a few medical journal references where a patient had consumed raw bee pollen and either died or nearly died because they had a strong allergic reaction.
He also confirmed what Finsky reported above. Allergic reactions are complex and to develop resistance to allergens requires very specific and controlled dosage of reactive agents over a very long period of time. Pollen is the activating agent for rhinitis (hay fever), but not for food allergy, bee-stings or pet allergies and such.
He offered some hope that in the future we may see some type of refined pollen product that could be taken orally, but it is probably several years away. It won't be raw pollen that would contain an uncontrolled mix.
He offered no opinion on the small pollen that remains in unfiltered raw honey from local production. It may be helpful, but benefits are very hard to quantify.
Finally, he offered this webiste as a useful resource for further research:http://www.pubmedcentral.nih.gov/
Of course, the best advice is see your own doctor.