Speaking of the hives I had in olives, they produced honey of gold in color, not as light as a star thistle honey, a tad darker, not a dark brown. Tasted like good sweet honey to me. I never witnessed the bees taking pollen on olives trees themselves. Olives are heavy pollinators, it blows everywhere, it was all over the hives too. Easy pickens, if they took any, I didn't see it or put it under a micro scope.
Personally, I would keep hives in olives again but I wouldn't rely on olive pollen alone, I think they look else first. According to the studies I've looked at where a microscope is used, they do take some (little) olive pollen in.
Lets not forget sources of nectar.