I have never heard that the entire hive only goes after one flower.
What I have heard, is that once an individual bee goes out foraging and finds a source, she will only work that type of flower before returning to the hive. In other words, she will only visit one type of flower on each foraging flight, she will not visit a clover, and then move over to a dandelion. She'd visit only clovers on that flight.
But as for the hive as a whole? I wouldn't think the entire hive would stick with one flower only. Because as you said, the different colored pollens coming in sort of rule out that theory. As a kid, I wondered how honey could be labelled as 100% pure clover(or other flower source). I couldn't believe that no other flowers were visited at all! I wondered if beekeepers locked their bees in a greenhouse with only clover!