I have quite a bit of various hyssops all over myyard and the bees love it, its constantly covered in bees all summer.
I have never noticed any taste of it in any of the honey.
I don't think you have anything to worry about.
As someone mentioned I think you would have to have fields of that and nothing else for them to forage on to make a difference, everything is pretty well blended when you only have a few of this and that.
They are a beautiful plant, they are hearty and last well into fall.
They get pretty big and grow quickly so keep that in mind for location when you plant.