Scadsobees, when harvesting a swarm, if you don't get most of the bees, there is a chance you didn't get the queen that the swarm had. The bees tend to hand around the queen, so next time you might want to keep the box (that you put the hive in) around the tree so that they can follow the queen into the box. I would gather the rest of the swarm (if there is a lot of them) and put it into the swarm you put into a brood chamber. Otherwise, if its really small, they will eventually start to fly off. I don't think they are aggressive if they are flying around, simply unaware of their surroudings, maybe to find their queen.
I don't think bringing a frame of brood to the rest of the swarm is going to do anything. Maybe.
I bet your getting a ton of grief - the swarm at a schoolyard and such.