You should shake the bees off. They however return their original hive.
Take a such frame, where bees are emerging. Otherwise smaller hive is in trouple when they must keep the brood warm.
Just 2 weeks ago I moved bees from another yard to a small colony. They killed the queen.