I doubt bees from a swarm would rob from the hive. The gorged themselves before leaving the hive, so I would think they may be just 'going back home'?
Can you get to the swarm and capture it? If not, can you set up a swarm trap. There are alot of post regarding swarm traps if you do a search on it.