I had a swarm do that last year. It was the only swarm I've ever gotten that I didn't put brood in either. It was about 1 1/2 full frames of bees. They seemed uninterested in anything, even taking syrup. They drew no comb and left about 5 days later. From the very beginning the swarm wasn't acting quite right, from my limited knowledge.