Also depends on the set up, whether they are in a full size hive or a nuc. My experience has proven to me that without providing (feed/pollen) for a small colony, they usually can't build comb, forage, tend to brood and most importantly keep the queen adequately fed/healthy.
Just this past week I went into a young lady's hive (a full deep) with 3/4 of one frame of bees. She had ordered a new, marked queen two weeks prior.
The queen was (marked) shriveled and runny and flew right off the frame never to return. I believe she was not being tended to/fed properly and it was just a matter of time for that lil colony to wither to nothing at all.
I removed a colony this past Saturday night and shook them into her set up with a plump queen the following Sunday, so she would have a hive in that set up.
BTW, an extremely small swarm is one that contains roughly 1,500 bees.