Ok, I was wondering how you might have come by a swarm at this time of year. You should change the location in your profile to show where you are so we can tell you about the bees where you are. My bees were out the day before yesterday looking for something to eat, today they've tucked in tight, don't let the bedbugs bite.
If you just got the swarm, they will need to hatch the baby bees. That takes 21 days. Since your worker bees are basically the oldest bees, and are in the process of setting up a new colony, they are dying off. Hopefully (most probably) they'll keep things together long enough to have the new bees hatching.
What should be a concern is if you open the hive and do not see any eggs, new larvae, or sealed brood. If the hive has no queen, in my experience they might hang around until there are just a teaspoon of bees. Mine stopped building and drank syrup, drew cells on any wax I put there but didn't make any comb.