unless i have a need to find the queen, i never look for her. i often see her, but that just comes from experience. as long as you see eggs or very small larvae, you are good. no reason to dig around.
once you see/find the queen, observe the behavior of the bees around her. when you are familiar with the behavior, she will start to jump out at you (visually) when you later inspect. very often you will find her on the frame with the youngest larvae/eggs. sometimes you'll find her on frames that have bees hatching out as she'll be filling the empty cells with new eggs.
you are fine adding your box at this time of the year. they will build up more quickly in the spring and you will do no harm with a medium on there.
addition...lotsobees and i read your configuration differently. i am talking about 2 boxes. i would not add a 3rd right now.