Goliath was not during the wanderings. It was during the reign of King Saul, long after the trip across the desert. But there are other references to Giants. The first:
"There were giants in the earth in those days; and also after that, when the sons of God came in unto the daughters of men, and they bare children to them, the same became mighty men which were of old, men of renown."--Genesis 6:4
This is pre-flood and pre-Abraham.
When they were in the wanderings and about to go into Canaan we have this:
"And there we saw the giants, the sons of Anak, which come of the giants: and we were in our own sight as grasshoppers, and so we were in their sight."--Numbers 13:33
There are 9 other references to giants, often talking about whole peoples of giants. Here's one with measurements:
"For only Og king of Bashan remained of the remnant of giants; behold, his bedstead was a bedstead of iron; is it not in Rabbath of the children of Ammon? nine cubits was the length thereof, and four cubits the breadth of it, after the cubit of a man."--Deuteronomy 3:11
9 cubits is over 13 feet. 4 cubits is just short of six feet. Keep in mind this is his bed, not the person...
And of course the classic Goliath with measurements:
"And there went out a champion out of the camp of the Philistines, named Goliath, of Gath, whose height was six cubits and a span."--1 Samuel 17:4
6 cubits is almost 9 feet. A span is 9" which makes him about 9' 9". Robert Pershing Wadlow (22 Feb 198-27 Jun 1940) was 8' 11.1" Less than an inch away from 9 feet.