—I stood between the LORD and you at that time to convey the LORD‘s words to you, for you were afraid of the fire and did not [וְלֹֽא־] go up [עֲלִיתֶ֥ם Qal perfect 2nd person masculine plural of עלה - you went up] the mountain— (Deut. 5:5, JPS Tanakh)
The wording of Deut. 5:5 seems to indicate that "they may go up on the mountain" at the of Exodus 19:13 meant all of Israel was supposed to have climbed the mountain.
12 You shall set bounds for the people round about, saying, ‘Beware of going up the mountain or touching the border of it. Whoever touches the mountain shall be put to death: 13 no hand shall touch him, but he shall be either stoned or shot; beast or man, he shall not live.’ When the ram’s horn sounds a long blast, they may go up on the mountain.” (Exodus 19:12–13, JPS Tanakh)