Moses and Aaron, but Joshua and Caleb especially, and the Lord MOST OF ALL!
Walk through the immediate passage leading up to and through the event (Num 14:1-10)
v2: The people grumbled against Moses and Aaron.
v3: The people grumbled against The Lord.
v5: Moses and Aaron are on their faces before the people.
vv6-9: Joshua and Caleb speak to the people.
v10: The assembly wants to stone "them".
The main characters that are in a face off with the assembly are very known and very active. There is no mention of trying to single-out any one or two of these four men in front of the assembly. The assembly is confronting all four men.
(E.g. no one in the crowd says something like, 'We like Aaron because he makes good golden calf idols, but the rest can all die!'—There is NOTHING like that at all, but there would need to be if we were to say that the crowd wanted to single someone out.)
Why Joshua and Caleb especially
If any were to be singled out, Joshua and Caleb would have been more likely to be singled out since they were the only two who had just finished speaking. But, they weren't "singled out", so we should consider them in the "spotlight".
Joshua and Caleb's speech was the the testimony against the "bad report" (Num 13:31-33), and theirs was the testimony that drew out the people's rage.
Moreover, it is somewhat understandable to have a motive to try to kill the witnesses, which Joshua and Caleb were.
And, considering the scene: Joshua and Caleb are looking at this crowd of people who are looking back at them—it's a back-and-forth dialog between dual-speaker and crowd. With Moses and Aaron on their faces, Joshua and Caleb would be in the spotlight.
But, Moses and Aaron are still there, against the crowd of people, just in the background.
Trying to 'stone The Lord' by proxy
Several verses that follow record God interpreting this as a rebellion—not against Moses, Aaron, Joshua, or Caleb—but against God Himself! (Num 14:11, 22, 23, 24, 27, 29, 35)
This is a similar view that God took when Israel rejected God by proxy of rejecting Samuel (1 Sam 8:7).
God was just as much the object of this assembly's wrath as the others. From God's words, He took the greatest offense of all. So, God should be regarded as the primary object of this particular rebellion.
Two main hermeneutical methods applied here
- Frequent mention: The four men, and also God in a different sense, are mentioned too many times to be exempt from this angry crowd's anger. It must be about all five parties because they all keep coming up; even God brings up Joshua and Caleb while talking to Moses (Num 14:24, 30).
- Theater: One of my professors at Moody, Dr. Laansma, sat with me in his office and explained that the way to understand the Gospel of John was like a stage play: A crowd comes onto the stage, something happens, everyone moves to the side while Jesus comes forward and talks and talks and talks and talks, front and center. That's just one example of how John has a different "scene setup" from the Synoptic Gospels and other Bible stories. Imagining the scene, where people would have been standing, et cetera, is somewhat as important as the maps at the back to provide context. This was a real event, so we must visualize what that reality would have looked like in order to understand it better. By imaging exactly what the text describes—Moses and Aaron on their faces, Joshua and Caleb trying to persuade an angry mob, then God flashing and shouting—we get a really good "picture" of who is angry at who. And, that helps answer this excellent Bible question.
On a hermeneutical-homiletical note, we may emphasize any of these four men (or God) in a Bible study lesson or sermon, but we cannot rule out anyone to say that the crowd was only seeking to stone one or some, but not the others. The reason we can't is because there are no words to that effect whatsoever of "singling-out", so we can't draw such a conclusion either.