Updated Answer due to new insights
(See original answer below this one)
In addition to my other cross-references and probably eclipsing them all is this one in Numbers. God tells Moses he's decided to disinherit the Jews and make from Moses a "new creation":
[Num 14:11-12 KJV] (11) And the LORD said unto Moses, How long will this people provoke me? and how long will it be ere they believe me, for all the signs which I have shewed among them? (12) I will smite them with the pestilence, and disinherit them, and will make of thee a greater nation and mightier than they.
However, Moses dissuades him because of his oath:
[Num 14:13-16 KJV] (13) And Moses said unto the LORD, Then the Egyptians shall hear it, (for thou broughtest up this people in thy might from among them;) (14) And they will tell it to the inhabitants of this land: for they have heard that thou LORD art among this people, that thou LORD art seen face to face, and that thy cloud standeth over them, and that thou goest before them, by day time in a pillar of a cloud, and in a pillar of fire by night. (15) Now if thou shalt kill all this people as one man, then the nations which have heard the fame of thee will speak, saying, (16) Because the LORD was not able to bring this people into the land which he sware unto them, therefore he hath slain them in the wilderness.
To renege on an oath was perjury and a very, very serious matter in the ancient world and a perjurer loses all credibility:
...No oath can be administered by a court to one who has once perjured himself in any of the cases mentioned above, even though the litigant against whom he had been called is willing to believe him on his oath; nor is his testimony admitted in evidence (Shebu. vii. 4; "Yad," To'en, ii. 12; Shulḥan 'Aruk, Ḥoshen Mishpaṭ, 34, 5; 92, 1, 2). If, however, he undergoes the punishment of scourging, or manifests such signs of contrition that the court is satisfied that he sincerely regrets his transgression, he is restored to his former position in the community ("Yad," 'Edut, xii. 9; ib. To'en, ii. 9, 10; see "Kesef Mishneh" and "Leḥem Mishneh" ad loc.; Ḥoshen Mishpaṭ, 34, 33; 92, 14).
So instead, he kills everyone of that generation except for two men and brings in the next generation instead. This seems to be the case for Israel. They had many judgments but all the promises to them were fulfilled in the time of the Messiah, then he destroyed their temple and thus their special relationship to himself as a nation.
Original Answer
Below are several suggestions from scripture why the final generation of the Jewish covenants received the stiff punishment that it did. The last one stands out as the most relevant to the question:
- Proverbs may be saying they were destined to be exceptional:
Pro 30:11-14 KJV - 11 There is a generation that curseth their father, and doth not bless their mother. 12 There is a generation that are pure in their own eyes, and yet is not washed from their filthiness. 13 There is a generation, O how lofty are their eyes! and their eyelids are lifted up. 14 There is a generation, whose teeth are as swords, and their jaw teeth as knives, to devour the poor from off the earth, and the needy from among men.
- to follow the examples of Sodom and Gomorrha and to act as an example:
Jde 1:7 KJV - 7 Even as Sodom and Gomorrha, and the cities about them in like manner, giving themselves over to fornication, and going after strange flesh, are set forth for an example, suffering the vengeance of eternal fire.
- to give them space to repent
Rom 2:4 ESV - 4 Or do you presume on the riches of his kindness and forbearance and patience, not knowing that God's kindness is meant to lead you to repentance?
Rev 2:21 KJV - 21 And I gave her space to repent of her fornication; and she repented not.
Mat 23:37 KJV - 37 O Jerusalem, Jerusalem, thou that killest the prophets, and stonest them which are sent unto thee, how often would I have gathered thy children together, even as a hen gathereth her chickens under her wings, and ye would not!
- God's judgment day had not yet been set:
Act 17:30-31 KJV - 30 And the times of this ignorance God winked at; but now commandeth all men every where to repent: 31 Because he hath appointed a day, in the which he will judge the world in righteousness by that man whom he hath ordained; whereof he hath given assurance unto all men, in that he hath raised him from the dead.
- Out of God's concern for his own reputation:
Deu 9:28 KJV - 28 Lest the land whence thou broughtest us out say, Because the LORD was not able to bring them into the land which he promised them, and because he hated them, he hath brought them out to slay them in the wilderness.
- they rejected the messiah, the spirit and the gospel and persecuted the apostles:
Jhn 19:15 KJV - 15 But they cried out, Away with him, away with him, crucify him. Pilate saith unto them, Shall I crucify your King? The chief priests answered, We have no king but Caesar.
Act 7:57-58 KJV - 57 Then they cried out with a loud voice, and stopped their ears, and ran upon him with one accord, 58 And cast him out of the city, and stoned him: and the witnesses laid down their clothes at a young man's feet, whose name was Saul.
Update (after reading Mac's insightful answer)
There were prior punishments but the permanent destruction of the temple (and the covenants) was because of the accumulated sin of the People since day one.
Another update!
It occurs to me that this is an example of "substitutionary punishment"!