This is my attempted answer:
God’s commandment in Deuteronomy 24:16 is to be applied to the Israelites themselves (e.g., they were not to stone a child to death for the sin of a father). However, it is not intended to limit God, who keeps His prerogative to mete out punishment as He sees fit. We therefore see throughout the Old Testament examples of children and infants dying for the sins of their parents:
(i) All of the children of the world were put to death in the great flood that was on the earth for 40 days (Gen 7:17).
(ii) God sent fire and brimstone from heaven (Gen 19:24) and overthrew Sodom and Gomorrah and all of their inhabitants (Gen 19:25). It is reasonable to assume that children and infants living in Sodom and Gomorrah were also put to death.
(iii) God commanded Saul to kill all of the Amelkites, including the children and infants (1Sa 15:1-3).
(iv) Nathan told King David that because of David’s sin, his child would die: “However, because by this deed you have given great occasion to the enemies of the Lord to blaspheme, the child also who is born to you shall surely die” (2Sa 12:14). The Lord struck David’s child with an illness (2Sa 12:15) and the child died after seven days (2Sa 12:18).
Whereas the Israelites were not permitted to mete out these punishments to the innocent children of sinners, God was permitted to do so.
I am not particularly fond of this explanation because it depicts God as harsh and unfair, but it is the only way I can reconcile the apparent contradiction between what is said in Deuteronomy 24:16 and what God actually does.