Answering questions of Why
The main hermeneutic approach for explaining why things happened in the Old Testament is that the events of the old testament are prophecies of Christ, all relating to Christ somehow, and then the early church fathers sought to answer "why" in terms of the prophetic nature of the events.
For example, we can ask "why did Isaac need to be sacrificed?", and we see that it is a type of Christ being sacrificed. And then we ask why was Isaac spared and the ram used? and answer that it is a type of the Christ's atonement for us. We can ask 'why did God create Eve from Adam's side?', and say it is a type for the bride coming into being from the wounds of Christ, being created when he was in "deep sleep", a type for death, etc. 'Why was Jonah in the whale for three days?' will yield also answers via the same hermeneutic.
Of course one can have different hermenueutic approaches and say that Eve was pulled from Christ's side as a commentary on womens' rights, etc, but then you need to specify what the approach is and try to apply it consistently throughout the Old Testament and then see how fruitful this approach turns out to be.
Now the advantage of the prophetic approach is that it was endorsed and used by Christ himself, by other books of the New Testament, and by Church fathers. So let's at least give it a try when trying to understand the old testament.
Spiritual significance of Amalek
Amalek was the tribe that waylaid the Israelites on their way out of Egypt but before they entered the promised land. Amalek is not one of the seven tribes in Canaan that needed to be conquered (c.f. Deut 7.1-2), it is a tribe that needed to be overcome in order to even enter the Promise.
Now we learn in Hebrews 3.15-19 that the promised land is equated with rest, the true Sabbath, and what prevented the Israelites from entering that rest was unbelief. This unbelief is associated with "dead works", or our own works, in which we toil because we don't believe God's provision is sufficient.
Thus this is a clue that Amalek represents unbelief, and specifically unbelief in God's power/provision. Now we have a candidate theory -- let's see if this theory explains Amalek in the scriptures:
The root '-M-L means "toil, labor".
Gen 36.16, Amalek is one of the three chiefs of Edom. Edom was a pun for "Adam" and thus a synecdoche for "heathens", "gentiles" just as Ephraim was a synecdoche for the northern tribes.
In Exodus 17.3-7, the people grumble that God is going to kill them with thirst in the desert, and then Amalek attacks in verse 17.8 "And Amalek came and fought with Israel at Rephidim.". Thus Amalek appears when the Israelites doubt God's power to provide for them.
When Joshua battles Amalek, the real battle is fought by Moses when he is praying. As he is praying, Amalek is being defeated, but when he stops, Amalek comes back (Ex 17.11). Amalek is not destroyed but beaten off. This is symbolic of unbelief not being totally eradicated but beaten off as we lift our hands in prayer.
In Exodus 17.16, we are told that the war between Amalek will be waged from generation to generation. Thus this is no ordinary war for territory, as Amalek's territory isn't even in the promised land. This is a war against unbelief.
In Deut 25.19, the Israelites are commanded to blot out the remembrance of Amalek. There are few sins that we are not allowed to even remember, but unbelief is such a thing.
In Judges the Amalekites (along with other tribes) cross the Jordan and conquer Israel again. They are delivered by Gideon, but really defeated by the bread that comes from Heaven, that is, Christ:
Judges 7.13:
When Gideon came, a man was recounting a dream to his friend, and he
said, “Behold, I had a dream; a round loaf of barley bread was
tumbling into the camp of Midian, and it came up to the tent, it
struck it, and it fell and turned it upside down so that the tent
fell.” [LEB]
..which itself is a reverse parable of Satan coming to steal the seed that falls to the ground:
Judges 6.3
And whenever Israel sowed seed, the Midianites, Amalekites, and the
people of the east would come up against them. [LEB]
Why Amalek must be completely destroyed
So now we see the reason why Amalek must be completely destroyed including all the women, children, cattle, etc. Although a remnant of gentiles can be saved and the people of God can take resources from the world and use them for their own benefit, but unbelief cannot so be used. It has to be completely blotted out, even though in this world it is a battle that must be fought in every generation.
Thus the people of Amalek needed to be completely destroyed because they were Amalekites and thus represented unbelief, and not for any other reason.
Note that some people will be offended at the idea that human beings can be mere metaphors for spiritual things, except that's what physical reality is. It's merely a metaphor - a type, or shadow - for the things of the spirit. It is not, as many believe, that the physical world is real and the spiritual world is a shadow of the physical. The situation is reversed!
Job 8.9 [LEB]
for we are of yesterday, and we do not know, for our days on earth are
a shadow.