Reading through the text in question one is automatically reminded of the prophecy of Baalam in Numbers 24:17 which states:
17 I shall see him, but not now: I shall behold him, but not nigh:
there shall come a Star out of Jacob, and a Sceptre shall rise out of
Israel, and shall smite the corners of Moab, and destroy all the
children of Sheth. (Num. 24:17 KJV)
The particular portion of this prophecy that begs consideration is the "smiting of the corners of Moab," which may commentators agree is fulfilled in the actions of David in the text we are discussing. Essentially, the language of the measuring line is indicating that two-thirds were put to death and one-third was kept alive which, by the way, was a much lessor punishment than those of Edom (I Kings 11:6) where all the males were cut off.
Biblically, there is a special enmity that God had toward the Moabites (even though David's ancestry through Ruth associates with the Moabites and his parents were placed in their care for a time)(I Sam. 22:3-4).
However, as the Bible Exposition Commentary states:
Because the Moabites had hired Balaam the prophet to curse Israel and
then led Moab in seducing the men of Israel (Num. 22-25), the Lord
declared war on Moab, and David was only continuing that crusade. Most
conquerors would have slaughtered the entire army, but David spared
every third soldier and settled for tribute from the nation.
This act of judgment is unparalleled in scripture and antiquity though, as stated in the Holman OT Commentary:
In an unparalleled act of judgment on an enemy, David made the
captured Moabite fighters lie down on the ground, then measured them
off with a length of cord. Every two lengths of them were put to
death, but the third length of warriors was allowed to live. This
action, however unthinkable it might seem today, served the dual
purpose of reducing the ranks of Moab's fighters to a nonthreatening
minimum, while at the same time insuring that Moabite men would be
able to produce tribute that would benefit the Israelites.
While we could go forward casting the narrative of unprecedented judgment, the reality of this severity goes backward to the war that God waged against Moab and its realization in the actions of David in the text you brought your question. Indeed, David fulfilled the smiting of the corners of Moab. The word "smite", as stated in the Complete Word Study Dictionary of the Old Testament,
māḥaṣ: A verb meaning to wound severely, to pierce through, and to
shatter. This word describes bodily destruction and is best
illustrated in Judges 5:26, where Jael pierced through Sisera's head
from temple to temple with a tent peg. David used this word to
describe some of his victories in which those wounded were not able to
rise again (2 Sam. 22:39; Ps. 18:38[39]). In all other instances of
this word, God is in complete control (Deut. 32:39; Job 5:18) and
completely shatters His enemies (Ps. 68:21[22]; 110:5, 6; Hab. 3:13).
This word occurs only in the poetical passages of the Old Testament,
which highlights the intensity of this word.
Of course, the ultimate realization of Balaam's prophecy would occur in Jesus Christ but, in answering your initial question, the measuring was literal. Hope this helps...