First I checked that the same phrase appears in both Col 3:16 and Eph 5:19. It does.
psalmois, humnois, kai odais pneumatikais
I then checked those words out in the lexicons and compared the words they translated in the LXX.
psalmois - often for neginah, which means song, or mizmor, also meaning song. Used 92 times in the LXX but mostly in the title lines of psalms (also the title lines of the Psalms of Solomon, an additional book in the LXX). In Job (21:12 and 30:31) it refers to flute songs. In Amos 5:23 it translates "music." In Zec 6:14, its phrase is not in the Hebrew. Isa 66:20 "with psalms" actually translates "in clean vessels." Obviously some paraphrase going on. Lam 3:14 and 5:14 again it translates neginah.
humnois - tehillah, a song of praise, thanksgiving, or adoration to God. 32 times in the LXX, 16 of those are in books that are not in the Protestant Bibles.
odais pneumatikais - odais is simply a word for songs (so is, shir, the Hebrew it translates). Pneumatikais means spiritual.
Then I hit pay dirt when I checked The Theological Wordbook of the Old Testament (s.v. shir).
It is also unclear if there is any appreciable difference between mizmœr "a psalm" and shîr "a song." Rabbinic interpretation (the Midrash to Psalms) recognized mizmœr as a psalm accompanied by instruments and shîr mizmœr as a psalm sung by a choral group alone. Several things are clear, however, in relating these two words. One is that the noun mizmœr appears exclusively in the Psalms and there always as a title and never in the body of a psalm. By contrast, shîr is not confined to the Psalter and within the Psalter itself is used both as a title and in the psalm proper. Second, although mizmœr as a noun is limited to psalm superscriptions the verbal form (from which the noun is derived), z¹mar, occurs in the Psalms and often in parallelism with the verb shîr : Psa 21:13 [H 14]: "We will sing (shîr) and praise (z¹mar) your power." This is comparable to the synonymous arrangement of the roots yšr/¼mr in Ugaritic (for example, Dahood, in Psalms, 11, AB, p. 54 on Psa 57:7 [H 8]).
A third observation to be made is that mizmœr is limited to religious song. shîr, on the other hand, may occasionally refer to secular songs. Isa 23:16 refers to the songs of the harlot (Tyre). Amos 8:10 would indicate that religious songs can be turned into lamentations just as feasts can be turned into funerals.
...
It is no accident that the hymn and the lament are the two leading types of psalms in the Psalter, both illustrating in turn the two ways of addressing God: praise and petition. In one way the hymn is an expansion of the lament. To illustrate, Psa 57 begins as a lament: "Take pity on me God... I take shelter in the shadow of your wings until the destroying storm is over... I lie surrounded by lions" (JB). By v. 7 [H 8], however, the Psalmist says, "My heart is steadfast, I will sing and make melody." The abrupt change of mood is obvious. Thus, most of the laments evolve into songs of praise in anticipation of God's deliverance. The Hebrew could always sing to his God in spite of forbidding circumstances.
The hymn and the song of thanksgiving praise God and sing to God not in anticipation for an expected deliverance but in response to something already experienced. And even here there is a distinction. One, the hymn, praises God for his actions or extols him for who he is (descriptive praise: God is... God does....). The song of thanksgiving praises God for a specific deed (declarative praise: God has...) (Westermann, p. 31ff, emphasis added).
With these three words, Paul is telling us to sing songs with instruments, and acapella, both about who God is (e.g. How Great Thou Art) and what He has done (e.g. Amazing Grace). And that God accepts both praise and lament as they come from the heart of the believer.