For there our captors
required of us songs,
and our tormentors, mirth, saying,
“Sing us one of the songs of Zion!”
How shall we sing the LORD's song
in a foreign land? (Psalm 137:3-4) [ESV]
A common explanation is given by Adele Berlin and Marc Zvi Brettler in their commentary:
2-4 The Babylonian captors demand musical entertainment but the captives, who can only cry, hang up their musical instruments and refuse to make music. They cannot express joy as long as they are in exile. Joy, which is synonymous with with being in God's presence, is no longer possible when the Temple is destroyed. Exile is equated with descent into the world of the dead; like the dead, the exiles are unable to praise God. (30.10; 88.11-13)
3-4 Songs of Zion, some scholars identify Zion songs as specific types of hymns abut the Temple (46; 48) or as the pilgrimage psalms (84; 120-134). More likely, the Babylonians are asking for any native Judean song. The psalmist equates them with a song of the LORD, that is, any song sung in the Temple, and therefore they can no longer be sung.
Songs are an expression of the joy which is found in the presence of God. They are sung in the Temple, or while making a pilgrimage to the Temple. Unlike the past where they were able to journey to the Temple, the exiles are like the dead, and are unable to praise God.
Another possibility is found n Deuteronomy, where the apostasy of the nation had been predicted:
28 Assemble to me all the elders of your tribes and your officers, that I may speak these words in their ears and call heaven and earth to witness against them. 29 For I know that after my death you will surely act corruptly and turn aside from the way that I have commanded you. And in the days to come evil will befall you, because you will do what is evil in the sight of the LORD, provoking him to anger through the work of your hands.” (Deuteronomy 31) [ESV]
The LORD promised the people would be exiled if they turned to other gods:
64 “And the LORD will scatter you among all peoples, from one end of the earth to the other, and there you shall serve other gods of wood and stone, which neither you nor your fathers have known. 65 And among these nations you shall find no respite, and there shall be no resting place for the sole of your foot, but the LORD will give you there a trembling heart and failing eyes and a languishing soul. 66 Your life shall hang in doubt before you. Night and day you shall be in dread and have no assurance of your life. 67 In the morning you shall say, ‘If only it were evening!’ and at evening you shall say, ‘If only it were morning!’ because of the dread that your heart shall feel, and the sights that your eyes shall see. (Deuteronomy 28)
The LORD had Moses teach the people a song:
16 And the LORD said to Moses, “Behold, you are about to lie down with your fathers. Then this people will rise and whore after the foreign gods among them in the land that they are entering, and they will forsake me and break my covenant that I have made with them. 17 Then my anger will be kindled against them in that day, and I will forsake them and hide my face from them, and they will be devoured. And many evils and troubles will come upon them, so that they will say in that day, ‘Have not these evils come upon us because our God is not among us?’ 18 And I will surely hide my face in that day because of all the evil that they have done, because they have turned to other gods. 19 “Now therefore write this song and teach it to the people of Israel. Put it in their mouths, that this song may be a witness for me against the people of Israel. 20 For when I have brought them into the land flowing with milk and honey, which I swore to give to their fathers, and they have eaten and are full and grown fat, they will turn to other gods and serve them, and despise me and break my covenant. 21 And when many evils and troubles have come upon them, this song shall confront them as a witness (for it will live unforgotten in the mouths of their offspring). For I know what they are inclined to do even today, before I have brought them into the land that I swore to give.” 22 So Moses wrote this song the same day and taught it to the people of Israel. (Deuteronomy 31 ESV)
This song could rightly be called a song of the LORD. It was to serve as a witness for the LORD against the people of Israel. The exiles refused to sing the song which "would confront them as a witness".
The refusal to sing the song is both a type of acknowledgement of their past wrong actions and a rejection of the meaning the song carries. The people will not sing as a witness against themselves. Instead, they will not forget Jerusalem (137:5-6) and will call upon the LORD to punish the Edomites and Babylonians.
- Adele Berlin and Marc Zvi Brettler, The Jewish Study Bible, Oxford University Press, 2004, p. 1435