The words apply to the idols. In reading this in English translation, the KJV is obscure, although with some effort you can read it as having this meaning; the NIV is much better: "Like a scarecrow in a cucumber field, their idols cannot speak; they must be carried because they cannot walk ..." The NAB is clear and gives us the picture Jeremiah had in mind. I include verses 3-6 for context:
"For the cult idols of the nations are nothing, wood cut from the
forest, Wrought by craftsmen with the adze, adorned with silver and
gold. With nails and hammers they are fastened, that they may not
totter. Like a scarecrow in a cucumber field are they, they cannot
speak; They must be carried about, for they cannot walk. Fear them
not, they can do no harm, neither is it in their power to do good. No
one is like you, O LORD, great are you, great and mighty is your
name."
With proper emphasis, we can see that Jeremiah chapter 10 is not at all about those who cut down trees, but about the fact that statues of gods are merely wood, cut from trees and adorned with silver and gold. Chapter 10 is part of a longer invective against the sins of Jerusalem: violence by neighbour against neighbour, deceit upon deceit, and they no longer know God. In Jeremiah 9:11-14, God will lay waste to the cities of Judah and make Jerusalem a heap of ruins and a haunt of jackals, because they have abandoned his laws and followed Baalim (an exaggeration, because Yahweh was the supreme national God of Judah, as he had been for centuries, even with other gods in the pantheon).