It means the He should have rule and dominion. Young's Literal Translation renders this as:
"For a Child hath been born to us, A Son hath been given to us, And the princely power is on his shoulder, And He doth call his name Wonderful, Counsellor, Mighty God, Father of Eternity, Prince of Peace."
There appears to have been a practice in the east of wearing a key on the shoulder as an emblem of the office and authority to open the king's treasury, or the king's palace, apparent so that all would know the importance of his personage.
Clarke's commentary includes:
"That is, the ensign of government; the scepter, the sword, the key, or the like, which was borne upon or hung from the shoulder. " (1)
Barnes Notes on this verse include:
"'How much was I delighted when I first saw the people, especially the Moors, going along the streets with each his key on his shoulder. The handle is generally made of brass (though sometimes of silver), and is often nicely worked in a device of filigrane. The way it is carried is to have the corner of a kerchief tied to the ring; the key is then placed on the shoulder, and the kerchief hangs down in front. At other times they have a bunch of large keys, and then they have half on one side of the shoulder, and half on the other. For a man thus to march along with a large key on his shoulder, shows at once that he is a person of consequence." (2)
As well as Jamieson-Faussett-Brown:
'So keys are carried sometimes in the East, hanging from the kerchief on the shoulder. But the phrase is rather figurative for sustaining the government on one's shoulders." (3)
As an aside, the English word "government" has been slipped into the English translations for a political purpose, and has caused a great deal of confusion. King James was in spiritual warfare with the Pope at Rome maintaining the divine right of kings to govern and rule their people; which concept did not sit well with the Pope. When King James commissioned the English translation of the Bible in 1604, the Jesuits conspired to assassinate King James in what was known as the Gunpowder Plot of 1605.
Of course, this attempt to murder him did not endear the Pope to King James and the political motive to separate fully all rule and authority from Rome was made clear in the translation of the Bible known as the Authorized Version of 1611. (4) (5)
The word "government" used in the English translations needs further discussion as the intent of the KJV was politically motivated to support King James' contention for his divine right to rule England alone. The original Hebrew and Greek words are not "government".
For instance, 1 Cor. 12:28 reads in the KJV as:
"And God hath set some in the church, first apostles, secondarily prophets, thirdly teachers, after that miracles, then gifts of healings, helps, governments, diversities of tongues."
But in the Interlinear, that word "governments" is actually "κυβερνήσεις" and is Strong's Gr. 2941 "kubernésis" and means a helmsman steering or piloting a ship. (6)
The subject matter of 1 Cor. 12:28 are those whom God set in the "church" or assembly and spoke of the order of their authority. It is not speaking of a secular political governing body. This has great implications on the current understanding and traditional teaching of Romans 13.
Notes:
1) Source: here
2) Source: Biblehub
3) Ibid
4) Source: King James I and the Divine Right of Kings
5) Source: The Story Behind the King James Bible
6) Source: Biblehub