Two reasons barrenness was undesirable
In antiquity there were typically two reasons that barrenness was undesirable. The first, which isn't really an issue in this text had to do with the security of the future. Children were the ancient equivalent of a retirement plan since there were no pensions, social security, etc. Therefore, the only ones to care for your welfare in your old age were your children and their family.
The second reason that barrenness was undesirable is an extension of the belief that maladies were the result of sin. Barrenness is an extension of that idea. We see this played out in Genesis 20 in which King Abimelech's wife and servants are barren because of Abraham's lie, in 2 Sam 6:20-23 where Michal, Saul's daughter mocks King David rendering her barren, and in Genesis 30:23 where Rachel remarks "God has removed my reproach" upon becoming pregnant.
In his commentary on Luke, Bruce Barton notes,
Zechariah and Elizabeth were both faithful people, yet they had been suffering. Some Jews at that time did not believe in a bodily resurrection, so their hope of immortality was in their children. In addition, children cared for their parents in their old age and added to the family’s financial security and social status. Children were considered a blessing, and childlessness was seen as a curse. Zechariah and Elizabeth had been childless for many years, but God was waiting for the right time to encourage them and take away their disgrace.
Two causes: Cause 1 - Unfaithfulness to God
Within scripture, there were two main reasons given for barrenness. The first reason comes from Exodus 23:24-26:
Do not bow down before their gods or worship them or follow their practices. You must demolish them and break their sacred stones to pieces. Worship the Lord your God, and his blessing will be on your food and water. I will take away sickness from among you, and none will miscarry or be barren in your land. I will give you a full life span.
And from Deuteronomy 7:12-15:
If you pay attention to these laws and are careful to follow them, then the Lord your God will keep his covenant of love with you, as he swore to your ancestors. He will love you and bless you and increase your numbers. He will bless the fruit of your womb, the crops of your land—your grain, new wine and olive oil—the calves of your herds and the lambs of your flocks in the land he swore to your ancestors to give you. You will be blessed more than any other people; none of your men or women will be childless, nor will any of your livestock be without young. The Lord will keep you free from every disease. He will not inflict on you the horrible diseases you knew in Egypt, but he will inflict them on all who hate you.
These both explain that as long as Israel is faithful to God, there will be no barren Israelite women. In the context of Elizabeth, the Old Testament (judges, the minor prophets, etc) and the Roman occupation of Israel had made it clear that Israel had been unfaithful to their covenant and had lost the land and God's favor. It would therefore be assumed that there would be barrenness in the land, and Elizabeth's barrenness could certainly be attributed to that were it not for the second cause of barrenness
Two Causes: Cause 2 - Infidelity
In Numbers 5:11-31 Israelite men are given a conditional curse from God that may be performed at the temple or tabernacle by a priest in order to determine if a woman suspected of infidelity was cheating. This curse only struck the participant if she had been unfaithful to her husband. When one remembers Genesis 20 in which infidelity caused barrenness and the idea that flagrant enough sins against God could cause all sorts of illnesses and deformities, the gossip about town would have been that "perhaps Elizabeth was barren because she had been unfaithful, but certainly she must have committed some sin to render her in this state" - thus leading to her sense of public shame.
Dr. I. Howard Marshall ultimately reaches a similar conclusion in his commentary, stating
Elizabeth interprets her pregnancy as being due to the gracious act of God. ἐφοράω (Acts 4:29**) is a variant for ἐπισκέπτομαι, ‘to visit’, and refers to God’s action in removing the barrenness which was regarded as a severe reproach by Jewish women (Gn. 16:4; 30:1; Dt. 28:18; 1 Sa. 1:6; 2 Sa. 6:23; SB II, 98). Like Rachel (Gn. 30:23) she praises the God who removes barrenness (Ps. 113:9).
Ultimately, the cause of Elizabeth's embarrassment was the suggestion that she had sinned; possibly by being unfaithful to her husband. This humiliation as a result of being unable to get pregnant provides an interesting literary contrast to Mary who faces public shame because can't help but get pregnant, yet in both cases, neither woman sinned and neither woman was unfaithful to their husbands. In both cases, it is faithfulness to God which results in their pregnancy - in one case leading to a resolution to public shame and in the other case causing it.