A little known fact that is often left out of this type of discussion is that we ourselves do not have 24-hour days. It also depends on what definition of "day" you use, you could use the "one full rotation of the Earth around its axis" (rotation period or sidereal day) or "the time it takes for the sun to go back to its position in the sky" (solar day).
For an excellent explanation of this see How long is a day?
Some relevant quotes from that site:
It actually takes the Earth slightly over 23 hours and 56 minutes to
rotate once around this axis. In this time all the stars appear to
revolve once around the Earth and return to their starting positions.
Astronomers call this period of time a sidereal day.
But the Earth's axis is inclined at an angle of about 23.5° from the
perpendicular to the plane of the Earth's orbit. And although the
Earth's orbit is close to being circular, it is actually an ellipse,
so it is shaped like a slightly squashed circle.
These two factors mean that the time taken for the Sun to return to a
position due south in the sky varies slightly throughout the year. On
average this period of time is 24 hours. But at some times of the year
it is slightly longer and at other times of the year it is slightly
So, how long is a day? We cannot say that a day is by necessity 24 hours. And therefore we also cannot say that a yom is 24 hours either. Especially since the sun (or "greater light to rule the day") was not created until the fourth day (Genesis 1:16).
In other words, it is eisegetical to insert the meaning of "24 hours" to the word "day" (even in English), since even in our present time, it is not so. We say a day is 24 hours out of convention and pragmatism (imagine if we started using strict sidereal days or solar days, the mess that would create), not any other reason.
As other answers point out, the day "yom" is not used to refer only to a solar day or rotation period, but can refer to events such as "the Day of the Lord", "the day I took you out of Egypt", etc. Again, as noted in other answers, you can check any concordance for this, and if you can read basic Hebrew, the BDB (Brown, Driver and Briggs A Hebrew and English Lexicon of the Old Testament) for occurrences of "יוֹם" and how it is used in each given context.
Another point is to say that in Genesis 1:5, God calls the light "day", not a specific quantity of time.
If we translate "yom" in Genesis 1 to "aeon" that would add unnecessary interpretation and perhaps even greater confusion, depending on how people understand the word "aeon" (an explanation of why this cannot be thousands or millions of years would require the use of doctrine, so would belong better in a different forum than this one). It would also create a greater conflict since the fact that the creation model of 6 yomim with the 7th yom for rest was what the law of the Sabbath was based upon.
"For in six days the LORD made the heavens and the earth, the sea and
all that is in them, and rested on the seventh day; therefore the LORD
blessed the sabbath day and made it holy." Exodus 20:11 (NASB)
So it is important to keep the translation coherent with this reasoning, hence the choice of "day" for other languages too.
To answer your final question, the controversy is actually only a century or two old, when people started believing in evolution and saying that the earth was millions of years old (and now billions). This obviously conflicted with the biblical account of how long life has been on Earth (notice I didn't say "of how old Earth is"), and the polemic evolved (pardon the pun) as people chose sides in the matter. Only then did some find it necessary to find a different meaning for the word "yom", to attempt to reconcile wanting to sit on both sides of the fence.