In Exodus 20:4 there is a phrase בַּמַּיִם מִתַּחַת לָאָרֶץ that is often translated as "in the water under the earth". What location is this a reference to?
The word תַּחַת is often translated "under". In Ex 19:17 it means "at the base of" (the people do not stand literally under a mountain), and it sometimes has other meanings. The meaning here is almost certainly positional in some way due to the mem prefix and adjoined lamed (see section 7b here).
The simplest explanation is that מִתַּחַת refers to all bodies of water. Water is always lower than -- below -- the adjacent land. Some water actually comes from below ground too, like springs, and we learn from Genesis 7:11 that sea-water comes from the depths below. But the present passage isn't telling us that the only forbidden inspirations for aquatic idols come from below the earth we're standing on; it's a broader reference to oceans, lakes, rivers, etc.
In the Masoretic tradition the phrase וַאֲשֶׁר בַּמַּיִם | מִתַּחַת לָאָרֶץ: is one phrase, not two, but there is a slight pause after "in the waters" before "under the earth" (where the vertical bar is in what I just quoted). Think of this as a comma but not a semicolon nor a full stop. I'm not sure if this affects how we should interpret the text, but I thought I'd point it out. The trope (which implies punctuation) is not part of the original text but is a received tradition recorded centuries later.
Please note: This answer was written for a neutral, academic audience and is not intended to be interpreted in the context of a religious belief or doctrine.
In fact here you have to merge two things. From the context you can understand that the writer refers to marine creatures and things. Translating Exodus 20:4 in modern terms: nothing coming from lithosphere, atmosphere and hydrosphere can become an idol.
Here is the point: what was hydrosphere for the writer? In fact wasn't a "sphere" at all. In the flat earth model we have water surrounding the perimeter of the lands, but we should consider even the hypothesis of water beneath the lands. In fact the term מִתַּחַת from base word תחת (ta-hhat) is a noun meaning "under," "below" or "lower." The prefix מ (me) means "from"1.
On this site you can find others biblical passages about waters under earth. Is quite obvious that ancient cultures thought about waters under earth: if you think that digging holes in the ground (e.g. building wells) many times they found water. So is possible that the authors thought that marine creatures, and others potential idols, could be in the waters beneath earth as well as in the seas.
1. Ancient Hebrew Research Center, Biblical Hebrew E-Magazine July, 2006, Issue #029