Subsequent to the publication of Greek Myths and Christian Mystery, referenced by another answer, Manolis Papoutsakis made an ingenious hypothesis that may finally solve the mystery as to how the odd translation arose.
In his paper, "Ostriches into Sirens: Towards an Understanding of a Septuagint Crux" (Journal of Jewish Studies, Vol. LV, No. 1, Spring 2004), he proposes that the usage of derives from paronomasia - intentional word play whereby the Aramaic word for Ostrich is associated with Naamah from Genesis 4. Below is a summary of his argument.
The mystery of how the Greek Σειρνε ("Sirens") entered into the Septuagint begins in an unlikely place - Genesis 4:19-22 (ESV):
And Lamech took two wives. The name of the one was Adah, and the name of the other Zillah. Adah bore Jabal; he was the father of those who dwell in tents and have livestock. His brother’s name was Jubal; he was the father of all those who play the lyre and pipe. Zillah also bore Tubal-cain; he was the forger of all instruments of bronze and iron. The sister of Tubal-cain was Naamah.
This passage outlines some of the decedents of Cain. At first glance, it does not seem to offer even the slightest clue as the where the Sirens came from. However, in the Targums (authorized Aramaic paraphrases of the Torah read in synagogues) something very interesting happens: three of the four extant Targums expand the account, associating Naamah with singing.
- Targum Pseudo-Jonathan: (all translations throughout are by Papoutsakis)
And the sister of Tubal-cain was Naamah; she was the mistress of qinin and songs.
And the sister of Tubal-cain was Naamah; she was the creator of qinin and songs
And the sister of Tubal-cain was Naamah, the inventor of qinin and
The precise meaning of qinin is unclear, but is some sort of music. Based on the Peshitta (which has the same expansion of the verse) use of ܩܝܬܪܐ, Papoutsakis concludes the word has a neutral or positive connotation. Interestingly, in the Syriac version of Physiologus ܩܝܬܪܐ is used to describe the song of the Sirens.
Based on the Targum evidence, it is thus likely that there was a long-standing tradition that associated Naamah with musical skill and singing. This conclusion is also supported by Bereshit Rabbah 23:22:
And the sister of Tubal-cain was Naamah. R. Abba b. Kahana said: Naamah was Noah’s wife; and why was she called Naamah? Because her deeds were pleasing. The Rabbis said: Naamah was a woman of a different stamp, for the name denotes that she sang to the timbrel in honour of idolatry
Furthermore, previous research (B. A. Pearson, "The Figure of Norea in Gnostic Literature" and other articles) was able to equate Naamah with Norea, a figure of great beauty found in various apocryphal writings.
As such it is not hard to see how Naamah, a figure of great beauty and musical talent who descended from the evil Cain, could become associated with the Greek idea of a Siren, an evil figure of great beauty and musical prowess.
But how does this get us any closer to understanding the use of Σειρνε in the Septuagint? The word does not substitute for Naamah in the Septuagint, after all. The key lies in realizing that the Aramaic words for ostrich and Naamah are homophones (נעמא).
To show the relevance of this observation, Papoutsakis demonstrates that the Septuagint translators where very familiar with Aramaic. For example, he cites Aramaicisms found is the translation of Isaiah. Thus, the translators read Isaiah 13:21 as an instance of paronomasia. Since they couldn't convey the dual meaning of ostrich/alluring musician present in the contemporary (Aramaic) understanding of the word, they chose to go with the alluring aspect, translating it as Sirens. (The Greek translation of Isaiah is pretty "loose" in general, preferring to convey ideas instead of precise wording.)
This hypothesis directly addresses three of the six cases of Σειρνε - Isaiah 13:21, Jeremiah 27:39, and Micah 1:8 - where it corresponds with the בנות יענה of the unpointed Masoretic text (the "owls" of the OP, but English translations vary; "ostriches" in this answer, following Papoutsakis).
In the remaining three cases (Isaiah 34:13 and 43:20, and Job 30:29), Σειρνε corresponds instead to תנים (the "jackals" of the OP, but again translation vary). Thus, Papoutsakis must offer an explanation for the discrepancy.
In the case of the two remaining Isaiah passages, the answer is straightforward. According to previous textual analysis, the Greek Isaiah appears to have been developed in (at least) two stages - an initial translation by one person and then a subsequent revision by one or more other translators. Papoutsakis proposes then that the original translator translated all three instances of בנות יענה as Σειρνε. The subsequent reviser, not understanding the association derived from Aramaic, thought the original translator had mistranslated בנות יענה as (whatever Greek originally appeared for תנים) and "fixed" the mistake.
The Job passage follows the same pattern (i.e. both בנות יענה and תנים appear in close association in the passage). Papoutsakis does not explicitly state how this passage came to be, instead simply saying it "agrees with the revised Isaiah passages." I suppose there are two options - 1) it followed the same translator, reviser pattern, or 2) the translator of Job used the revised Isaiah as a guide since he wasn't quite sure what the original Hebrew meant. (Animal names are notoriously difficult to translate accurately.)
To support his hypothesis, Papoutsakis notes that the context of Isaiah 34:13–14 provides evidence that the בנות יענה -> Σειρνε is the original reading in the Greek Isaiah. Specifically, this passage also contains the hapax legomenon לילית. This word appears to be some sort of nocturnal animal, but the Septuagint translates it as ονοκενταυροι, another mythical creature. Transliterated, the Hebrew word is "Lilith".
The Syriac Apocalypse of Baruch (II Baruch) provides strong evidence for Papoutsakis' reconstruction:
I will summon the Sirens from the sea;
And you, Liliths, come from the desert,
And (you), demons and jackals, from the forests:
The language here closely mirrors Isaiah 34:13-14, suggesting a literary dependence (or inspiration), but note that Sirens is standing in for ostriches, not jackals. Apparently, the author of Apocalypse of Baruch knew Isaiah 34 with Sirens and jackals, not ostriches and jackals. (Note: Papoutsakis says literary dependence on the Septuagint is not necessary for his point - this could derive from the Hebrew/Aramaic directly under the same understanding of ostrich=Naamah=Siren word play, thus providing independent evidence of the tradition. Either way, II Baruch provides strong evidence for his reconstruction of Isaiah 34:13-14.)
Furthermore, something very interesting happens in later writing: Naamah becomes associated with a new figure named Lilith. For example, in the Zohar:
There was a certain male who came into the world from the spirit of the side of
Cain, and they called him Tubal-cain. And a certain female emerged with him,
and human beings go astray after her, and she was called Naamah... And Naamah makes a roaring noise and cleaves to her side, and she still survives. And her dwelling is among the breakers of the great sea, and she goes out and makes sport with men, warming herself on them in dreams with human desire, and cleaving to them... The sons that she bears from mortal men present themselves to the
females among mankind and they become pregnant by them and bear spirits.
And they all go to ancient Lilith and she rears them.
In addition to the association between Lilith and Naamah, note that the description of Naamah sounds a lot like a Siren.
Prior to the translation of the Septuagint, a tradition had developed that associated Naamah with roughly the same attributes ascribed to Sirens - a sinister nature, musical/singing skill, and physical beauty. Contemporary with the translation, there was an Aramaic association with ostriches and Naamah based on paronomasia. The original Septuagint translators read the passages about ostriches as having this same double meaning - both the physical bird and the attributes of Naamah. They made a choice to capture the alluring musician meaning in their translation rather than the animal meaning, causing Siren to enter the text. The exceptions to this hypothesis (i.e. translations of "jackal") are explained by a reviser not understanding the word play and "fixing" the apparent error.