The scriptures offer no specific definition, although we can infer one from Exodus 22:22-24 (NASB) (BHS vv. 21-23):
"You shall not afflict any widow or orphan [יתום, yātôm] . If you afflict him at all, and if he does cry out to me, I will surely hear his cry; and My anger will be kindled, and I will kill you with the sword, and your wives shall become widows and your children fatherless [יתומים, yātômîm]."
Seemingly, the implication is that even while the mother is still alive the child is called a יתום (yātôm), lending support to the "fatherless" translation. This would make sense historically in that the father was the head of the family and the breadwinner. More importantly, the loss of the father had huge legal implications, which is not the case with the mother. The more significant loss was definitely the loss of the father, hence the word orphan might be applicable.
In this context we should look at all possible definitions/translations of the word orphan. Is an orphan a child who has
(1) Lost both parents?
(2) Become fatherless?
(3) Lost a parent (either mother or father)? or
(4) Become motherless?
Based at least on the verses above and the historical importance of the roles in the family, number four is not an option.
While I wouldn't discount (1) entirely, since the verses may imply the mother will die soon after the father, leaving the children orphaned, that reading has little textual basis other than the convenient order. Moreover, the verses don't seem to describe something overly literal. To infer the definition of the term/concept from that context therefore seems unwarranted, though I am still keeping (1) as an option.
The verses from Exodus, then, might imply (2) or (3). Interestingly, the Jewish law as codified by Maimonides applies to both fatherless and motherless orphans (Mishneh Torah, Section Knowledge, Chapter 6, Clause 17), which would lead to a reading aligned with (3).
While there might be room to suggest (2) is more plausible based on the proximity of אלמנה (ʾalmānâ, widow) and יתום (yātôm) in their appearances throughout the Old Testament, the two words can be read as separate and distinct. There is room to suggest, however, that the references are so consistent because what is being depicted is the broken family unit. The loss of the father has created (and maybe defined) these two entities. While this would support (2) strongly, it doesn't discount (3) entirely. It would, however, suggest that (1) is incorrect.
I don't have a definitive answer, although it appears (1) is less likely for the reasons given above. This is an interesting conclusion because the general connotation when using the term "orphan" is one having lost both parents - which would make readings (2) and (3) (which appear to me more likely) somewhat innovative.
EDIT
Lamentations 5:3, which seems to be a slightly more explicit version of the inference in Exodus, gives credence to (2) more than (3) and rules out (1). I argue, however, this is poetic license, and not a formal definition, since the Masoretic text reads either יתומים ואין אב (orphans and fatherless) or יתומים אין אב (orphans, fatherless). If there is a separating "and" it is easier to read the expression not as explanatory but as a somewhat separate clause--provided for emphasis, which sits well with (3). Nevertheless, the translation "fatherless" fits the text most comfortably.