There are (at least) four issues that must be dealt with in order to sufficiently answer this question, and much ink has been spilled on all of these issues.
Issue 1: Context (and identification) of the scripture being cited
If the author of James is citing another text, then understanding the original context of the quoted passage may shed some light on its use in this epistle. Unfortunately, we'll have no such luck. The specific text cited is unknown to scholars today. There is certainly no shortage of attempted explanations, but ultimately these are all speculative as the source of the quotation remains unknown.
Issue 2: Alternate reading
The second issue is the presence of an alternate reading in the Byzantine manuscript tradition (as well as in a few other manuscripts). There is a concept in textual criticism referred to as itacism (or 'iotacism'), which is the tendency for textual errors to occur during the (especially dictated) transmission of the text where vowel pronunciation "drifts" toward the sound of the iota, whereby two similar sounding words could easily be confused. Metzger explains the two readings and how it affects the translation of this passage:
The two verbal forms, which, because of itacism, were pronounced alike, have slightly different meanings: κατῴκισεν is causative (“the spirit which he [God] has made to dwell in us”), whereas κατῴκησεν is intransitive (“the spirit [or, Spirit] which dwells in us”). On the score of external evidence κατῴκισεν is somewhat better attested ... than κατῴκησεν.... On the score of transcriptional probability, since κατοικίζειν occurs nowhere else in the New Testament, copyists were more likely to replace it with the much more common κατοικεῖν, than vice versa.1
The NET translators made the causation even clearer in their translation (but shifted the subject of ἐπιποθεῖ from "God" to "spirit", as in Metzger):
Or do you think the scripture means nothing when it says, "The spirit that God caused to live within us has an envious yearning"? (emphasis mine)
A translation that follows the majority (Byzantine) text reading of the intransitive (κατῴκησεν) might read as follows:
Or do you think that the Scripture says in vain, "The spirit which dwells in us longs to envy"?
However, the causative (κατῴκισεν) is the preferred reading based on internal and external evidence,2 in which case "God" is the subject of κατῴκισεν.
Issue 3: Who (or what) is the "spirit"?
"Spirit" (πνεῦμα) can refer either to the human spirit or the Holy Spirit. The ESV translation cited in the question (perhaps wisely) preserves this ambiguity. The reality is that this is ambiguous. These are all possible readings of this clause:
- "the Spirit who lives in us yearns jealously"
- "he who caused the Spirit to live in us yearns jealously"
- "the spirit he caused to live in us yearns jealously"
- "he jealously yearns for the Spirit he made to live in us"
The NET translators address their translation choice (which corresponds to option #3 in the above list) in their commentary:
Interpreters debate the referent of the word “spirit” in this verse:
- The translation takes “spirit” to be the lustful capacity within people that produces a divided mind (1:8, 14) and inward conflicts regarding God (4:1–4). God has allowed it to be in man since the fall, and he provides his grace (v. 6) and the new birth through the gospel message (1:18–25) to counteract its evil effects.
- On the other hand the word “spirit” may be taken positively as the Holy Spirit and the sense would be, “God yearns jealously for the Spirit he caused to live within us.” But the word for “envious” or “jealous” is generally negative in biblical usage and the context before and after seems to favor the negative interpretation.3
The NET translators appear to basing their argument for the human spirit as the intended meaning of πνεῦμα and subject of ἐπιποθεῖ largely on the fact that "φθόνος and its cognates are never used positively in the NT, rarely so in secular Greek ... and equally rarely in patristic Greek...."4 However, for this same reason, they reject "God" as the subject of the clause to avoid ascribing negative "jealous longing" to him, seeing the clause with πρὸς φθόνον as unbefitting of God.
Issue 4: Who is yearning jealously?
While God is clearly the subject of κατῴκισεν, is he also the subject of the clause with πρὸς φθόνον?
It follows that since God is the subject of both δίδωσιν and λέγει, it is likely that God is also the subject of the main clause. Davids agrees with the NET translators concerning the idea of πνεῦμα being understood as the human spirit, but argues (along with most translators and commentators) that God is the one who yearns/longs enviously/jealously, stating:
Attractive as [the interpretation and consequent translation choice of the NET] is, it has its problems. First, this translation does not fit well in context, for although the structure is parallel to 4:4, this translation ignores 4:4 and jumps back to 4:1–3. The citation, then, does not support the parenesis of 4:4. Second, as 1 Macc. 8:16; 1 Clem. 3:2; 4:7; 5:2; and Test. Sim. 4:5 show, ζῆλος and φθόνος are parallel terms. Since James has already used ζῆλος so negatively, he may deliberately select φθόνος as a synonym not yet used. Third, the alternative interpretation fits well with 4:4, giving a basis for the statement.
Most commentators, then, do see God as the subject.... That God longs for his creation is a theme found in the OT (e.g. God is the subject of ἐπιποθεῖν in Theodotion’s version of Jb. 14:15b; ... cf. Jeremias; [Apocalypse of Moses] 31:2). That God is jealous is also a well-known theme (Ex. 20:5; 34:14; Dt. 4:24; etc.), which could logically be translated by either ζῆλος or φθόνος. That God put the spirit in man is also well established (Gn. 6:3 LXX; 6:17; 7:15; Ps. 104:29–30; Ezekiel 37; Wis. 12:1). What is more, several passages refer to the need to keep this spirit pure.... Thus if the spirit turned toward the world, God’s jealousy would be aroused. Such a threat clearly supports Jas. 4:4.5
Conclusion
In summary:
The passage cited by the author of James is unknown and thus its context cannot be determined to aid in the translation of this text.
There are three potential subjects of the quoted clause:
- God
- The Holy Spirit
- The human spirit
The greater internal and external evidence supports the causative reading (κατῴκισεν), in which case "God" is clearly the subject of κατῴκισεν.
The context strongly indicates that the human spirit is the intended meaning of πνεῦμα.
Most translators and commentators see God as the subject of the clause, which has well-established precedent in biblical literature.
In conclusion, I believe the NRSV is an excellent translation of this verse because it clarifies 'God' as the subject of the clause yet retains the ambiguity of 'spirit' (although I personally think 'human spirit' is the most likely meaning):
Or do you suppose that it is for nothing that the scripture says, "God yearns jealously for the spirit that he has made to dwell in us"?
Footnotes
1 Bruce Manning Metzger, United Bible Societies, A Textual Commentary on the Greek New Testament, Second Edition, A Companion Volume to the United Bible Societies’ Greek New Testament (4th Rev. Ed.) (London; New York: United Bible Societies, 1994), 612.
2 I didn't dive very deep into the evidence. There is a certainly an argument to be made for the majority text reading as well, based on internal and external evidence. However, my time is limited and folks like Metzger and the ESV, NRSV, and NET translation committees are much smarter than me and have sided with the causative reading for good reasons.
3 Biblical Studies Press, The NET Bible First Edition Notes (Biblical Studies Press, 2006), Jas 4:5.
4 Peter H. Davids, The Epistle of James: A Commentary on the Greek Text, New International Greek Testament Commentary (Grand Rapids, MI: Eerdmans, 1982), 163.
5 Ibid., 164.