There are at least three reasons why the nominative ὁ θεός is understood to be functioning as vocative:
- Lexical analysis
- Grammatical analysis
- Literary structure
The Lexicons state ὁ θεός is being used as vocative in this passage:
❷ Some writings in our lit. use the word θ. w. ref. to Christ (without necessarily equating Christ with the Father, and there in harmony w. the Shema of Israel DT 6:4; cp. Mk 10:18 and 4a below), though the interpretation of some of the pass. is in debate. In Mosaic and Gr-Rom. tradition the fundamental semantic component in the understanding of deity is the factor of performance, namely saviorhood or extraordinary contributions to one's society...Hb 1:8,9 (in a quot. fr. Ps 44:7,8) S. TGlasson, NTS 12, '66, 270-72. Jd 5 P72. But above all Ignatius calls Christ in many pass.: θεός
❸ God in Israelite/Christian monotheistic perspective, God the predom. use, somet. with, somet. without the art.
h. ὁ θ. is used as a vocative Mk 15:34 (Ps 21:2, twice at the beginning of the invocation of a prayer. Ael. Dion. θ, 8; Paus. Attic. θ, 7; 'θεός θεός'); Lk 18:11; Hb 1:8 (Ps 44:7; MHarris, TynBull 36, '85, 129-62); Ps 39:9); AcPl Ha 3, 10; 5, 12;31. S. also 2 and 3c and the beg. of this entry.
As noted, the use of ὁ θεός in Hebrews 1:8-9 is considered vocative. The paper by Murray J. Harris cited in support of this is an extensive gramatical analysis (the full copy which can be read here). He says:
Some scholars are reluctant to express a preference as to
whether ὁ θεός is nominative or vocative in v. 8, declaring
that both interpretations are admissible and make good
sense. But the overwhelming majority of grammarians, commentators, authors of general studies and English translations construe ὁ θεός as a vocative (O God').
Given the affirmation of v. 3 that the Son is the effulgence
of God's glory and the visible expression of his being, it
is difficult to avoid the conclusion that when the author
affirms further that God the Father addresses his Son as θεός at his resurrection he intends to signify that,
equally with the Father, Jesus possesses the divine natures.
In the footnotes Harris identifies which grammarians, commentators, authors of general studies and English translations interpret the phrase as vocative.
In addition to the semantics and grammatical arguments in favor of the vocative, the literary structure into which vv. 8-9 has been placed should be considered. The overall passage (1:5-13) has seven OT quotes prefaced by who it applies to, angels or the Son. Six of the seven apply to the Son and one to angels. Of the six which apply to the Son, two are introduced as contrasting with angels; two with καὶ, and two which identify the Son. The writer used the method of introduction to arrange the seven chiastically:
A 1:5a For to which of the angels did He ever say (angels in contrast)
“You are my Son. Today I have fathered you”? [Ps 2:7]
B 1:3b and again
“I will be a father to Him, and He will be a son to Me”? [2 Samuel 7:14]
C 1:6 And again, when He brings the Firstborn into the world He says
“And let all the angels of God give-worship to Him” [Deut 32:43]
X: 1:7 And with regard to the angels He says [Ps 104:4]
“The One making His angels winds and His ministers a flame of fire”
C' 1:8 But with regard to the Son He says
“Your throne, God, is forever and ever. And the scepter of straightness
is the scepter of Your kingdom. 1:9 You loved righteousness and hated
lawlessness. For this reason God, your God, anointed You with the oil
of gladness beyond Your companions” [Ps 45:6-7]
B' 1:10 And, “You, Lord, laid-the-foundation-of the earth at the beginnings, and the
heavens are works of Your hands. 1:11 They will perish, but You continue. Indeed
they will all become-old like a garment, 1:12 and You will roll them up as-if
a cloak. They will indeed be changed like a garment. But You
are the same, and Your years will not end” [Ps 102:25-27]
A' 1:13 And with regard to which of the angels has He ever said (angels in contrast)
Be sitting on My right side until I put Your enemies as a footstool of Your feet
The structure begins and ends using the same introduction: angels, in contrast to the Son. The center of the chiasm is likewise introduced with angels, this time in the affirmative to demonstrate the Son's superiority.
The chiastic partner to 1:8-9 is verse 6, which says all the angels of God will worship the Son which almost demands the vocative address in verses 8-9. The literary structure allows the reader to understand angelic worship of the Son is at the same time worship of His Father, or to God in all fullness.
An aspect of the question of why the vocative θεέ was not used may be seen by comparing the LXX Psalm 102 to the Masoretic Text:
Here, the LXX does use the vocative κύριε ("Lord"). However, apparently that translator did not think it appropriate to preserve the original יָסַ֑דְתָּ ("you laid the foundation") and replaced "you" with "Lord." So the vocative is a more formal address than the original. In addition, the introductory καὶ which connects the use of the Psalms also indicates the deity of the Son addressed as the "Lord" who creates and recreates.
Normally the vocative is the correct case to identify who is being addressed. However, when addressing God, the LXX consistently uses the nominative ὁ θεὸς. This is similar to "vocalizing" the Divine Name YHVH as "Adonai" and writing "Lord." As cited in Hebrews, the common theme in both Psalm 45 and 102 is the appropriate form of address when man addresses God. In both cases the LXX translation reflects an address which shows greater respect to God than was present in the Hebrew. That is to say, the translator deviated from the correct grammatical form as it is less respectful when used to address God. The writer of the letter preserved both addresses to support his appeal for the reader to consider who Christ is.
1. Fredrick William Danker, A Greek-English Lexicon of the New Testament and Other Early Christian Literature, The University Chicago Press, 2000, p. 450-451 [Also William F. Arndt F. Wilbur Gingrich, A Greek-English Lexicon of the New Testament and Other Early Christian Literature, The University Chicago Press, 1957, p. 357-358]
2. Murray J. Harris, THE TRANSLATION AND SIGNIFICANCE OF ‘O ΘΕΟΣ IN HEBREWS 1:8-9, Tyndale Bulletin 36 (1985), pp. 146-149