A fair assumption
Thus far in Genesis 1-5, elohim has consistently been used to refer to the singular creator 'God'.
In Genesis 1, this is evident by the use of verbs in the singular form, as discussed in this answer to a previous question. In Genesis 2-4, the term elohim is found in tandem with Yahweh, specifying a single deity, 'Yahweh God'. Genesis 5, where elohim is used alone (that is, without the tandem 'Yahweh'), picks up themes and ideas from Genesis 1; it is evident that elohim in this chapter is again the singular creator God.
When we arrive at Genesis 6, I think it is fair for readers to initially assume that ha elohim here is referring to the same singular creator 'God'. The redactor unifying the alternate source traditions in Genesis 1, 2-4, and 5 would certainly have intended as much.
As safe as this assumption would be to make, we want actual evidence, if there is any available.
Intertextuality of Genesis 6.1-4
Genesis 6.1-4 is identified as part of the Yahwist narrative that begins in Genesis 2-4.
In Genesis 2-4, the narrator regularly calls the lone creator deity by the name Yahweh elohim ('Yahweh God'). However, in 3.1-5 the creator is called elohim, without Yahweh. This happens again in Genesis 4.25.
This gives us explicit precedent that even in the Yahwist source, Yahweh elohim may be identified as simply elohim.
'Sons of God' elsewhere
The phrase 'sons of God' appears at other points in the Hebrew bible, with slight variations on the wording.
The variants bene elohim (Deuteronoy 32.8; Job 38.7) and bene elim (Psalms 29.1; 89.7) both translate to 'sons of God'. The variant bene elyon (Psalm 82.6) means 'sons of the Most High'.
The specific phrasing of Genesis 6.1-4, bene ha elohim, is found in Job 1.6 and 2.1. In Job 1-2, we are introduced to the individual supreme God by the name elohim, and he is consistently attached to singular verb forms. In this context, when 'sons of God' is used, elohim is best understood as referring to that individual supreme God who is seen presiding over the sons of God.
Thus the only identical verbal parallel to Genesis 6.1-4 has elohim as a singular deity, not a pantheon.
Cultural backdrop
In Canaanite literature we find use of the phrase bn ʾil. Here, ʾil is the individual deity El, who at one point was identified by the Canaanites as the supreme creator deity, as Yahweh is presented in these early chapters of Genesis.
The story found in Genesis 6.1-4 has no known parallels in Canaanite literature, but the essential equivalency between Ugaritic bn ʾil and Hebrew bene ha elohim (and variants) suggests the ancient Israelites took up the phrase and used it in a similar way.
Hence, the argument of Ronald Hendel, 'The Nephilim Were on the Earth', The Fall of the Angels (Auffarth, Stuckenbruck, eds.), 18:
The implication of divine parentage contained in the phrase "Sons of God" is best regarded as an archaism, derived from the older Canaanite phrase "Sons/Children of El" (bn ʾil), and the concept that the active gods are all, or nearly all, the offspring of the high god El (lit. "God").
In other words, the authors of the Yahwist source, received the phrase, originally about the individual god ʾil, and applied it to their own individual god Yahweh, which they have already identified as elohim throughout the preceding stories found in Genesis 2-4.
Conclusion
The term elohim in this instance almost certainly is meant to be understood in as a singular, i.e. 'God'. The swath of evidence available heavily points in that direction.
