At the end of Chapter 4 of Genesis there is a statement in verse 26, "It was then that men began to invoke the Lord by name." Other translations have "call on the name" or "worship the LORD by name". Before this it seems like Adam and Abel worshiped God. And Cain certainly spoke to God.
In any event, this statement seems to indicate that mankind in some way changed how they related to God at that point. But how? What does it mean to "call on the name of the Lord"?
[Note: per Jon Ericson's advice and an edit to the question on which this was an original answer, this answer has been moved, with minor edits, from here, due to edits made to that question to keep it from being a duplicate of this one.]
Calling on the name of Yahweh means that the Sethites began to engage in the public, communal worship of God.
I originally heard of this in a sermon on Genesis 5 (not his sermon on 4:17-26) by Pastor David Reese. (The whole sermon is good, but jump to about 11 minutes in if you want to get right to the relevant part.) Pastor Reese points out (no doubt following Calvin and Vos) that the phrase is a synecdoche for the entirety of the worship of God:
This phrase...is almost certainly referring to the public, corporate word of God, in that it is usually combined...with other acts of public worship in the Bible.
He then cites the following passages. Remember in the following passage that he was travelling with Lot and a very large household (several hundred):
From there [Abraham] went on toward the hills east of Bethel and pitched his tent, with Bethel on the west and Ai on the east. There he built an altar to the Yahweh and called on the name of the Yahweh. —Genesis 12:8
Notice in the following passage the conjunction of "call on the name" and "worshippers":
Then will I purify the lips of the peoples,
that all of them may call on the name of the Lord
and serve him shoulder to shoulder.
From beyond the rivers of Cush
my worshippers, my scattered people,
will bring me offerings. —Zephaniah 3:9-10
And similarly:
I will lift up the cup of salvation
and call on the name of the Lord.
I will fulfill my vows to the Lord
in the presence of all his people. —Psalm 116:13-14
Many other psalms use the phrase in the same manner.
Calvin seems to be off the same opinion. If I understand him rightly, he is saying that in the third generation there were enough worshippers of the true God that public assemblies began to make sense:
Adam and Eve, with a few other of their children, were themselves true worshippers of God... We may readily conclude that Seth was an upright and faithful servant of God. And after he begat a son, like himself, and had a rightly constituted family, the face of the Church began distinctly to appear, and that worship of God was set up which might continue to posterity.
This is also explained by J.G. Vos, in his Genesis commentary (109):
This does not mean that faith in Jehovah, the covenant God of redemptive grace, first began in the time of Enos. It only means that formal public worship of Jehovah began at that time. The religion of faith in Jehovah began with Adam and Eve. By the third generation, about the time of the birth of Enos, regular public assemblies for the worship of Jehovah had originated.
And
We may venture the opinion that this "calling upon the name of the Lord" took place on the weekly Sabbath, and that it included prayer and offering of sacrifices. But beyond this we dare not speculate.
Jon Ericson and Amichai have brought up some interesting points in their answer, but I must disagree with their conclusion that men first began to call him Yahweh during this time. Parallelism is important to recognize in exegesis, particularly of Hebrew texts, and doubtless polytheism was beginning to about at this time. (I think there is an argument to be made that God did not reveal himself as Yahweh until the time of Moses, but that is beyond the scope of this answer.)
Though there is a sort of parallelism between "called his name" versus "called upon his name," the discontinuity is significant. I checked in my Hebrew Bible to make sure this was not simply a translational issue, and it is not. The verb is in a different from in the two cases (though, yes, it is the same verb), and in Enosh's case, the word name is preceded by the direct object marker (אֶת־שְׁמוֹ); whereas when Yahweh is mentioned, there is indeed a preposition before the name (בְּשֵׁם). Jon and Amichai have read too much into the apparent parallel. Pastor Reese, in his sermon on 4:17-26, deals with the significance of Enosh, and the stronger (antithetical) parallel is with Cain's son Enoch.
(My bigger objection to their position is that God is never named by men. He names himself to them. That is significant biblical-theologically but exceeds the scope of this post. Also I disagree that calling him Yahweh is a sign of distance; the name is covenantal and as, intimate even while supremely holy.)
Abel, and even Cain in an external manner, had already been a worshipper of the Lord (verses 3-4). Therefore, even if you don't agree with Calvin about Adam and Eve (if you don't, listen to more of Pastor Reese's sermons; the topic is not insignificant), this cannot simply mean that it was the first time since the fall anyone had worshipped the Lord at all.
It cannot mean that men now had to work their way up to God; that has already been debunked by the failure of the fig-leaves, the shedding of animal's blood to provide covering from shame for Adam and Eve, and the radical wickedness when the woman's firstborn turned out to be a seed of Satan and a killer of the seed of the woman rather than a killer of the seed of Satan.
This phrase is best understood to indicate the public, communal worship of God.
First I note that there is a parallel between the first half of the verse:
To Seth also a son was born, and he called his name Enosh.
And the second:
At that time people began to call upon the name of the LORD. (Genesis 4:26 ESV)
The words qara' <07121> "call" and shem <08034> "name" are repeated, which indicated there is a tie between the first event and the second. And the same formula is used for the naming of Seth in Genesis 4:25 (ESV):
And Adam knew his wife again, and she bore a son and called his name Seth, for she said, “God has appointed for me another offspring instead of Abel, for Cain killed him.”
(I don't know enough grammar to know if the variations in the Hebrew characters invalidate this observation or not. Please correct me if my ignorance of the language has led me astray.)
The second thing I notice is that this marks the end of the section of Genesis that starts back in Chapter 2. The next section starts off with a recap of everything from the beginning to the birth of Seth. Genesis 5:1-3 (ESV):
This is the book of the generations of Adam. When God created man, he made him in the likeness of God. Male and female he created them, and he blessed them and named them Man when they were created. When Adam had lived 130 years, he fathered a son in his own likeness, after his image, and named him Seth.
This is followed by the genealogy (which doesn't use the formula) down to Noah's family. Genesis 5:28-29 (ESV):
When Lamech had lived 182 years, he fathered a son and called his name Noah, saying, “Out of the ground that the LORD has cursed, this one shall bring us relief from our work and from the painful toil of our hands.”
Putting the two observations together, I suggest that this is where people first gave God a name. That's unusual since, before then, a person who "parented" (or created in the case of God to Adam and Eve) another provided their offspring's name. Sons don't name their own fathers. (Or at least they don't have that authority.) So this line might indicate that something is upside down in the world. (I'm reminded of the children's book, Wacky Wednesday, where babies push their parents in strollers.)
It ends the section (or book), which started so promising, on a negative note. Given the life of Cain, which ends the section, that seems an appropriate tagline.
A possible reason why they might need to begin naming the LORD at this time is that humanity's relationship with God had grown even more distant. Perhaps, as this answer suggests, they began to have other "gods" and so the LORD became just one of many objects of worship. Or they simply may have stopped talking to the LORD face to face as Adam did, in which case they would need to "get His attention" by calling His name.
At any rate, it seems difficult to justify translating the phrase as "worship" or "invoke" given the repeated use of the same word to mean "call". It seems translators felt the tension of man providing God a name and resolved it prematurely.
This phrase indicates that humanity's relationship with God became more distant at that time, not that they suddenly began worshiping the LORD.
Cassuto in his commentary to Genesis:
Then men began once more to call upon the name of the Lord —an example of paronomasia: above we read, and he called his name Enosh. There is a parallelism of both language and theme here: a human being is called by a name suited to him—Enosh; and God is called by a name befitting Him—Lord [YHWH].
(This is the same parallelism that Jon Ericson pointed out in his answer.)
...The expression to call upon the name of the Lord does not necessarily imply a ritual act. It is a general phrase that conveys only the idea of calling, without indicating either the significance or the purpose of the invocation. These may vary with circumstances and can be determined only by reference to the context. It is possible to say even of God Himself that He calls upon [E.V. proclaims] the name of the Lord (Exod. xxxiii 19; xxxiv 5–6), and it is obvious that such an invocation has not the same character as that uttered by human beings. In the present case, too, the expression is to be understood according to the context. Cain could not teach his offspring to know the Lord and to call upon His name, that is, to address Him by His particular designation and to feel His personal proximity, for he was compelled, as stated, to hide himself constantly from the face of the Lord. The antithesis seen by the Israelites between calling upon the name of the Lord and being estranged from the Lord’s face [i. e. presence] is demonstrated by Isa. lxiv 7 [Hebrew, v. 6]: There is no one that calls upon Thy name . . . for Thou hast hid Thy face from us. As for Adam and Eve, we have already noted in our commentary on the previous verse why they were unable, during their period of mourning, to call upon the name of the Lord, and were forced to restrict themselves to the name ’Elohim. But on the birth of Enosh, when Adam and Eve perceived that not only had a third son been born to them to replace the first two sons whom they had lost, but that, as additional compensation, they had also been vouchsafed a grandson through their third son, forming the beginning of a new generation and bringing incipient hope for the future, they were comforted from their mourning. They again felt, because of the blessing that rested upon their home, the nearness of the Lord, and once more they were able to call upon the name of the Lord in religious joy.
This interpretation is well suited to the wording of the verse. The adverb Then cannot refer, as is usually held, to the lifetime of Enosh, but only to what has been previously mentioned, namely, the time when Enosh was born. With respect to the verb הוּחַל huhal, it should be borne in mind that in classical Hebrew no distinction is made between the initial performance of an action and its reiteration. Just as the verb בָּנָה bana can mean both to build and rebuild, so the verb הֵחֵל hehel can signify to commence and recommence; here it connotes recommence.
Then men began once more to call upon the name of the Lord— a happy ending and a parallel to the close of the next section (vi 8): But Noah found favour in the eyes of the Lord.
And to Seth, to him also there was born a son; and he called his name Enos: then began men (ḥālal) to call upon the name of the LORD.—Genesis 4:26 (KJV)
In the above the phrase "then began men" is translated from one word ḥālal. The Brown-Driver-Briggs Hebrew and English Lexicon:
ḥālal:
pollute, defile, profane
Theological Wordbook of the Old Testament notes:
660 חָלַל (ḥālal) I, wound (fatally), bore through, pierce. Survives in Arabic ḥalla "pierce through." Occurs ninety-six times, including derivatives.
It appears to be saying around this time frame is when men began to profane the name of Yahweh, someone with a background in Hebrew will hopefully be able to bring clarity to this.
Here is the whole verse:
"And to Seth, to him also there was born a son; and he called his name Enos: then began men to call upon the name of the LORD." Genesis 4:26 NKJV
or
"To Seth also a son was born, and he called his name Enosh. At that time men began to call upon the name of the LORD." Genesis 4:26 RSV
The text says "call on the name of the Lord" Lord is usually translated as Yĕhovah. They were calling on Yĕhovah, not some generic god.
Looking at the context, the statement appears just after the announcement of Enoch. It follows that men began to call on God's name due to the faith of Enoch. (Hebrews 11:5)
Ge 4:26 And to Seth, to him also there was born a son; and he called his name Enos: then began men to call upon the name of the LORD.
Ps 116:13 I will take the cup of salvation, and call upon the name of the LORD.
Ps 116:17 I will offer to thee the sacrifice of thanksgiving, and will call upon the name of the LORD.
Zep 3:9 For then will I turn to the people a pure language, that they may all call upon the name of the LORD, to serve him with one consent.
Ro 10:13 For whosoever shall call upon the name of the Lord shall be saved.
The scriptures indicate that calling on the name of the Lord is a good thing. They became closer to God.
