Hebrews 4 is, of course, the continuation of the argument that started in Chapter 1 about the Supremacy of Christ over all that has come before Him. But those that came before him are also shadows of Him. In chapter 1 & 2 Jesus is manifestly greater than the Angels though he was made lower than them. In chapter 3 He is greater than Moses, the faithful messenger of God who led His People to the point of Entering into the Promised Rest.
Who was faithful to him that appointed him, as also Moses was faithful in all his house. For this man was counted worthy of more glory than Moses, inasmuch as he who hath builded the house hath more honour than the house. For every house is builded by some man; but he that built all things is God. And Moses verily was faithful in all his house, as a servant, for a testimony of those things which were to be spoken after; But Christ as a son over his own house; whose house are we, if we hold fast the confidence and the rejoicing of the hope firm unto the end. (Heb 3:2-6)
But they entered not in because of their unbelief. At this point Paul (who I am convinced is the author), brings in Psalm 95, where the Holy Ghost explains why they did not enter into the rest on that day when Moses brought them close:
Wherefore (as the Holy Ghost saith, To day if ye will hear his voice, Harden not your hearts, as in the provocation, in the day of temptation in the wilderness: When your fathers tempted me, proved me, and saw my works forty years. Wherefore I was grieved with that generation, and said, They do alway err in their heart; and they have not known my ways. So I sware in my wrath, They shall not enter into my rest.) (Heb 3:7-11)
Paul, applies this immediately to his reader:
Take heed, brethren, lest there be in any of you an evil heart of unbelief, in departing from the living God. But exhort one another daily, while it is called To day; lest any of you be hardened through the deceitfulness of sin. For we are made partakers of Christ, if we hold the beginning of our confidence stedfast unto the end; While it is said, To day if ye will hear his voice, harden not your hearts, as in the provocation.(Heb 3:12-15)
Why is it so important to apply it Today while it is still today?
For some, when they had heard, did provoke: howbeit not all that came out of Egypt by Moses. But with whom was he grieved forty years? was it not with them that had sinned, whose carcases fell in the wilderness? And to whom sware he that they should not enter into his rest, but to them that believed not? So we see that they could not enter in because of unbelief. (Heb 3:16-19)
None that did not believe on that day, entered into his rest. And this exhortation is given to support his main premise: the danger of neglecting this most excellent salvation (a warning which he repeats over and over in this epistle). This is then the immediate preceeding context of Chapter 4 in the OP's question, and important to understand the conclusion with which this passage starts:
Let us therefore fear, lest, a promise being left us of entering into his rest, any of you should seem to come short of it. For unto us was the gospel preached, as well as unto them: but the word preached did not profit them, not being mixed with faith in them that heard it. For we which have believed do enter into rest, as he said, As I have sworn in my wrath, if they shall enter into my rest 1: although the works were finished from the foundation of the world. For he spake in a certain place of the seventh day on this wise, And God did rest the seventh day from all his works. (Heb 4:1-4)
In other words, the seventh-day rest of God, is used here as an allegory and shadow of the rest that God's people are called to enter into: a rest that God has promised in his wrath, is conditioned upon faith.
"But wait," says the Hebrew that knows his history, "Afterwards (40 years later) they did all enter into the promised land under Joshua (Jesus)! Did they not all enter into his rest then, what more is there to this rest?"
Notice in support of this objection, the occurrence of Rest in the book of Joshua.
Remember the word which Moses the servant of the LORD commanded you, saying, The LORD your God hath given you rest, and hath given you this land.(Jos 1:13)
Until the LORD have given your brethren rest, as he hath given you, and they also have possessed the land which the LORD your God giveth them: then ye shall return unto the land of your possession, and enjoy it, which Moses the LORD'S servant gave you on this side Jordan toward the sunrising. (Jos 1:15)
And the name of Hebron before was Kirjatharba; which Arba was a great man among the Anakims. And the land had rest from war. (Jos 14:15)
And the LORD gave them rest round about, according to all that he sware unto their fathers: and there stood not a man of all their enemies before them; the LORD delivered all their enemies into their hand. (Jos 21:44)
And now the LORD your God hath given rest unto your brethren, as he promised them: therefore now return ye, and get you unto your tents, and unto the land of your possession, which Moses the servant of the LORD gave you on the other side Jordan. (Jos 22:4)
And it came to pass a long time after that the LORD had given rest unto Israel from all their enemies round about, that Joshua waxed old and stricken in age.(Jos 23:1)
To which our brilliant Paul draws them again to the Psalm which he quoted in the previous passage as the words of the Holy Ghost:
And in this place again, If they shall enter into my rest. Seeing therefore it remaineth that some must enter therein, and they to whom it was first preached entered not in because of unbelief: Again, he limiteth a certain day, saying in David, To day, after so long a time; as it is said, To day if ye will hear his voice, harden not your hearts. For if Jesus (Greek for Joshua) had given them rest, then would he not afterward have spoken of another day. There remaineth therefore a rest to the people of God. (Heb 4:5-9)
He demonstrates again to them the allegory of the promised rest fulfilled in the ultimate Joshua, since if the first Joshua did bring in this rest, why is the Holy Ghost speaking in Psalm 95 through the mouth of David talking of another day: "Today"?
There remaineth therefore a rest to the people of God. For he that is entered into his rest, he also hath ceased from his own works, as God did from his. Let us labour therefore to enter into that rest, lest any man fall after the same example of unbelief. (Heb 4:9-11)
What is the "another day"?
The day in verse 8 is TODAY! This day is the day, if you are hearing God's voice in the offer of his great salvation, to enter into his rest (peace with God) by ceasing from your own work, as He did from his, and believing in Him alone to take you into his promise through the Work which he has done, in Christ the better Moses and the better Joshua.
In answer to the OP's implicit question about the application of this conclusion to seventh-day Sabbatarianism:
This is NOT a passage that supports or reject seventh-day Sabbatarianism for Christians because Paul is not quoting or referencing any of the Sabbath commands or ordinances. He is explaining the shadow of that rest that the People of God receives through faith in entering in through the Promise. Israel entered into a shadow of that rest through Joshua, and still had to obey the Sabbath commands after entering into the promised land - as a matter of fact, that is one of the reasons their rest failed: they failed to uphold the Sabbath ordinances. Also Psalm 95 which Paul relies on for his argument that there still remains another day when the Peopl of God will enter into the Rest of God, is the joyous song of the greatness of God with a stern warning of not hardening your heart to God's call and its sung by Jews on Wednesdays - it has nothing to do with the Seventh-day sabbath.
Paul is writing to Hebrews who had an Old Testament injunction to keep the seventh-day Sabbath forever (Exo 31:13-18). He would NEVER have instructed or even suggested to Jewish Christians that the Law of Moses (which include all of the Sabbath commands and ordinances) should not be obeyed by them (see James' defence of Paul in Acts 21:20-24). The word Sabbath does not even once appear in the whole Book of Hebrews, so to stretch the application of the argument presented here in chapter 4 is not a sound hermeneutic.
1 Neither the Hebrew nor the Greek has a negation in Psalm 95:11, but its rather a conditional statement that goes both ways: "believe NOT and you will NOT enter, Believe and you WILL enter." Most translations include the "NOT" here as they did in Heb 3:11, but the premise has changed, in Heb 3:11 the premise is those that did not believe, so the consequence is "shall not enter", but in Heb 4:3 the premise is "we which have believed" and therefore "do enter into rest, as he said"