Bottom line: It is ambiguous. There is no certain answer here. There is a long history of various interpretations. The text itself does not provide a "clear" answer no matter how you want to parse it.
If you'd like a really great detailed read "The Beloved Disciple" by James Charlesworth from 1995. It's probably the best analysis of the whole thing. He spends about 50 pages carefully digging into the history of this.
Great minds from Augustine to Luther to Wesley read it as "believing what Mary told them, that the tomb was empty." NOT resurrection belief. They see the force of verse 20:9 necessarily demanding that they don't know.
Bultmann, the 20th century 800lb gorilla of Johannine studies, recognized the force of 20:9, and so he decided it was a late gloss, and preferred to cut it out such that 20:8 was resurrection belief. But, if true, what that means is that a very early scribe that is universally attested added a comment to clarify his interpretation of the ambiguous verse 20:8 such that it would describe merely believing that the tomb was empty.
The Codex Bezae contains a variation at exactly this verse where it says "and saw and did not believe." That's clearly a minor attestation, but also someone trying to clarify in favor of empty tomb belief in the 5th century.
A major reason for forcing the "resurrection belief" conclusion is that otherwise, the beloved disciple leaves the narrative without professing belief. That's problematic if John (son of Zebedee, or anyone else) is the witness of the gospel. It seems like the disciple should be the first to believe or that his belief should be declared somewhere in his gospel.
Charlesworth solves this problem by identifying Thomas as the Beloved Disciple. In this way, Thomas (as the BD) provides a validation of the woman's testimony by two male voices in 20:8-9, and then at the end of the chapter, as Thomas explicitly, declares his belief and makes the most stunning identification of the entire gospel. And by being shown as "hard to believe the disciples claims" (e.g. doubting thomas), he is then shown to be an incredulous witness who is hard to convince. Someone reliable.
Some try to connect John 20:29, "Blessed are those who have not seen and yet have come to believe." to 20:8. There is a sense that the BD believed without actually seeing Jesus. I don't buy this. Verse 20:8 literally says "and he SAW and BELIEVED..." his belief was based on seeing evidence. It couldn't be further from the claim in v29 which I believe is targeted at later generations of community members who simply could not be part of the events described in the gospel. All of the disciples "saw and believed." It seems to me that v29 is a fourth wall break kind of like Puck at the end of Midsummer Night's Dream. First Jesus talks to Thomas "you have believed because you have seen" (that is, he is an eye-witness and Jesus' words validate it).
After that, Jesus looks off into the distance, speaking to future readers "happy/blessed are those who believe without seeing these things." Then the narrator DIRECTLY addresses the reader in verse 30-31 using the second person.
Belief, in John, is a process. The verb "to believe" is used 96 times and the noun, so often used in Paul, is completely absent. If the gospel is a witness account designed to help you believe, then the idea that the story tells a careful progressively witnessed narrative that concludes with Thomas' declaration "my lord and my god," after being shown to be hard to convince, is pretty sound.
But again, all of this, as with most biblical studies, is speculative. There is no dative phrase following the word for belief. The contents of verse 9, that they didn't know the scriptures, is an extreme challenge for any claim to resurrection belief (though people wiggle around it).
A final note that is convincing to me:
If the Beloved Disciple had believed in the resurrection of Christ, why did he and Peter merely return to their homes? How could they leave Mary weeping? Why not console her with the news?! That doesn't sound like a compassionate action of a fellow disciple. Why didn't they run and tell everyone?
Also, Charlesworth makes the fascinating argument that the reason that Thomas only appears "after 8 days" (20:26), is because, as the BD, he entered the tomb and became ritually impure according to Numbers 19:16 for seven days after the first. After 8 days, corresponds to exactly when Thomas would be purified and able to return to the group. But only if he acted as the Beloved disciple in the beginning of chapter 20. If that is your theory, then 20:8 must be "empty tomb" belief validating each phase of the resurrection.
First he validates the crucifixion and the death with the spear. )(Joseph and Nicodemus validate the burial). Second he validates the empty tomb. Finally, he validates the resurrection. If he had validated the burial, then he would have been unclean for the visit to the tomb with Mary on the third day.
Ultimately, however, what the BD believed will be up to you, the reader, and the interpretive "gap filling" theory that you bring to the text. Plenty of people have seen either full resurrection belief, the weak seed of resurrection belief (e.g. John Calvin), or belief in what Mary said.