The confusing passage here seems to be Matthew's account, which we will come to in due course. The other accounts, including the apocryphal Gospel of Peter give rather clear indications of timing, so we begin by examining them:
When the Sabbath was over, Mary Magdalene, Mary the mother of James, and Salome bought spices so that they might go to anoint Jesus’ body. Very early on the first day of the week, just after sunrise, they were on their way to the tomb and they asked each other, “Who will roll the stone away from the entrance of the tomb?”
But when they looked up, they saw that the stone, which was very large, had been rolled away. As they entered the tomb, they saw a young man dressed in a white robe sitting on the right side, and they were alarmed.
“Don’t be alarmed,” he said. “You are looking for Jesus the Nazarene, who was crucified. He has risen! He is not here. See the place where they laid him.'
Mark 16:2-6 NIV (emphasis mine)
Mark gives us two chronological markers, which I have emphasized above. The first is "when the Sabbath was over", but then he introduces it again with, "Very early on the first day of the week, just after sunrise." While the timing of the resurrection itself is not noted, the implication from Mark's text seems to be that Jesus rose at dawn.
On the first day of the week, very early in the morning, the women took the spices they had prepared and went to the tomb. They found the stone rolled away from the tomb, but when they entered, they did not find the body of the Lord Jesus. While they were wondering about this, suddenly two men in clothes that gleamed like lightning stood beside them. In their fright the women bowed down with their faces to the ground, but the men said to them, “Why do you look for the living among the dead? He is not here; he has risen!"
Luke 24:1-6a NIV (emphasis mine)
Luke's account likewise places the arrival of the women at the tomb shortly after dawn on the first day of the week. He, likewise, continues the tradition of noting that it was the first day of the week.
Early on the first day of the week, while it was still dark, Mary Magdalene went to the tomb and saw that the stone had been removed from the entrance. 2 So she came running to Simon Peter and the other disciple, the one Jesus loved, and said, “They have taken the Lord out of the tomb, and we don’t know where they have put him!”
John 20:1-2 NIV (emphasis mine)
Again, it is the first day of the week, and John's account likewise indicates that it is morning; the phrase "while it was still dark" indicates that if it is not dawn already, dawn is close and darkness is about to pass. This is easily reconciled with Mark and Luke simply by noting the theme of light and dark in John's gospel. Likely he emphasizes the darkness to illustrate Mary of Magdalene's transition from "blindness" to "sight" (so Carson).
We can consider as well the relevant passage in the apocryphal Gospel of Peter, which you have alluded to in your question. I believe it is a later gnostic work of the mid-to-late second century, and hence derivative of rather than formative to the four canonical gospels, but it still might afford some insight into how early Christians conceived of the resurrection for our purposes:
But early when the Sabbath was dawning, a crowd came from Jerusalem and the surrounding area in order that they might see the sealed tomb. But in the night in which the Lord's day dawned, when the soldiers were safeguarding it two by two in every watch, there was a loud voice in heaven; and they saw that the heavens were opened and that two males who had much radiance had come down from there and come near the sepulcher. But that stone which had been thrust against the door, having rolled by itself, went a distance off the side; and the sepulcher opened, and both the young men entered.
And so those soldiers, having seen, awakened the centurion and the elders (for they too were present, safeguarding). And while they were relating what they had seen, again they see three males who have come out from they sepulcher, with the two supporting the other one, and a cross following them, and the head of the two reaching unto heaven, but that of the one being led out by a hand by them going beyond the heavens. And they were hearing a voice from the heavens saying, 'Have you made proclamation to the fallen-asleep?' And an obeisance was heard from the cross, 'Yes.'
The Gospel of Peter translated by Raymond Brown (emphasis mine)
The temporal markers at the beginning are a bit confusing since it introduces a crowd but then seems possibly to back up into the night. Yet, it's clear that several watches have already happened, that people have been sleeping for some time and need to be awoken, and that the resurrection itself is seen as the dawning of a new day. Together what we have seen so far then indicates that the early Christians conceived of the resurrection happening at or near dawn on the first day of the week following Jesus' crucifixion.
So what about Matthew's account? Matthew 28:1 begins:
Ὀψὲ δὲ σαββάτων, τῇ ἐπιφωσκούσῃ εἰς μίαν σαββάτων, ἦλθεν Μαριὰμ ἡ Μαγδαληνὴ καὶ ἡ ἄλλη Μαρία θεωρῆσαι τὸν τάφον.
Most modern translations have something like:
After the Sabbath, at dawn on the first day of the week, Mary Magdalene and the other Mary went to look at the tomb.
Based on your question, though, it seems you consider the proper English translation to be something like: "Late on the Sabbath..." The grammar is not easy, but there are at least two possible solutions, which would allow Matthew's account to be harmonized with the other accounts. The first is followed by A. H. McNeile: that "Late on the Sabbath" is the correct translation, but that Matthew is using the Roman way of starting/ending days at dawn rather than at sunset. Given the distinctly Jewish character of the rest of Matthew's gospel, though, it is probably better to adopt the view of most modern commentators and BDAG (3), who take ὀψέ as a preposition ("after") rather than an adverb ("late"). The strength of the other traditions placing the resurrection at or near dawn recommends one of these two approaches.
All of this places the bounds on the resurrection as sometime after the Sabbath sundown and before the women arrive early on the first day of the week. While the strong implication of all of these texts is that Jesus rose shortly before the women arrived, further evidence for this can be deduced by the place of "dawn" and its association with resurrection in early Christian thought.
Jesus, "the resurrection and the life", was referred to as the "morning star" (Rev. 22:16). He is said to fulfill Isaiah, who writes that a light has dawned on those dwelling in darkness (Matt. 4:14-16), and he is called the "rising sun" come to us from heaven (Luke 1:78). In other words, Jesus himself was symbolically linked in early Christian thought to the dawning of a new day.
We know as well that the ideas of sleeping and waking were often used by early Christians (and Jesus Christ) of death and resurrection. For instance, in Matthew 9:24, Jesus refers to the dead girl as only sleeping because his plan is to wake her up (i.e. resurrect her). The same is found in John 11, where Jesus tells the disciples "Our friend Lazarus has fallen asleep; but I am going there to wake him up" and when pressed on this he says, "Lazarus is dead" and we of course later see him resurrected. So too the analogy is seen clearly in 1 Thessalonians 4:14 where Paul writes, "For we believe that Jesus died and rose again, and so we believe that God will bring with Jesus those who have fallen asleep in him."
We see in 1 Thessalonians 4 as well the clear link between an eschatological new day and resurrection. So also in Romans 13 we read, "The night is nearly over; the day is almost here." The Romans are therefore to live in light of the resurrection as one living in daylight.
The summation of this points to a deep link in Christian thought between resurrection and the dawn of a new day, which in turn should lead us to believe that the authors of the gospels all intend their readers to grasp that Jesus' resurrection came at the dawn of a new day and a new week (both historically and symbolically).