At this point Moshe is aware of only one book (sefer) that God has written -- the law that he had taken down the mountain only to smash in reaction to the golden calf. While the tablets, the physical encoding of that book, were destroyed, that doesn't mean that the text was. While we don't know what was going through Moshe's head at that time, it is not unreasonable for him to think that the perfect God who wrote the law once can produce another copy of the "book He wrote". It's not a new text, only a new copy. When he tells God to erase him, he is likely saying "leave me out of your story next time; don't record me for posterity". Rashi on Shemot 32:32 says something similar:
from Your book: From the entire Torah, so that they will not say about me that I was unworthy to beg mercy for them [the Israelites].
So the book is the book of the law, the torah. (Yes, this means accepting that the tablets contained all, or at least a substantial portion of, the torah that we have today. While this raises questions about Moshe's advance knowledge of events in the desert, it also answers how we have, for example, the early part of Genesis.)
Whom will God blot out? This is far less clear.
First, note this in v34:
...But on the day I make an accounting [of sins upon them], I will bring their sin to account against them."
So there is blotting out, and there is making an accounting. Are they related? God certainly seems to make an accounting; for example, the first thing God does after answering Moshe is to send a plague:
35 Then the Lord struck the people with a plague, because they had made the calf that Aaron had made.
(Or, in other translations, God "smote" them.)
This plague obviously doesn't wipe out everybody, and probably doesn't even wipe out everybody who was involved in the sin of the golden calf. We know this because the people as a whole survived; some individuals were killed, but not the "corporate body" of Israel.
We know that God is endlessly patient, slow to anger, forgiving iniquity and rebellion and sin. While all the people may have deserved to be blotted out, it was mitigated by the beginning of the passage quoted in the question:
But now, if You will, forgive their sin --
God did that. People were still held accountable, but the nation survived and the book (torah) records these events, so it seems that in the end God did not blot anybody out from His book.
But what about God's statement that He will blot out sinners from His book? It appears that, despite what God said, despite this coming in response to Moshe's "if...", God nonetheless exercised that divine patience and didn't blot everybody out -- because if He had, the nation would not have survived to enter the land of Israel 40 years after Egypt.