Paul is arguing that "fleshly" descent from Abraham does not in itself make one of Israel. Not only, however, does this apply to Christians, but even to the Jews themselves.
Normally we think of similar arguments he makes in the context of Christians being able to call themselves of Israel even if they are not genetically descended from Abraham. For example:
Galatians 3:26, 29
For ye are all the children of God by faith in Christ Jesus. And if ye be Christ’s, then are ye Abraham’s seed, and heirs according to the promise.
But here, in Romans 9:6ff, Paul is explaining that this even applies to the Jews themselves. Not all children of Abraham's flesh can be called Jews: the children of the bondwoman and Keturah, for example, are not considered to be part of Israel (Genesis 16:15; 25:1-4). These are not the children of God, writes Paul; it is, rather, the children of the promise (i.e. the promise that in Isaac shall thy seed be called) that are counted for the seed.
A modern commentary on this particular passage in Romans explains:
Concerned about what may be inferred from his lament [v.1-5], his anguish over Israel's rejection of the promised Savior, the Apostle hastens to declare that there has been no failure; God's promises have indeed been effective and do remain: this word has not failed. Some Jews have believed, and what is more, some Gentiles, too. These believers constitute the real Israel. Not all of those who are biological descendants of Abraham deserve the name - only those who have put their faith in Christ and are thus "a new creature", "the Israel of God" (Galatians 6:16)
Not all the fleshly sons of Abraham are his children, "his seed"; take, for example, the children of the bondwoman and Keturah, whom he had married after Sarah's death (Genesis 16:15; 25:1-4). But, as the Apostle reminds us, it was "in Isaac" that Abraham's "seed should be called" (Genesis 21:12; Hebrews 11:18). This does not mean, however, that fleshly descent from Isaac is the guarantee of being "counted for the seed" [cf. Galatians 3:26,29].
This new understanding of "the seed" had been foretold by the Psalmist, when he put these words in the mouth of the Man, the Anointed or Christ: "But I have been appointed king by Him on Zion His holy mountain ... The Lord said unto me, Thou art my Son, this day I have begotten thee. Ask of me, and I will give thee the nations [LXX: τὰ ἔθνη, "the Gentiles"] for thine inheritance" [Psalm 2:6-8].*
* Dmitry Royster, St. Paul's Epistle to the Romans (St. Vladimir's Seminary Press, 2008), pp.234-235.