I'm not sure that Paul is referring to Israelites in this verse. If he is referring to gentiles here, the quote from Psalms makes more sense to me. He definitely refers to Jews in vs 19:
Romans 10:19 But I say, surely Israel did not know, did they? First Moses says, "I WILL MAKE YOU JEALOUS BY THAT WHICH IS NOT A NATION, BY A NATION WITHOUT UNDERSTANDING WILL I ANGER YOU." [NAU]
However, in 18 he just says "they" and previously has not specified Jews before hand except to say that there is no distinction in God between them and gentiles. Previously, "they" has been part of "whoever."
Romans 10:12 For there is no distinction between Jew and Greek; for the same Lord is Lord of all, abounding in riches for all who call on Him; 13 for "WHOEVER WILL CALL ON THE NAME OF THE LORD WILL BE SAVED."
14 ¶ How then will they call on Him in whom they have not believed? How will they believe in Him whom they have not heard? And how will they hear without a preacher? 15 How will they preach unless they are sent? Just as it is written, "HOW BEAUTIFUL ARE THE FEET OF THOSE WHO BRING GOOD NEWS OF GOOD THINGS!" 16 However, they did not all heed the good news; for Isaiah says, "LORD, WHO HAS BELIEVED OUR REPORT?" 17 So faith comes from hearing, and hearing by the word of Christ.
18 ¶ But I say, surely they have never heard, have they? Indeed they have; "THEIR VOICE HAS GONE OUT INTO ALL THE EARTH, AND THEIR WORDS TO THE ENDS OF THE WORLD." 19 But I say, surely Israel did not know, did they? First Moses says, "I WILL MAKE YOU JEALOUS BY THAT WHICH IS NOT A NATION, BY A NATION WITHOUT UNDERSTANDING WILL I ANGER YOU." 20 And Isaiah is very bold and says, "I WAS FOUND BY THOSE WHO DID NOT SEEK ME, I BECAME MANIFEST TO THOSE WHO DID NOT ASK FOR ME."
21 ¶ But as for Israel He says, "ALL THE DAY LONG I HAVE STRETCHED OUT MY HANDS TO A DISOBEDIENT AND OBSTINATE PEOPLE." [NASU]
In 20 he refers to gentiles as those "who did not seek [Him]." Then in 21, he contrasts them with Israel. A case could be made that the same kind of contrast is going on in 18 and 19. 18 refers to gentiles and 19 to Jews.
The quote from Psalm reminds me of a rabbinic discussion on how God would judge the gentiles. Surely He would not hold them accountable to the entire law of 613 commands. They concluded that there were Seven Laws of Noah for the gentiles to obey. First and foremost among those laws is "Avoid idolatry." It was believed that no matter what else, creation testified to the oneness of God (like Paul says in Rom 1:20) and that righteous gentiles would understand that.