I don't think there is a perfectly clear answer here.
It certainly has a general application to all men, calling to mind the Gospel teaching Judge not, that ye be not judged. For what judgment ye judge, ye shall be judged (Matthew 7:1-2; see also Luke 6:37).
But in the context of the verse it could refer to any or all of the pagans, Jews, rulers, or subject. (Orthodox Archbishop) Dmitry Royster comments on this verse:
The question that naturally occurs to us is, "to whom is this admonition addressed? O man?" To mankind in general? To pagans or to the Jews? According to St. John Chrysostom, "These things he says, with an aim at the rulers, inasmuch as that city [i.e. Rome] had the rule of the world put into its hands" (On Romans, Homily V). Then he adds, "He is not addressing himself to the rulers only, but to private individuals and subjects also ... they judge those that offend, in conversations and public meetings, and by the vote of their conscience" (ibid., on v.2). Whether their sin of judgment remains unexpressed or not, they will be judged for it.
There is no doubt that it is also to the Jews that this is addressed, because so many of them thought that God's giving them the Law made them superior to and judges of all. And, to return to the sense of 1:18, judgment of others and condemnation are part of the unrighteousness against which God's wrath is revealed.
St. Paul's Epistle to the Romans: A Pastoral Commentary