The question has three levels, on the first level the answer is NO, on the other two levels the answer is YES.
Sexual relationship which is necessarily connected with a bodily desire is not sinful in the context of marriage, thus neither does it fall under the censure of Jesus, unless childbearing is also sin, which it cannot be. Thus, therefore the desire of another woman than one's own is at stake. "Marriage is holy and the marriage bed undefiled" (Hebrews 13:4).
Sexual relationship even in the context of marriage should not go to excesses, for even the marriage according to the NT is a great mystery that, among other things, also accounts to reduction of excessive sexual passions, "burning", in humans (1 Cor. 7:9). Now, if someone enters marriage, on the contrary, for giving a free vent to his excessive sexual phantasies and proclivities, moreover coercing his own spouse to the same, then such a marriage will cease to be soul-improving and God-pleasing, but will lead to a moral depravity of both spouses. In this sense, excessive sexual passions and desires should be checked also in the context of marriage, so that Jesus' words then apply also to the case, when one looks at one's own wife with a selfish lustful desire, for as a famous saying goes "lust kills love".
Jesus' words can address, among the others, also those who wish to be perfect in a more short-cut and intensive fashion, the voluntary eunuchs, who can contain His words about remaining celibate (Matthew 19:12), who dare to expose their hearts and lives to such an excess of divine grace, as to overcome even the sexual urges and feel happy even without sexual relationships and long more towards God than towards even the most romantic love-affair with a woman, or, in case of a woman, with a man. Such small amount of Christians have their souls already in an angelic state, the state of Jesus' soul, which state all Christians should aspire at, even the married ones, for in the Heavenly Kingdom there will be neither marriage nor sex (Matthew 22:30). Now, if such a voluntary eunuch does not protect his elevated calling and will miss his previous stage and succumb to his desire for women, this will be a sin, for he will exchange a better for the worse, which means that he will no more bear the intensity of grace that enabled him to abstain from women without any infringement upon his happiness in God.