God gave circumcision as an external sign in response to Abraham's faith. Nigel J points out to a very interesting verse highlighting that
A servant who is born in your house or who is bought with your money shall surely be circumcised; thus shall My covenant be in your flesh for an everlasting covenant.
Genesis 17:13 NASB https://bible.com/bible/100/gen.17.13.NASB
But at some point in history, circumcision became more important than the decision to believe and follow God.
Circumcise yourselves to the Lord And remove the foreskins of your heart, Men of Judah and inhabitants of Jerusalem, Or else My wrath will go forth like fire And burn with none to quench it, Because of the evil of your deeds.”
Jeremiah 4:4 NASB https://bible.com/bible/100/jer.4.4.NASB
What matters to God is doing the things for the right motives.
Moreover the Lord your God will circumcise your heart and the heart of your descendants, to love the Lord your God with all your heart and with all your soul, so that you may live.
Deuteronomy 30:6 NASB https://www.bible.com/bible/100/DEU.30.6.NASB
That's what leads Paul to be consistent in his speech, who also does so in Romans 2:29 while quoting from Jeremiah 4:4 and Deuteronomy 30:6 when speaking of circumcision of the heart. He wanted to give emphasis to genuine loyalty to God. That's what God's good, pleasant and perfect will is.
If by now you're still considering if God cares about our heart, then why circumcision... go ahead and read Romans 3 where that's addressed.
Are you so foolish? Having begun by the Spirit, are you now being perfected by the flesh?
Galatians 3:3 NASB https://bible.com/bible/100/gal.3.3.NASB