The Hebrew word is כִּלְיָה (kilyah) mean "kidney". The Septuagint is also a witness for this, where לבבות וכליות was translated νεφροί καὶ καρδίαι (nephroi kai kardiae) - lit. "kidneys and hearts". In ancient Greek, the kidneys were believed to be the seat of the emotions and the source of courage, hence they were sometimes referred to metaphorically as the "seat of emotions". Something similar was true in Hebrew.
Rein is an archaic English word for kidney, derived from the Norman French rène and ultimately from the Latin ren. At the time the King James Bible was translated, the meaning was still current but it has since fallen into disuse.
"Hearts and kidneys" seems strange in modern contexts, so modern versions like the ESV have reworded to something like "hearts and minds".
The same Hebrew phrase לֵֽבָבֹ֥ות וּכְלָיֹֽות (levavot u'klayot) - hearts and reins - appears in two other places (albeit with hearts and reins reversed):
Examine me, O LORD, and prove me; Try my reins and my heart [כְּלָיוֹת וּלְבָבוֹת] (Psalm 26:2).
But, O LORD of hosts, that judgest righteously, that triest the reins and the heart [כְּלָיוֹת וּלְבָבוֹת], let me see thy vengeance on
them: for unto thee have I revealed my cause (Jeremiah 11:20).