Eli's Failure
Somewhat regardless of whether the word כָּהָה (kāhâ) should mean "rebuke" or "restrain," at the point which the sons refused to obey their father Eli (1 Sam 2:25), Eli should have had his sons killed on the basis of two, and possibly three points of the Law (quotes from NASB):
Dishonoring God's Law—Lev 3 and Lev 4 with Num 15:30-311
30 But the person who does anything defiantly, whether he is native or an alien, that one is blaspheming the Lord; and that person shall be cut off from among his people. 31 Because he has despised the word of the Lord and has broken His commandment, that person shall be completely cut off; his guilt will be on him.’
By not performing their priestly duties properly regarding sacrifices (1 Sam 2:12-17), they were no longer "unintentionally" sinning, but "defiantly" sinning, and should have been "cut off."
Adultery (probably)—Lev 18:20 and Deut 22:22 (cf. Exo 20:14)
Lev 18:20 You shall not have intercourse with your neighbor’s wife, to be
defiled with her.
Deut 22:22 If a man is found lying with a married woman, then both of them shall die, the man who lay with the woman, and the woman; thus you shall purge the evil from Israel.
Eli's sons, according to 1 Sam 2:22, "lay with the women who served at the doorway of the tent of meeting." These relations most likely consisted of some adulterous ones, or otherwise improper for priests to partake in.2
Law of the Rebellious Son—Deut 21:18-21
18 If any man has a stubborn and rebellious son who will not obey his
father or his mother, and when they chastise him, he will not even
listen to them, 19 then his father and mother shall seize him, and
bring him out to the elders of his city at the gateway of his
hometown. 20 They shall say to the elders of his city, ‘This son of
ours is stubborn and rebellious, he will not obey us, he is a glutton
and a drunkard.’ 21 Then all the men of his city shall stone him to
death; so you shall remove the evil from your midst, and all Israel
will hear of it and fear.
After being "rebuked" by their father Eli for sins they already should have been put to death for, yet still not listening to their father, Eli himself should have initiated their stoning.
So Eli, of all people, should have been leading the charge on multiple levels to have his sons put to death, and effectively removed from their office. But 1 Sam 2:29 states the issue with Eli (emphasis added):
Why do you kick at My sacrifice and at My offering which I have
commanded in My dwelling, and honor your sons above Me, by making
yourselves fat with the choicest of every offering of My people
Israel?
So if Eli would not fix the issue, God would, and judge Eli and his whole house in the process.
Possible Meaning of כָּהָה (kāhâ) in 1 Sam 3:13?
As a side point, the word כָּהָה is elsewhere given the idea of "grow dim" or "faint" both in the Qal and Piel (BDB), and only in 1 Sam 3:13 is the idea of "rebuke" or "restrain" brought up.
It seems to me that it would be best to keep that idea in 1 Sam 3:13, so a modified NASB translation of the verse be:
For I have told him that I am about to judge his house forever for the
iniquity which he knew, because his sons brought a curse on themselves
and he did not grow faint [or dim] in them.
This language could then be a reference back to the 1 Sam 2:29 accusation God makes about Eli honoring his sons above God. They should not have had such a bright spot in his eyes that they eclipsed God's law. The language would be more poetic, along the lines of dimness expressed by Job 17:7:
My eye has also grown dim [Qal of כָּהָה] because of grief, And all my members are as a shadow.
Eli should have been experiencing an intensified grief, and thus an intensified faintness or dimness (Piel of כָּהָה) toward his sons for the reproach they were making upon God's name and ways, but he was not experiencing that, according to this reading of 1 Sam 3:13.
Such a reading (a) makes sense in context, (b) fits the usage of the term otherwise, and (c) resolves the apparent contradiction that initiated your question. A win on all accounts.
1 I view the Pentateuch as a unified unit, chiefly composed by Moses and completed prior to Israel coming into Canaan, and so the whole Law part of the background to Eli and his sons.
2 All the texts from the Pentateuch that explicitly charge men with committing adultery are when they have had intercourse with a married woman. It is challenging from just reading the Pentateuch to find where intercourse with an unmarried woman constituted adultery, even if the man was married. Phinehas was for sure married (1 Sam 4:19), so if adultery was considered simply on the basis of the man being married, then he was for sure guilty, but if not, then it would require that the women he was laying with be married to qualify.
However, priests were also held to a higher standard. Lev 21 states that they could "not take a woman who is profaned by harlotry, nor shall they take a woman divorced from her husband" (v.7). They were only to marry a virgin (v.13-14). So what Eli's sons were doing in 1 Sam 2:22 was wrong, and Eli considered what they did reprehensible (since 1 Sam 2:23 is the beginning of the "rebuke," which follows upon the statement of their sexual behavior).