The previous verses helps to answer this question:
וְכִֽי־יַכֶּה֩ אִ֨ישׁ אֶת־עַבְדֹּ֜ו אֹ֤ו אֶת־אֲמָתֹו֙ בַּשֵּׁ֔בֶט וּמֵ֖ת תַּ֣חַת יָדֹ֑ו נָקֹ֖ם יִנָּקֵֽם
"And when a man strikes his servant or his maidservant and he dies under his hand, vengeance shall surely be taken" (Ex. 21:20).
The term "dies under his hand" seems to indicate perishing directly as a result of a beating, as opposed to one who can stand up after one or two days (recovers). I therefore must disagree with the other answers that attempt to interpret this verse as saying, "if he doesn't die after one or two days, the master is innocent." I believe it is saying, if he can stand on the first day or second day after the beating, no vengeance shall be taken. However, if the beating results in a protracted, serious condition, or death, I don't believe this verse is indicating innocence merely because of a few days passing. The blows do have to directly result in death, however ("under his hand"), and therefore if he can walk on the first day, or second day, death or sickness after that is not to be taken to be as a direct result of the blows, and the master is innocent, i.e. not liable to the "rising up [in judgment]". However, if he still cannot walk on the third day, and subsequently dies in his bed, or remains sick, I believe this verse indicates legal vengeance is STILL to be taken.
In fact Ex. 21:19 teaches us the same principle:
אִם־יָק֞וּם וְהִתְהַלֵּ֥ךְ בַּח֛וּץ עַל־מִשְׁעַנְתֹּ֖ו וְנִקָּ֣ה הַמַּכֶּ֑ה רַ֥ק שִׁבְתֹּ֛ו יִתֵּ֖ן וְרַפֹּ֥א יְרַפֵּֽא
"If he rises and walks outside on his staff, then the smiter is innocent, except for the [cost of] his rest he will give to him, and cause him to be fully healed."
This further teaches us that if the servant rises, even after the second day, and walks outside on his staff [or stands up at all, v. 21], the master shall no longer be held liable (to "vengeance" [in judgement] for monetary compensation/blood money) should the servant die in his sickbed. It also teaches us that even a servant is to be compensated for his sick time by his master who beat him by fully healing him (and continuing his wages/compensation).
Our most problematic verse is לֹ֣א יֻקַּ֔ם כִּ֥י כַסְפֹּ֖ו הֽוּא "Don't rise up against him [in judgment], for he is his silver." For me, this proves that the verse is not speaking of the vengeance of the death penalty, but of paying the man's family the price of having severely wounded or killed a servant (accidentally). Because the person is able to stand up after a day or two, he or his family is not entitled to further compensation. Just to clarify, I am interpreting this to be an accidental death, not falling under the category of murder.
Further, when death by stoning is prescribed, the term, "he shall surely die" or "they shall surely be stoned" is used emphatically. Here, the less severe term of "vengeance shall surely be taken" is used. Clearly we are referring here to less severe forms of punishment than death. Clearly its a different type of situation than murder, involving the destroyed value of a man (here called "his silver") that may be compensated for accordingly. However, the fierce anger of a blood-avenger was seemingly condoned in Scripture in manslaughter cases should a civil solution fail, or should the blood avenger get ahold of the man before he could flee to a city of refuge.