It is omitted in versions where the committee of experts behind the translation determined that those words were most likely not in the original text of Luke. In this particular case, the evidence that these words were not original is very strong, though not completely overwhelming.
The shorter version is found in:
- Both extant Papyrus texts (P45 and P75, dated circa 250 and 200 CE)
- The four oldest bibles (Vaticanus, Sanaiticus, Alexandrinus, Ephraemi Rescriptus, 4th and 5th century)
Although later documents are more split, such complete agreement among all our early texts is very strong evidence.
Codex Bezae (around 400) has a unique reading with half of the longer version. Codex Bezae has a lot of unusual readings, and is usually considered somewhat unreliable. There are some early Latin and Syriac documents with the longer reading. In fact, the vast majority of Latin manuscripts have the longer reading, which explains how it got into say the KJV.
I've taken this information from here, since it is available free online. A more reliable source would be Nestle-Aland.
Finally, a good place to get quick summaries of issues like this one is the NET bible footnotes. Here the relevant footnote (justifying the shorter choice) reads:
Many mss ([D] K Γ Θ Ë1,13  700 2542 pm it) have at the end of the
verse (with slight variations) “and he said, ‘You do not know what
sort of spirit you are of, for the Son of Man did not come to destroy
people’s lives, but to save [them].’” This variant is clearly
secondary, as it gives some content to the rebuke. Further, it is
difficult to explain how such rich material would have been omitted by
the rest of the witnesses, including the earliest and best mss.