How are we expected to "calculate the number"?
As someone has already noted, the dictionary meaning of ψηφίζω (psēphizō) is to count or add up. The word is related to the noun ψῆφος (psēphos), meaning pebble - a device commonly used for counting things. In this sense, psēphizō literally meant something like "to pebble" - though this form doesn't make sense to us in our time.
The indication here is, I think, not how we calculate the number itself, but rather how we add up the number to deduce or infer something.
Meaning of ψηφίζω
It's a very uncommon word in the Bible (New Testament and Greek Septuagint Old Testament), only occurring twice. Luke 14:28 is the other occurrence:
For which of you, intending to build a tower, sitteth not down first, and counteth the cost, whether he have sufficient to finish it?1
In addition to count, the word can also mean to vote or decide by vote - because pebbles were also used as markers for votes as well as for counting things. An Example from Thucydides The Peloponnesian War (VII.48):
ὁ δὲ Νικίας ἐνόμιζε μὲν καὶ αὐτὸς πονηρά σφῶν τὰ πράγματα εἶναι, τῷ δὲ λόγῳ οὐκ ἐβούλετο αὐτὰ ἀσθενῆ ἀποδεικνύναι, οὐδ᾽ ἐμφανῶς σφᾶς ψηφιζομένους μετὰ πολλῶν τὴν ἀναχώρησιν τοῖς πολεμίοις καταγγέλτους γίγνεσθαι
Nicias, though he also thought their estate bad, yet was unwilling to have their weakness discovered, and, by decreeing of their departure openly with the votes of many, to make known the same to the enemy.2
A better translation than "calculate" here might be "reckon". "Reckon" can to mean to "count" (though it is not that common in modern English), but it can also mean something like to think things through and come to some sort of conclusion, or to come to some sort of judgment. It is used in something like this sense in Demosthenes Against Macartatus (XLIII.6):
ἐχούσης δὲ τῆς μητρὸς τοῦ παιδὸς τουτουὶ τὸν κλῆρον, ἐπειδὴ ἐνίκησεν ἐν τῷ δικαστηρίῳ ἅπαντας τοὺς ἀμφισβητήσαντας ἑαυτῇ, οὕτως εἰσὶν μιαροὶ οὗτοι, καὶ οὐκ οἴονται δεῖν οὔτε τοῖς νόμοις τοῖς ὑμετέροις πείθεσθαι οὔτε τοῖς γνωσθεῖσιν ἐν τῷ δικαστηρίῳ, ἀλλὰ πάντα τρόπον ἐπιχειροῦντες ἀφελέσθαι πάλιν τὴν γυναῖκα τὸν κλῆρον ὃν ὑμεῖς αὐτῇ ἐψηφίσασθε
But, although the mother of this boy here became possessed of the inheritance, since she had prevailed in the suit over all those who disputed her claim, these men are abominable, as you see, and imagine that they need obey neither your laws nor the decisions of your courts, but they are trying by fair means or foul once more to take away from the lady the inheritance which you awarded to her.3
The oldest complete commentary on Revelation was written in the late 6th or early 7th century by Andrew of Caesarea, a late Greek Church Father. He comments:
Here is wisdom: let him who has a mind reckon the number of the beast, for it is the number of a human. And his number is six hundred and sixty-six
The exact sense of the numerical cipher, as well as the rest of the things written regarding this, time and experience will reveal to those who live soberly. Divine grace was not well pleased to set down the name of the destroyer in the divine book.4
So, according to Andrew, the number is arrived at not through cold logical deduction, but will reveal itself in time to those who live soberly.
Nonetheless, as Andrew himeslf notes, "many names are to be found contained in this number."5 The method of equating names to numbers is called gematria, explained by Lawrence Farley in his book, The Apocalypse of Saint John: A Revelation of Love and Power:
The method of using numbers to indicate a name is called gematria, and it was quite common in the ancient world. (Indeed, in the graffiti found in Pompeii, archaeologists have uncovered one saying, “I love her whose number is 545.”) It is here used because it was too dangerous to name the emperor himself, clearly and plainly, as the beast and enemy of God.
In like manner, St. Paul, in writing to the Thessalonians, shrinks from clearly saying, “The emperor and his empire is that which hinders the final revelation of the man of lawlessness. The emperor must be removed before this can happen.” To write so clearly about the “removal” of the emperor would be to risk a charge of treason. Instead, he says in a roundabout way, “Do you not remember that while I was yet with you, I said these things to you? You know what holds [the Antichrist] back now [viz. the Roman Empire]. . . . He who presently holds him back [viz. the presence of the emperor] will do so until he is taken out of the way” (2 Thess. 2: 5– 7). Paul writes in such an elusive way so as to be understood by the faithful, but not by others.
It is the same here. The practice of gematria (whereby one adds up the numerical equivalents of a word and substitutes the number for the name) serves as a cover for the faithful. The Church is meant to understand the riddle, whereas the hostile pagan world will not.
One application of gematria here actually yields an abbreviation found on coins issued during the rule of the Roman Emperor Domitian, who was the Emperor responsible for exiling John to Patmos:
A. KAI. DOMET. SEB. GE.
AUTOKRATOR KAISAR DOMETIANOS SEBASTOS GERMANIKOS
Using the numerical equivalents for the Greek letters, the abbreviated form adds to 666:
A = 1
K = 20
A = 1
I = 10
D = 4
O = 70
M = 40
E = 5
T = 300
S = 200
E = 5
B = 2
G = 3
E = 5
Clearly this is not a unique solution, but the example does illustrate - with your question in mind - exactly how one might reckon the name from the number.
How did some copyists get the number wrong?
As I wrote above, I don't think the verse is referring to "calculating" the number itself, but deals rather with what we infer or deduce from the number.
Nonetheless, your point is well taken.
As early as the 2nd century, it was known that there were variants in the extant manuscripts. Irenaeus (130-202) wrote:
I do not know how it is that some have erred following the ordinary mode of speech, and have vitiated the middle number in the name, deducting the amount of fifty from it, so that instead of six decades they will have it that there is but one [i.e. 616 instead of 666]. (I am inclined to think that this occurred through the fault of the copyists, as is wont to happen, since numbers also are expressed by letters; so that the Greek letter which expresses the number sixty was easily expanded into the letter Iota of the Greeks.)6 Others then received this reading
without examination; some in their simplicity, and upon their own responsibility, making use of this number expressing one decade; while some, in their inexperience, have ventured to seek out a name which should contain the erroneous and spurious number. Now, as regards those who have done this in simplicity, and without evil intent, we are at liberty to assume that pardon will be granted them by God. But as for those who, for the sake of vainglory, lay it down for certain that names containing the spurious number are to be accepted, and affirm that this name, hit upon by themselves, is that of him who is to come; such persons shall not come forth without loss, because they have led into error both themselves and those who confided in them. Now, in the first place, it is loss to wander from the truth, and to imagine that as being the case which is not; then again, as there shall be no light punishment [inflicted] upon him who either adds or subtracts anything from the Scripture.7
The most important manuscript containing 616 instead of 666 is the Codex Ephraemi, which dates from the 5th century. From Irenaeus' testimony (above), we know that there must have been some manuscripts circulating in the 2nd century that also contained this reading, but they are now lost.
Metzger's Textual Commentary on the New Testament gives some additional background on the variant readings:
Instead of ἑξήκοντα, which is strongly supported by 𝔓47 א A P 046 051 all extant minuscules itgig vg syrph, h copsa, bo arm al, δέκα is read by C some manuscripts known to Irenaeus (who, however, says that 666 is found “in all good and ancient copies,” and is “attested by those who had themselves seen John face to face”) and Tyconiuspt. According to Tischendorf’s 8th ed., the numeral 616 was also read by two minuscule manuscripts that unfortunately are no longer extant (nos. 5 and 11; cf. C. R. Gregory, Prolegomena, p. 676).2 When Greek letters are used as numerals the difference between 666 and 616 is merely a change from ξ to ι (666 = χξς and 616 = χις). Perhaps the change was intentional, seeing that the Greek form Nero Caesar written in Hebrew characters (נרון קסר) is equivalent to 666, whereas the Latin form Nero Caesar (נרו קסר) is equivalent to 616.
Thus, there seem to be no final conclusions as to why some copyists "got it wrong", but the above provides a number of different plausible explanations.
2. Hobbes translation
3. From Perseus
4. E. Constantinou, Andrew of Caesarea and the apocalypse in the ancient church of the East: Studies and Translation. PhD thesis, Quebec: Université Laval (2008), p.146
6. 666 is represented numerically by the letters χξς; 616 is represented by χις. The difference is in substituting ι ("iota") for ξ ("xi").
7. Against Heresies V.30.1