Contemporary Jewish Apocalypses
2 Esdras is a Jewish apocalypse with later Christian additions. One chapter, written by the original Jewish author, has the following:
In the thirtieth year after the destruction of the city, I was in
Babylon — I, Salathiel, who am also called Ezra. I was troubled as I
lay on my bed, and my thoughts welled up in my heart, because I saw
the desolation of Zion and the wealth of those who lived in Babylon.
(2 Esdras 3.1-2, NRSV)
'Then I said in my heart, Are the deeds of those who inhabit Babylon
any better? Is that why it has gained dominion over Zion? For when I
came here I saw ungodly deeds without number, and my soul has seen
many sinners during these thirty years. And my heart failed me,
because I have seen how you endure those who sin, and have spared
those who act wickedly, and have destroyed your people, and protected
your enemies, and have not shown to anyone how your way may be
comprehended. Are the deeds of Babylon better than those of Zion?' (2
Esdras 3.28-31, NRSV)
According to this text, 'Ezra' is writing thirty years after the 587 BC destruction of Jerusalem. Internally, this would date the book to c.557 BC, a full century before the biblical Ezra was even active, which is simply not feasible. Among other telltale signs that this was not written by the Ezra, the book is accordingly dated to thirty years after the second destruction of Jerusalem, which took place in 70 AD. Thus, 2 Esdras 3.1-2 was written c.100 AD by a Jewish author who cast Rome in the role of 'Babylon'.
The same appears to be the case in 2 Baruch, an apocalyptic book written about the same time, where the author assumes the name of the biblical Baruch, writing about the 70 AD destruction of Jerusalem as if it was the 587 BC destruction. Consequently, 2 Baruch 10.1–3, 11.1, and 67.7 have the author writing about 'Babylon', which are considered once more to be references to Rome.
The Sibylline Oracles
The Sibylline Oracles (not to be confused with the non-extant Sibylline Books), a large set of apocalyptic visions, were added to by Jews, Christians, and Gnostics over several centuries. (The theology in different passages conflicts, and represents the various editors.) Book 5.187-205, written c.130-160 AD, very clearly equates Rome with Babylon when it puts the blames for the temple's destruction on Nero (the 'mighty king' who 'laid his hands upon the womb', i.e. murdered his pregnant wife):
When one from Italy shall smite the neck
Of the isthmus, mighty king of mighty Rome,
A man made equal to God, whom, they say,
Zeus himself and the august Hera bore
He, courting by his voice all-musical
Applause for his sweet Songs, shall put to death
With his own wretched mother many men.
From Babylon shall flee the fearful lord
And shameless whom all mortals and best men
Abhor; for he slew many and laid hands
Upon the womb; against his wives he sinned
And of men stained with blood had he been formed.
And he shall come to monarchs of the Medes
And Persians, first whom he loved and to whom
He brought renown, while with those wicked men
He lurked against a nation not desired
And on the temple made by God he seized
And citizens and people going in,
Of whom I justly sang the praise, he burned
(Sibylline Oracles, Book 5.187-205, Milton S. Terry revised translation)
Why Call Rome 'Babylon'?
While we do not know when exactly the symbolic name was attached to Rome (certainly after 70 AD, but possibly not until the 80s or 90s), the reason behind the identification of Rome with Babylon in all of the above instances is clearly because Rome, like Babylon, was responsible for the destruction of a temple in Jerusalem.
Rome and 1 Peter 5.13
As has been pointed out, there is no known tradition in early Christian history that places Peter at the actual city of Babylon in the middle east. In the first century, Babylon was in ruins, so any excursion there would have been pointless to the evangelistic efforts otherwise described for Peter.
The earliest implication that Peter traveled to Rome is 1 Clement 5. The author, a church overseer (i.e. bishop) writing from Rome c.95 AD, groups the death of Peter and Paul into a single paragraph.
The first explicit reference that Peter was associated with Rome, is Ignatius to the Romans 2.6, written c.100-110 AD.
Between the earliest known Christian traditions that Peter went to Rome, and that 'Babylon' was a regular codename for Rome, it is highly probable that 1 Peter 5.13 also refers to Rome.
Rome and the Revelation
While extra-biblical evidence is helpful, it is still necessary to examine the internal evidence of the Revelation, to see if the book truly does have Rome in mind when it speaks of 'Babylon'.
Revelation 17 identifies Babylon as 'the great city', saying she sits on 'seven mountains' or 'hills'.1 John was writing c.95 AD, and has identified his primary audience as Christians living in Asia, so we are well within the realm of the Roman empire. Historically, it is most probable this 'city of seven hills' was a clear identification of Rome, since the city was widely known by such a nickname.2
Adding directly to this, imperial coins minted c.71 AD depicted Roma, the goddess personifying Rome, as seated on the city's seven hills. Taking the Revelation in its historical context, the prostitute seated on seven hills is likely a criticism of the image. View the coin here: iCollector.com.
Understandably, there is a lot of disagreement on what the Revelation exactly is trying to describe because of just how difficult the book is to interpret. But because the following points have been raised by commentators at all, I will include them for consideration.
The most famous piece of circumstantial evidence is the 'number of the beast' (666). It has widely been suggested to be gematria of the name 'Neron Caesar', the first Roman emperor to engage in (non-official) persecution of Christians, having made them the scapegoat of the Great Fire of Rome (64 AD).
Revelation 1.16,20, 2.1, and 3.1 describe Jesus as holding seven stars in his right hand. This is widely thought to reference imperial coins minted c.95 AD, which depicted the son of emperor Domitian seated on the globe of the earth on the reverso, holding seven stars over his head. View the coin here: Wikimedia Commons.
Revelation presents God sitting on his throne in heaven. The author states that God resembled jasper and carnelian (4.2-3), two stones that are often a deep red. Immediately after this, the author describes Jesus as a victorious conqueror (5.5) who presents a sacrifice before the throne (5.6). Later still, Jesus is shown riding on a white horse in conquest (19.11ff). This imagery is altogether reminiscent of a triumph ceremony for generals in Rome, who would ride a white horse to the Temple of Jupiter Optimus Maximus of the Capitoline Hill, where a sacrifice would be offered before the red-colored statue of Jupiter.3
There are several other possible anti-Roman symbols in the Revelation, but the above is sufficient to make the point.
Bearing in mind the historical context, the agreement of contemporary sources, and internal indications, it is very certain 'Babylon' refers to Rome, in both 1 Peter and the Revelation.
1 The Greek noun, ορος, most often means 'mountains', but can rarely be used for 'hills'. Source: Perseus Digital Library.
2 Cicero to Atticus, letter 6.5; Virgil, Georgics 2.535; Virgil, Aeneid 6.781-783; Sextus Propertius, Elegies 3.11.55-57; Horace, Secular Hymn 7,11; Ovid, Tristia 5.69; Martial, Epigrams 4.64; Sibylline Oracles 2.19; 11.145-154; 13.61; 14.138.
3 Pliny the Elder, Natural History 35.157; Johnston, Religions of the Ancient World, p. 618; and McDonald & Walton, Greek and Roman Theatre, p. 186.