From the Greek it's pretty difficult to tell whether Paul was using metaphor or was literally referring to a fight with beasts. But from cultural and historical contexts we might glean some understanding. Below are my notes from when I had this same question during my comprehensive exams, in no particular order. I may revise this later for readability. See conclusion for a TL:DR answer.
ἐθηριομάχησα (I fought with beasts) and its morphological sisters is used in contemporary contexts to refer to the Gladiatorial fights that would have happened in the Roman Empire, but it is unlikely that they would have happened in Ephesus as there was no arena there and the evidence at the Ephesian amphitheater would suggest that it was never used for animal fights. Perhaps staged battles between individuals, but certainly no animals.
The term for beast fight is technical, meaning that it may only, or at least primarily, refer to a literal fight with animals.
The Fight as Metaphor or Literal
Some commentaries identify the beasts metaphorically. Sometimes as reference to real people, to evil spirits, or as a sarcastic response to a false rumor.
Ignatius speaks of the supposed beast fight, but makes the argument that Paul uses it as metaphor, but it is at default a literal event. Ignatius seems to take the meaning to refer to the guards at Paul's prison, as he refers to his own captors as "leopards."
kata anthropon in Pauline usage refers primarily to deviating from the truth. MacDonald refers to Rom 3:5; 1 Cor 3:3; 9:8; Gal 1:11 for examples of this to indicate that Paul uses kata anthropon in this case of 1 Cor 15 to indicate that his fight with beasts is simply not true - pointing out that Paul may be using the Corinthians’ own belief in this fantastic story to call to task their lack of faith in the resurrection. MacDonald points to vv29-30 to parallel this type of attack on the Corinthians’ hypocrisy: they practice a ritual of baptism for the dead that betrays a belief in the resurrection, despite their denial of it.
Reworking the language of the text in 15:31 allows us to read the lion fight as the invention of Corinthian legend. MacDonald reads it as “By YOUR own boast, brothers, if, humanly speaking, I fought with wild beasts at Ephesus, what have I gained?” This implies that the Corinthians themselves were talking about a beast fight that Paul supposedly participated in.
Paul, then, seems to be responding to an unhistorical legend, which would explain the difficulty in harmonizing the literal interpretation with Roman law (a Roman citizen cannot be put to the beasts) and the silence of Acts and 2 Cor. as well as making the metaphorical explanation wholly unnecessary.
MacDonald also points out that Paul rails against “boasting” in 2 Cor. - that those who claim to be powerful miracle workers are to be avoided and treated with scrutiny. MacDonald suggests that the Corinthians’ penchant for the miraculous is consistent with the idea that they would have attributed Paul with some super-human abilities.
If the fight were a real event, the fact that Paul should have survived his encounter is also not beyond reality, because it happened to others as well. So much so that it became a motif of ancient art and Nero made a law restricting the granting of freedom of a prisoner to the emperor.
Sherman Johnson states that the riot in Ephesus mentioned in acts 19:23-41 may have been an indication of the climate that Paul had received in that city and may lend credence to his having fought beasts there, metaphorical or real. The silversmiths mentioned in that riot would have had a large source of clientele from the temples in the area who were deeply committed to their deities.
The devotion to the goddess Artemis was real and great in the area, so much so that Alexander the Great couldn't even get them to place his name on the temple. Because of this devotion, and the devotion to other gods, as well as being a center of medicine and a thriving port the city earned a special place under Roman rule, meaning that although there may have existed an unsatisfied minority in the first century, Paul's early mission there had a deep tradition to work against. Perhaps inciting some resentment from the locals.
Abraham Malherbe argues that Paul's utilization of diatribe and quoting from pagan sources, v33, indicates that something of a rhetorical argument is at hand. This could be a bit in support of MacDonald's claim that the Corinthians were spreading rumors.
Malherbe also notes that referring to one’s opponents as beasts is not unusual, especially when those opponents were the passions of the spirit. Heracles was seen at this time as a great example of self-control and called a "theriomaxos" inspiring moralists as an example of one having control over one’s self.
Odyssius' men are given as a counter example to heracles. They are turned into swine, wolves, and all manner of beasts when they are drugged and give in to their passions.
The use of beast metaphor as an assault on the spirit and mind is also used in Hebrew and other pagan writings (ie. the snake attacks reason in Genesis).
Malherbe tries to fit his metaphorical interpretation into the context of 1 cor 15.32 by saying that if he fought in without the hope of resurrection, there would have been no need to resist the beastial passions in the first place because there is no morality in death without the resurrection.
According to Robert Osborne, Paul was painting himself in the role of the hardened fighters that would have been in the arenas: as one trained and ready for battle.
Osborne concludes that the beasts were ‘hostile jews’ or ‘legalists,’ but does not commit to the claim, leaving it an open question.
Guy Williams observes that beasts were commonly used in Judaism as a symbol of evil spirits. Many apocalyptic images from psalms, Daniel, 4 Macabees, Revelation, etc. Beasts are used time and again in places where it would be fitting to speak of demons or evil spirits. Williams also points out that several ‘defensive spells’ against wild animals have survived Wild animals played a crucial role in the magician’s gaining power. Dead and living animals are used. This is particularly true for the cult of Artemis, where many depictions of the goddess are accompanied by animals.
A particular charm appealing to Artemis is of particular interest:
Come to me, horned-faced, light-bringer, bull-shaped, horse-faced,
goddess who howls doglike; come here, she-wolf, and come here now,
mistress of night and chthonic realms, holy, black clad.
Not only did Jewish tradition use wild beast imagery but also the magical texts of other traditions and cultures.
Paul encountered many exorcisms, magical rivalries, and violent controversies regarding idolatry in Ephesus.Though it is dubious to rely upon these claims of magic, Paul did live in a world where people really did perform these rituals and believed in their power.
Early reception of the beast fight took a more literal approach because, although during Paul’s lifetime it was not so, Christians had begun being put to the beasts in the arenas of Rome. And although many early christian writers took a literalist approach, Origen at least seems to back up the metaphorical thesis put forward by Williams - that the beasts referred not to people or passions, but to the evil spirits and satan - though Williams notes that Origen may have believed the therion to be in real conflict with humanity, not merely a symbol.
In his closing statement Williams addresses the phrase 'in the manner of men,' which under his thesis makes more sense because if he had fought with merely his flesh he would have lost but he presumably fights with the spirit against his spiritual adversaries and prevails.
In response to your actual question, it seems that we should understand Paul's meaning to be in a sense both metaphorical AND literal. Paul was literally facing challenges from what he would have identified as 'wild beasts' in the form of the spirits from the cult of Artemis and he was using the image of the beast fighter in the arena to portray himself as one well armed against such opponents. It also seems there was a penchant for rumor in Corinth, and Paul was using that to his advantage to speak about the truth of the Resurrection.
It is likely that Paul uses Ephesus because of his encounter with the silversmiths there as well as the staunch opposition and useful imagery given by the cult of Artemis.
While there were indeed animal fights staged in the local amphitheaters, they were nowhere near the scale reached by those in the Colosseum in Rome. So, the Corinthians would have understood the idea of a beast fighter.
As for the use of beasts as a means of religious persecution, this did not likely happen until a few generations after Paul's ministry as Christians were mostly regarded as divergent Jews, against whom the Romans had no explicit prohibitions at that time.
Reading the question as counter-factual might be a good approach, suggesting a rhetorical or hypothetical argument. I'm not sure this adds significantly more meaning than a metaphorical reading though.
Johnson, Sherman E. "The Apostle Paul and the riot in Ephesus." Lexington Theological Quarterly 14 (1979): 79-88.
MacDonald, Dennis Ronald."A Conjectural Emendation of 1 Cor 15:31-32: or the case of the misplaced lion fight." Harvard Theological Review 73 (1980): 265-276.
Malherbe, Abraham J. “The Beasts at Ephesus.” Journal of Biblical Literature 87 (1968): 71-80.
Osborne, Robert E. “Paul and the Wild Beasts.” Journal of Biblical Literature 85 (1966): 225-230.
Williams, Guy. "An Apocalyptic and Magical Interpretation of Paul's 'beast fight'; in Ephesus (1 Corinthians 15:32)." Journal Of Theological Studies 57 (2006): 42-56.