It should be noted that Mark split the story of the fig tree in two parts, in between is the cleansing of the temple courts.
In the first part (Mark 11:12-14), Mark described Jesus was hungry. He saw a fig tree in leaf but not bearing fruit, then He said to the tree "no one ever eat fruit from you again" and his disciples heard it.
In analysis to the first part, we may recall Jesus was tempted to turn stone into bread and he didn't do it (Luke 4:2-4). In Mark 13:28, Jesus told His disciples “Now learn this lesson from the fig tree: As soon as its twigs get tender and its leaves come out, you know that summer is near." So Jesus apparently knew He should barely find fruit on the fig tree as it was early April. Therefore it may hardly be explained that Jesus was upset with the fig tree for He couldn't have fruit to ease His hunger.
Continue to the second part, we may see the act of Jesus was actually a metaphor teaching to His disciples.
First of all, Jesus did not directly response to Peter's comment, not even objected to his word "curse". But He said;
23 “Truly I tell you, if anyone says to this mountain, ‘Go, throw yourself into the sea,’ and does not doubt in their heart but believes that what they say will happen, it will be done for them.
24 Therefore I tell you, whatever you ask for in prayer, believe that you have received it, and it will be yours.
25 And when you stand praying, if you hold anything against anyone, forgive them, so that your Father in heaven may forgive you your sins.” (Mark 11:23-25 NIV)
If we thought Jesus used the fig tree represent the Jewish nation, their faith in God is the leaves, and their reward is the fruit. As their faith withered, their fruit is gone. The metaphor was physically illustrated when Jesus cleansed the temple court, which mean Jesus removed their centre of their faith, and no longer anyone could benefit from it.
Perhaps Mark 11:25 is a response to Peter about his word "curse". Jesus asked people to forgive each other, as He forgives our sin when we repented. Otherwise, the sinners will meet their destruction, which becomes their curse. Jesus did not object to Peter using 'curse' in his sentence, for in some situation, Peter was not totally wrong.