Paul speaks of tongues throughout chapter 14, reviewing the entire chapter will prove useful. I've split my response into the answer in summary first and my work following it. Also, I've split my work into sections (1,2,3) and (A,B,C) which you can refer to when reading the answer.
The Answer
Tongues are mentioned along side prophecy many times and mention is even made of speaking in the tongues of angels. "Prophecy" doesn't always mean foretelling the future, it can simply be delivering a convicting message of truth.(A)(2) If, and this is simply an "if", tongues were used for delivering such messages then what Paul is saying would still apply. They would need an interpreter or the message would not be understood and if the message cannot be understood then why deliver it? (B)
Paul is saying that the purpose of speaking in tongues is to be a sign, a proof of sorts, to the unbeliever. Therefore, there is no reason for a group of believers to be speaking in tongues in this way to one another, for they already believe and do not need a miraculous sign. (1)
This is why Paul says that a nonbeliever would think the believers mad if he walked in on them speaking in tongues to each other. You see, the tongues are supposed to be a sign to him, the nonbeliever, directed to him and for him. (C)
If I understand the question correctly, the confusion is mostly with the verses 24 and 25.
Paul never said prophecy was only effective on believer. When he says "But if all prophesy" he is not saying they are all prophesying to each other only, but that they all possess the gift of prophecy. The contrast between the two scenarios is to show that a message of truth is effective and edifying for nonbelievers as well.(D)
Best yet is tongues to the nonbeliever, followed by convicting words of prophecy and all the gifts of the Spirit working together for the benefit of all members of the church.
1 Cor 14:26 How is it then, brethren? when ye come together, every one of you hath a psalm, hath a doctrine, hath a tongue, hath a revelation, hath an interpretation. Let all things be done unto edifying.
Paul's point is more about how the stranger perceives them from the outside. Improper tongues looks crazy, proper prophecy is moving.
Also if the tongues spoken of by Paul to the Corinthians was the type of ecstatic, religious experience "Glossolalia" that was common among pagan (including Greeks of Corinth) religions, then the meaning is further enhanced. Such tongues serve no purpose to a nonbeliever and even serve to turn them away, but divine truth can convict their hearts in a way that no pagan religion ever had.(3)
The Work
Uses of Tongues
"Tongues" takes on a number of meanings and uses in the New Testament.
- A literal language
- Praying in tongues
- Speaking in Angelic tongues
(1) Literal Language
Acts 2:3 And there appeared unto them cloven tongues like as of fire, and it sat upon each of them.
4 And they were all filled with the Holy Ghost, and began to speak with other tongues, as the Spirit gave them utterance.
5 And there were dwelling at Jerusalem Jews, devout men, out of every nation under heaven.
6 Now when this was noised abroad, the multitude came together, and were confounded, because that every man heard them speak in his own language.
7 And they were all amazed and marvelled, saying one to another, Behold, are not all these which speak Galilaeans?
8 And how hear we every man in our own tongue, wherein we were born?
Act 10:44 While Peter yet spake these words, the Holy Ghost fell on all them which heard the word. 45 And they of the circumcision which believed were astonished, as many as came with Peter, because that on the Gentiles also was poured out the gift of the Holy Ghost.
46 For they heard them speak with tongues, and magnify God. Then answered Peter,
In every instance of the use of "tongues" outside of 1 Corinthians 12 and 14, it is in the context of a literal language that is heard and miraculously understood.
It is always for the purpose of communicating the truth of the Gospel to unbelievers. Even if you wish to understand the use of tongues in Acts 10:46 to be Angelic tongues that the Gentiles were simply impressed by, it was still with the purpose of their conversion.
(2) Praying in Tongues
Paul does not forbid the speaking of tongues without recipient or interpreter (an unknown tongue). Rather, he simply says that if no one (seemingly including you) can interpret, then it is just for you and your spirit, for speaking with yourself and God.
1 Cor 14:4 He that speaketh in an unknown tongue edifieth himself; but he that prophesieth edifieth the church.
1 Cor 14:13 Wherefore let him that speaketh in an unknown tongue pray that he may interpret.
14 For if I pray in an unknown tongue, my spirit prayeth, but my understanding is unfruitful.
15 What is it then? I will pray with the spirit, and I will pray with the understanding also: I will sing with the spirit, and I will sing with the understanding also.
1 Cor 14:28 But if there be no interpreter, let him keep silence in the church; and let him speak to himself, and to God.
(3) Tongues of Angels
There isn't much here in Scripture. We have Paul's mention of it in 1 Corinthians 13:1
1 Cor 13:1
Though I speak with the tongues of men and of angels, and have not charity, I am become as sounding brass, or a tinkling cymbal.
Some will contend that the tongues spoken of in 1 Corinthians 14 is this angelic language, and that is why it was "unknown". However, two things should be noted:
1) The term "unknown" was inserted by translations as an aid, there is nothing in the Greek to directly translate there.
2) They were unknown, not because they were not necessarily languages of men, but because there was no one there who spoke that language and could interpret, either someone else or the speaker. That is why Paul says in 1 Cor 14:14 "For if I pray in an unknown tongue, my spirit prayeth, but my understanding is unfruitful." His understanding is unfruitful because even he does not understand the language.
There are other extra-biblical accounts of speaking angelic language. In the case of the Testament of Job, it includes the speaker (or singer) having some knowledge of what they are speaking. But that does not seem to be the case in what Paul is writing.
Unfortunately the concept of "Glossolalia" is most commonly seen outside Christianity. Many pagan priests and priestesses used it. The Greeks of Corinth would have been well aware of this and perhaps that is the issue at hand with the church at Corinth. If Paul is speaking of Glossolalia in 1 Cor 14 then it would seem he is warning against it while respectfully not outlawing it, not wanting to stand against a possible working of God. This would also explain why it does not come up in this context in any other New Testament book, but was a specific issue to Corinth.
(A)
1Co 14:3 But he that prophesieth speaketh unto men to edification, and exhortation, and comfort. 4 He that speaketh in an unknown tongue edifieth himself; but he that prophesieth edifieth the church.
I'll also refer to Christianity Stack Exchange, which seems appropriate, all upvoted answers agree at least on this point, that prophecy is for divine instruction. It may be for the future, but it is just as often for the present.
(B)
1 Cor 14:6 Now, brethren, if I come unto you speaking with tongues, what shall I profit you, except I shall speak to you either by revelation, or by knowledge, or by prophesying, or by doctrine?
7 And even things without life giving sound, whether pipe or harp, except they give a distinction in the sounds, how shall it be known what is piped or harped?
8 For if the trumpet give an uncertain sound, who shall prepare himself to the battle?
9 So likewise ye, except ye utter by the tongue words easy to be understood, how shall it be known what is spoken? for ye shall speak into the air.
1 Cor 14:16 Else when thou shalt bless with the spirit, how shall he that occupieth the room of the unlearned say Amen at thy giving of thanks, seeing he understandeth not what thou sayest?
17 For thou verily givest thanks well, but the other is not edified.
(C)
When directed at others, it looks and sounds ridiculous. We see in Acts 2:13 that those who did not receive the Spirit and were involved in the speaking of tongues, that is speaking or being spoken to, saw all those speaking in tongues as being drunk and mocked them.
Also:
1 Cor 14:10 There are, it may be, so many kinds of voices in the world, and none of them is without signification.
11 Therefore if I know not the meaning of the voice, I shall be unto him that speaketh a barbarian, and he that speaketh shall be a barbarian unto me.
The word for "voice" (φωνή, phone) can also be understood as a noise or sound, but also as a language. While it may be odd that Paul switches to this term when he has been using "tongues" (γλῶσσα,glossa), however when we consider what he is saying, it is very appropriate that he uses a term that means both language and sound. For, as he says, if you do not understand the language, it just sounds like noises. That is why
(D)
After reading the "if" statement of v23, we expect a similar inverse reaction in v24 and 25. That the prophecy would not effect him because we were just told that prophecy is for believers. So it seems logical to conclude that, as a nonbeliever, prophecy should then not be for him. But instead of the nonbeliever being confused by walking in on a group prophesying to each other, he is convicted. We should not see v23 as an inverse to v24,25, but rather how each scenario would effect the same stranger. While the nonbeliever thought tongues between believers was madness, he is convicted when he see's them prophesying to one another. This actually enhances the point that prophecy is to be used internally, for the stranger reacts positively when he see's the believers using it properly.
A real inverse of v23 would be something like "If a group of believers were out on the street prophesying to strangers, would not the nonbeliever think them mad?" But rather than inverting the wrong use with another wrong use, he contrasts the incorrect with the correct.
Returning to v23, the mockers of Acts 2:13 were not involved like the unbeliever in v23 and therefore it was ineffectual. But how does Peter respond to their mocking?
Acts 2:14 But Peter, standing up with the eleven, lifted up his voice, and said unto them, Ye men of Judaea, and all ye that dwell at Jerusalem, be this known unto you, and hearken to my words:
He responds with a sermon, a word of truth and prophecy. If prophecy was ineffective on the nonbeliever, why would Peter have responded in such a way. Why would he not have spoken to them in tongues? While the narrative does not say if Peter's sermon did effect the mockers specifically, we are told many heard and responded to his words:
Act 2:41 Then they that gladly received his word were baptized: and the same day there were added unto them about three thousand souls.
It's getting late so I've tried to make it coherent enough to post, let me know if something got jumbled