Matthew 5-7 is most naturally read as a single literary unit. Jesus went up to a mountainside and his disciples went with him (Matt. 5:1-2). However, by the end of the discourse we notice that the crowds are amazed by Jesus' teaching. It seems that the crowds found Jesus and the disciples.
I contend that Jesus' primary audience are his disciples. At this point it is not delineated as to whether or not this means the Twelve, the seventy, or a larger group of disciple. However, I read 5:1 to indicate a withdrawal by Jesus from the larger multitudes. That's not to say that they had nothing to gain from the teaching, but that Jesus was specifically addressing the disciples and the remainder were welcome to listen.
My sense is that the "crowds" serve as a larger people group and are held over against the disciples, however many they are. This would be consistent with Matthew's portrayal of "the crowds" as being a large group of people who were looking for a miracle, or people who were there for the show. I guess I'd call them "potential insiders" but not "insiders" like the disciples.
There are no breaks in the discourse of Matt 5 - 7 that would indicate a shift of attention to this broader audience.
The so-called Sermon on the Plain has too much literary and conceptual overlap to not be describing the same event. Differences between the two are explicable (but that'd be a different question). Luke still has the disciples as his primary audience in his version (Lk. 6:20) but his account reads more along the lines that the crowds had already found them.
So a reasonable timeline would be:
- Jesus sees the large crowds
- Jesus withdraws to pray
- The disciples (apparently a large group) go to find Jesus
- Jesus talks to his disciples as the crowds find them