Paul is referring to the inclusion of non-Jews in the religion, without conversion to Judaism or becoming Jewish. Not the fact of the Messiah itself; although various details of the Messiah such as his death and suffering were not clearly understood by most. They were understood by a few, but were not so clear. There are some teachings in the Jewish sources that talk about the Messianic age, but as the prophecies they too maybe vague and hidden.
John Eadie commentary states on Eph 3:5
The general sense of the verse is evident. The apostle does not seem to deny all knowledge of the mystery to the ancient world, but he only compares their knowledge of it, which at best was a species of perplexed clairvoyance, with the fuller revelation of its terms and contents given to modern apostles and prophets; or as Theodoret contrasts it-οὐ γὰρ τὰ πράγματα εἶδον, ἀλλὰ τοὺς περὶ τῶν πραγμάτων προέγραψαν λόγους. In Vetere Testamento Novum latet, et in Novo Vetus patet. The scholium in Matthiae—“that the men of old knew that the Gentiles should be called, but not that they should be fellow-heirs,” contains a distinction too acute and refined. The intimations in the Old Testament of the calling of the Gentiles are frequent, but not full; disclosing the fact, but keeping the method in shade. The apostle James refers to this in Act 15:14. But after the death of Christ, which, by its repeal of the ceremonial code, was the grand means of Judaeo-Gentile union, a church, without reference to race, was fully organized. The salvation of guilty men of all races became a distinctive feature of the gospel, and therefore the incorporation of non-Israel into the church, revealed to Peter and Paul by the Spirit, was more clearly understood from the results of daily experience and the fruits of missionary enterprise. Act 11:17-18; Act 15:7; Act 15:13.
Also, it is a human tendency to ignore some plain teachings under the traditional mindset. The Jews believed that the gentiles are created just as fuel for hell fire; and ironically, the new covenant gentiles believe the same thing about unbelievers. Thus, a hidden minor fact about the gentiles inclusion in the Messianic age, to be mystery, is not surprising.