Malachi was not speaking to the lineage of the priesthood, but of the duty of the priest as a messenger / angel of the LORD. They were to keep the knowledge of the law and to teach the law to the people, and as such were messengers of the law of God. Mal. 2:7 is a statement of generic function of the priesthood which applied to all of the original, true priests of Levi.
Matthew Poole's Commentary here:
"Those forementioned excellent priests did so teach, and so live, forasmuch as they did well consider it was their duty to be well acquainted with, and to have a great insight into, the law of God.
The priest’s lips should keep knowledge; it is that their office binds them to; it is the duty of all God’s people to know his law, but the priest’s duty to know it more than others, Leviticus 10:11, for they were to teach Israel, Deu 33:10.
And they, the people of Israel, should seek the law at his mouth; in difficult cases, in controversies, &c., the people were to consult and advise with the priests, and inquire what the law said in the case.
For he is the messenger, interpreter, ambassador, or legate, of the Lord of hosts with the people, lieger among them, and who therefore ought to be advised with about his Lord’s mind. " Source: Biblehub
Mal. 3:1 was speaking of two messengers (angels). The first was set to prepare the way before the Lord (Christ), and was therefore speaking of John the baptizer. The second messenger of the covenant was Christ.
Ellicott's Commentary on Mal. 3:1 -
"My messenger.—Heb., Malachi, my angel, or my messenger, with a play on the name of the prophet. In Malachi 2:7, he calls the priest the angel or messenger of the LORD. There can be little doubt that he is influenced in his choice of the term by his own personal name (see Introd.). This “messenger,” by the distinct reference to Isaiah 40:3, contained in the words, “and he shall prepare,” &c., is evidently the same as he whom [the deutero-] Isaiah prophetically heard crying, “In the wilderness, prepare ye the way of the LORD, make straight in the desert a highway for our God.” Moreover, from the nature of his mission, he is proved to be identical with the “Elijah” of Malachi 4:3. These words had their first, if not their perfect fulfilment in John the Baptist (Matthew 17:12).
The Lord.—This word “Lord” occurs eight times with the definite article, but always, except here, with the name of God following it: viz., Exodus 23:17, followed by “Jehovah;” Exodus 34:23, by “Jehovah, the God of Israel;” in Isaiah 1:24; Isaiah 3:1; Isaiah 10:33; Isaiah 19:4, by “Jehovah Zebaoth;” and in Isaiah 10:16, by “the Lord of Zebaoth.” And here, as elsewhere, it must mean God Himself, because He is said to come “to his temple,” and because He is said to be He “whom ye seek:” i.e., “the God of judgment” (Malachi 2:17).
Even—i.e., “namely,” for so the Hebrew conjunction “and” is frequently used: e.g., Exodus 25:12; 1Samuel 28:3.
The messenger (or angel) of the covenant.—This expression occurs only in this passage. Identified as He is here with “the Lord,” He can be no other than the Son of God, who was manifested in the flesh as the Messiah. In the word “covenant” there is, perhaps, some reference to the “new covenant” (Jeremiah 31:31), but the meaning of the word must not be limited to this." Source: Biblehub
The other commentaries at the source are even more eloquent on these two messengers in Mal. 3:1.
While the verses in Malachi do not openly bring in the Levitical line of the priesthood, it is surely understood as Heb. 7:11-19 does openly signify it.
"11 If indeed, then, perfection were through the Levitical priesthood -- for the people under it had received law -- what further need, according to the order of Melchisedek, for another priest to arise, and not to be called according to the order of Aaron?
12 for the priesthood being changed, of necessity also, of the law a change doth come,
13 for he of whom these things are said in another tribe hath had part, of whom no one gave attendance at the altar,
14 for [it is] evident that out of Judah hath arisen our Lord, in regard to which tribe Moses spake nothing concerning priesthood.
15 And it is yet more abundantly most evident, if according to the similitude of Melchisedek there doth arise another priest,
16 who came not according to the law of a fleshly command, but according to the power of an endless life,
17 for He doth testify -- `Thou [art] a priest -- to the age, according to the order of Melchisedek;'
18 for a disannulling indeed doth come of the command going before because of its weakness, and unprofitableness,
19 (for nothing did the law perfect) and the bringing in of a better hope, through which we draw nigh to God." (YLT)
For when his cousin John, who was of the line of Aaron (Luke 1:5) baptized / immersed Christ in the river Jordan (Matt. 3:13-17), with the Holy Spirit ascending as a dove, Christ was annointed by a Levitical priest / prophet and fulfilled the prophesy of Jacob given so long ago in Gen 49:10.
"The sceptre turneth not aside from Judah, And a lawgiver from between his feet, Till his Seed come; And his [is] the obedience of peoples." (YLT)
So the true Levitical priesthood represented by John - as opposed to the false priests appointed under Roman rule - annointed the new High Priest after the order of Melchisedek signifying the change from the law of flesh to the law of the spirit; from the law of the old fleshly covenant to the law of the new covenant.
Just as Malachi had spoken of the priests who had corrupted the covenant of Levi (Mal. 2: 4, 5, 8), the priests under Herod were not of the Levitcal line, and could not have anointed Christ. Only John was appointed by God to perform that service.
A very interesting article on the transfer of the priesthood from Levi to Christ can be read TheTransferofThePriesthood.