2 Samuel 8:18 says: "...and David's sons were priests."
How could David's sons be priests if he was a Judahite and priests were Levites?
Excellent question. Let's explore some explanations.
1) The first explanation is simply that they were indeed unlawful priests (c.f. Judges 17).
2) That the text would mention this transgression without consequence seems strange to many commentators who propose a second explanation - that the word "priest" here means "advisor". Let's examine a textual basis for this claim:
Samuel II 20:25-26
Sheva was secretary; Zadok and Abiathar were priests; and Ira the Jairite was David's priest
This is strange wording because all three priests should have been mentioned in one breath. Moreover, what does it mean that the first two were "priests" while the latter was "David's priest" (the job is not a personal one)?
Targum Jonathan indeed translates the first use as regular "priests" but the second use as "chief". So too many commentators explain that he was an advisor to the king. Indeed most of these commentators use the same explanation for David's sons. Targum Jonathan there too translates as "chiefs".
This sits well with Chronicles I 18:17, which is a match for our verse, although reads
Zadok son of Ahitub and Ahimelek son of Abiathar were priests; Shavsha was secretary; Benaiah son of Jehoiada was over the Kerethites and Pelethites; and David’s sons were chief officials at the king’s side.
A clause in Isaiah 61:10 reads כחתן יכהן פאר which uses the same word "priest" as a verb, but is explained by some commentators there (see Metzudot) to simply mean "make grand". Again, the root "priest" is used as a term of grandeur and position, rather than specifically tribal priest.
3) A third explanation is that they were in charge of the priests. This resolves the conflict and has a textual basis. Our verse in Samuel II 8:18 verse reads
וּבְנָיָ֙הוּ֙ בֶּן־יְהֹ֣ויָדָ֔ע וְהַכְּרֵתִ֖י וְהַפְּלֵתִ֑י וּבְנֵ֥י דָוִ֖ד כֹּהֲנִ֥ים הָיֽוּ׃
The first clause is translated as "and Benaiah the son of Jehoiada was over both the Cherethites and the Pelethites;" The term "over" isn't mentioned in the Hebrew, but is implied by the context - all previous verses in this chapter mention that Mr. X was over (i.e. in charge of) Y. So the second clause can be read in the same way - and David's sons were over the priests. I.e. They were administrators. Managerial staff for the priestly office.
This textual understanding is due to Prof. Yehuda Elitzur.
4) There are other explanations that I don't think warrant much explanation, although I'll mention them in passing. For example, that similar to Judges 17, at first they were indeed unlawful priests, and then later they were replaced by real priests and became government officials (that settles the difference between Chronicles and Samuel, they discussed different time periods). Another explanation is that they were priests in only part of the sense of the word. Not that they performed priestly duties, but that they were judges and legal adjudicators as per Deuteronomy 17.
I am working a thesis right now on 1-2 Samuel and the role of the Davidic covenant in moving forward the narrative. This verse may play into a larger theme of a Davidic priesthood that is alluded to in the book and a few other places in the OT (namely, Psalm 110, though possibly also Zechariah).
In 1 Sam. 2:27-35, an unnamed man of God comes to Eli and prophesies not only the end of his priesthood but also its replacement by a priesthood who would be faithful, do according to what was in God's heart and mind, and who would go in and out before God's anointed for the duration of his life (my translation), and who would receive a "sure house."
Most commentators suggest this is fulfilled in Zadok (see 1 Kings and Chronicles), yet Samuel never makes this connection--a Zadokian priesthood receives no explicit attention. Others identify Samuel as this priest, yet his children are rejected and cannot be said to have been given a 'sure house'. Samuel also dies before Saul and David. The author of Samuel makes a deliberate effort throughout the book, however, to paint David in priestly terms (wears an ephod, intercedes for the people, makes sacrifices, uses the urim and thummim (possibly)).
Most significantly, the only other occurrence of "sure house" is in 2 Sam. 7, referring to God's covenant with David. In light of this, that David's sons were priests might be a intended to connect this prophecy in 1 Sam. 2 with its fulfillment in David and his sons (Solomon in 1 Kings also performs a few priestly tasks, but it is more ambiguous). The above explanations are reasonable as well.
Saul's was removed from being King because he performed priestly duties intended for only Priests ( Levi's sons). And at a higher plane, represented the law. Because of the law, people always run short of God's glory. David was a prophet, King and a priest. He represented Grace and truth. He represented the Christ, and the Higher and Superior level of divineness. Christ then come, not from Levi, but from David's proesthood. Which is superior to the first. David played a Christ-like role. His ordainment was from the Father, rather pure and more divine than the blood lineage of the sons of Levi. Though by blood it is good, spiritual priesthood offered more salvation for all people.