There are different kinds of service being discussed. In Num 8.24:
those 50 and over will return from:
tseva ha avodath --> "the work of the service"
Whereas those 25 and older will:
tsevo tsavah ba avodath --> "serve the service in the work"
Traditional view
Rashi suggests that they are in fact studying for five years:
מבן חמש ועשרים FROM TWENTY AND FIVE [YEARS] AND UPWARD [THEY SHALL GO
IN TO WAIT UPON THE SERVICE] — But in another passage, it states,
(Numbers 4:3) “from thirty years old [and upwards … all that enter
into the host to do the work in the tabernacle]”! How are these
apparently contradictory passages to be reconciled? In the following
way: From twenty five years on he (the Levite) comes to learn the laws
regulating the service, and he studies for five years, and at the age
of thirty he may actually do the service. — From here we may derive
the principle that a pupil who does not see an auspicious omen (i. e.
success) in his study during five years will never see it. (Sifrei
Numbers 62; Chullin 24a.)
Rashi Commentaries. (n.d.). (pp. 573–574).
Of course by "study", this did not involve reading books (there were no books), but an apprenticeship relationship involving graduating when they reach the age of full adulthood, which is 30. That was the age needed to participate in the work of the service, but from age 25, Levites served (tseva) but did not work (avodath)
Assessing the traditional view
Was Rashi right? All we have are the text and the traditions associated to it. There are no other sources of information. But from the fact that different language is used, we can infer that it was a different type of service, and from the fact that 30 is the traditional age of adulthood, it would make sense that the responsibilities were contingent on age and adulthood status -- indeed it would be strange if this wasn't the case.
What is also nice about the traditional view is that
at the beginning there were no apprentices, because they were all apprentices. Only after a period of initial service so that the adults understood what their roles were did a second census take place and apprentices joined them.
between the time of the first census and the second census the offerings were made for the raw materials and the objects were fashioned. This involved collecting and distributing huge amounts of gold and silver, so it made sense that no minors were serving during this time.
Between the first census and right before the second census the Levites were consecrated. They did not take on any apprentices until after they themselves were consecrated. That makes sense, too.
So with all of these considerations, the traditional view fits nicely into the narrative. That doesn't mean it is inspired scripture, but at least it makes sense.
In terms of questions about military age and recruitment, the requirements of fighting and the requirements of serving in the tabernacle were not the same requirements, just as the age at which people were eligible to marry was even earlier than the age at which they were recruited to fight.