This is indeed a fascinating and cryptic statement... As was pointed out in a comment on the question, this phrase actually occurs twice: earlier, as Elijah was ascending to heaven, Elisha sees "a chariot of fire and horses of fire" (II Kings 2:11), and then cries out, "My father, my father! The chariot[s] of Israel and its horsemen!". Biblical scholars and commentators have different opinions about the symbolism of this phrase.
Many (like the Aramaic Targum cited in another answer) understand this as a reference to the prophet [Elijah] being more important to Israel, especially in its military defense, than chariots and horsemen. Interestingly, a few chapters later (II Kings 6:13-18) Elisha is surrounded by the enemy forces of the king of Aram, including horses and chariots; Elisha's servant/helper fears, but Elisha prays and the youth's eyes are opened and he sees that Elisha is surrounded by "horses and chariots of fire". The Dutch Biblical scholar Martinus Adrianus Beek writes that this story illustrates the theme of the prophetic message: "'those who are on our side are more than those on theirs' [II Kings 6:16]... Therefore every prophet has a right to the title rekhev yisrael upharashaw [chariot and horsemen of Israel]." Beek then connects this to the story of the Exodus from Egypt and the Song of the Sea, where the words "chariot" and "horsemen" also feature prominently, to suggest that this phrase emphasizes the role of the prophet in focusing the people's devotion and trust to G!d rather than Egypt/military might/political alliances. Cf. the passages Beek brings: II Kings 18:21-24, Isaiah 31:1-3, Psalms 20:7, and others, that demonstrate that this title for the prophet represents the direct contrast between the physical horses and chariots of Pharaoh and other mortals versus the word of the prophet, the carrier of G!d's will.
Note: Beek's article, "The Meaning of the Expression 'The Chariots and the Horsemen of Israel'" can be read online here.
Similarly, medieval Jewish commentators (like Rashi, Abarbanel, and Gersonides), understand this as referring directly to the figure of the prophet (Elijah or Elisha) as being the symbolic "chariot and horsemen of Israel". Biblical scholar Joseph Blenkinsopp suggests that this draws on the imagery of YHWH as master charioteer of the hosts of heaven (cf. Psalms 68:5, 68:18), and thus reinforces the image of the prophet as carrier of the Divine word.
Thus, if this reading is correct, Jehoash is referring to Elisha and mourning the loss of the great prophet and spiritual defender of Israel, rather than what might appear to be an unrelated cry regarding Israel's army.