Both
On the one hand, Matthew seems to link the fulfillment to the journey to Egypt. But the quotation from Hosea makes clear that the journey from Egypt (the Exodus) was what Hosea had in mind. (But see also: Is Hosea 11:1 referencing the initiation of the Exodus or the sojourn in Egypt?) Since Jesus was born in Judea, it would be necessary for him to travel to Egypt in order to be called out of that country.
Reading the rest of Hosea 11, you can see that the prophet is making a broad statement about Israel's rocky history with God. Hosea is warning the Northern Kingdom that Assyria is coming to carry out God's judgement against them. But that's not the end of the story:
They shall come trembling as a bird out of Egypt, and as a dove out of the land of Assyria; and I will make them to dwell in their houses, saith the LORD.—Hosea 11:11 (JPS)
In order for the nation to be returned, they must first be carried away from the land.
Joseph as a type for Christ
As I thought about the journey to Egypt and back, it occurred to me that Joseph's life is neatly paralleled by this story of Jesus' nativity. Both are:
- taken from Israel to Egypt in order to avoid being murdered, and are
- returned to Israel after a change in regime.
There are other similarities (the importance of dreams, both are first-born in a sense, etc.), but the key is that God sent them away in order to return them. In Joseph's case, he was returned as a mummy and in Jesus' case, he was returned while still a child. Matthew, I think, is hoping to draw a connection between Joseph's life and Jesus'.