Short Answer: There is no contradiction. John is not claiming that Jesus went into Galilee immediately after His baptism -- he is claiming that Jesus went into Galilee immediately after the day recorded in 1:29-34 -- a day in which John the Baptist testified to the Jews that he had already baptized Jesus some time prior. So, to answer the question directly, it would seem (based on what we know from Scripture) that Jesus went to Galilee at least 40 days after His baptism.
Let's examine carefully what John is saying (and why.) First, John testified via his Gospel...
. . . so that you may believe that Jesus is the Christ, the Son of God; and that believing you may have life in His name. -John 20:31
Now let's follow John's narrative flow from the opening verses in Chapter 1.
1-18 provides numerous testimonies about the true identity of Jesus as the Christ, the Son of God -- including John the Baptist's
19-28 has John the Baptist telling the Jews that his purpose is to testify about the true identity of the Christ
29-34 has John the Baptist actually testifying that Jesus is the Christ
35ff has John the Baptist's disciples responding to his testimony and following Jesus, the Christ
Before we move on to reconciling accounts, it is important to recognize that the reason John included this information was to convince his readers to believe that Jesus was the Christ, and he was using John the Baptist's testimony as evidence of this. He was not including this information for the purpose of chronicling Jesus' baptism; His readers already knew about that from the Synoptic tradition. (See here for more on that.)
Now let's take a look at the day recorded in 1:29-34.
The next day [John the Baptist] saw Jesus coming to him and said, “Behold, the Lamb of God who takes away the sin of the world! This is He on behalf of whom I said, ‘After me comes a Man who has a higher rank than I, for He existed before me.’ I did not recognize Him, but so that He might be manifested to Israel, I came baptizing in water.”
John testified saying, “I have seen the Spirit descending as a dove out of heaven, and He remained upon Him. I did not recognize Him, but He who sent me to baptize in water said to me, ‘He upon whom you see the Spirit descending and remaining upon Him, this is the One who baptizes in the Holy Spirit.’ I myself have seen, and have testified that this is the Son of God.”
In other words, John the Baptist saw Jesus approaching, and said: "Look! There He is! That's the Christ!" ...But he then goes on to explain that he did not recognize Him as the Christ prior to His baptism, because it wasn't until he baptized Him that he saw the Spirit descending on Him, and could thereby identify Him as the Christ.
So John the Baptist must have already baptized Jesus prior to saying that. How many days prior? I don't know, but it could have easily been 40 days prior.
So, in summary, there is no reason to think that this passage is evidence that John's Gospel is historically inaccurate, or inconsistent with the historical claims of the Synoptics.