One of the two great themes of the book of John is that those who believe can have life in Jesus’ name. (John 20:31) In John 14-17 Jesus revealed the magnitude of this teaching, showing that he desires people to be one with him, just as he is one with the Father. In the next few verses after John 14:2, Jesus tried to emphasize that he was going to the Father. He did not emphasize that he was going to heaven (although by inference that is clear). He specifically emphasized that we was going to the Father, to be in the Father. (John 14:20)
Jesus started chapter 14 by talking about dwelling places. He carried that concept forward through the rest of John 14 and 15. In verse 23 he says, “If anyone loves Me, he will keep My word; and My Father will love him, and We will come to him and make Our abode with him.” This is where the Father dwells, where his house is – in those who love him and keep his word. Then again in chapter 15:4-5 Jesus said, "Abide in Me, and I in you. As the branch cannot bear fruit of itself unless it abides in the vine, so neither can you unless you abide in Me. I am the vine, you are the branches; he who abides in Me and I in him, he bears much fruit, for apart from Me you can do nothing.” These are the dwelling places Jesus was talking about – abiding in him.
Jesus finally concluded his discourse with a prayer, saying, “I do not ask on behalf of these [the disciples] alone, but for those also who believe in Me through their word; that they may all be one; even as You, Father, are in Me and I in You, that they also may be in Us, so that the world may believe that You sent Me.” Note that Jesus was not here praying for unity of believers with one another, as important as that is. He was praying for unity of believers with himself. He continued, “The glory which You have given Me I have given to them, that they may be one, just as We are one; I in them and You in Me, that they may be perfected in unity, so that the world may know that You sent Me, and loved them, even as You have loved Me.” (John 17:20-23) Jesus wants believers to be one with him as he is one with the Father. So when Jesus was talking about his Father’s house, he was talking about his own body. (John 2:21) He was talking about his Spirit dwelling in the hearts of those who believe in him.