While before the 20th century there was common agreement on common authorship between the Gospel and Epistles of John, there is, as you mention, no such agreement today. At the same time, we are quick to note, however, that John and 1 John share a vocabulary of words and thought forms to such an extent that no one has mounted a serious proposal that they are independent works.
There have certainly been attempts to quantify the unity of these works. Vern Sheridan Poythress, for instance, wrote articles for Novum Testamentum and the Westminster Theological Journal purporting to determine Johannine authorship on the basis of a statistical study of conjunctions. There is an excerpt available online.
Craig Keener notes that others, such as Schnelle, "who argue against common authorship note that some key Gospel words (such as Scripture, glory, seek, judge, lord, law) are missing from 1 John, and terms in 1 John (such as antichrist, hope, sacrifice, fellowship, and anointing) are missing from the Gospel."1
Thoughts Not Words
Frankly I find neither of these studies nor others of their ilk very convincing. They seem strangely aloof of the way words are actually used and texts actually written. They would argue that "The Problem of Pain" could not have common authorship with "Screwtape Letters" because it lacks certain key words or phrases such as "Dear Wormwood" or "Our Father Below". Or they presume that an author would never self-consciously adopt a particular literary style like children's fairy tales (or perhaps the apocalyptic/prophetic style of Daniel). Likewise I know from personal experience that I choose different generic pronouns depending on various whims.
Similarly, studies that compute literary dependence or common authorship based on raw word vocabulary overlap, to my mind, fall short. We see that "father" is used 1200+ times in the Bible. "Son" is used over 3000 times. The verbs for "send" or "sent" appear over 900 times. Even the word "world" appears over 250 times. But the thought form of the "Father" "sending" the "Son" into the "world" is unique to John (e.g. 3:16 and many others) and 1 John (4:9-14).2
I distinguish this "vocabulary of thought forms" from theology. No doubt the synoptic writers would agree with John's theology that Jesus' mission is at the behest of God. But they say it and show it in different terms and stories: the overshadowing of Mary by the Holy Spirit, the descent of the Spirit like a dove at Jesus' baptism, or in parables like the parable of the tenants.
Besides the aforementioned "Sending", a number of other thought forms link the Gospel with the Epistle:
The Testimony of what is Seen and Heard
The Completeness of Joy in Fellowship
The connection of Love and (New) Commands
The joint Fellowship with the Father and the Son
The antitheses of Light/Darkness, Above/Below, Life/Death
The World and its Hatred (of God, of the Son, and of his disciples)
The Laying Down of One's Life
I won't take the time to go through each of these, and no doubt more examples could be added. But their combination already suggests a strong dependence of some sort between the two works. These collections of thoughts are of course marked by similar vocabulary. But as with the "sending" example, it has to do with more than their mere words. They are a way of thinking. And the more the collections of thoughts in the Gospel and Epistle begin to cohere the more we might say they are "of the same mind."
This gets us close to being able to say something about common authorship. But there are other stories we could tell here. For instance, it's natural for a student to develop a close mind with his/her teacher. I remember reading one review of Köstenberger's BECNT commentary on the Gospel of John that said one might as well just use Carson's commentary because Köstenberger (who studied under Carson) says largely the same thing.
So what we could have here is two people of a close mind (or perhaps some sort of "school of thought"), such that it's hard to tell with such a small sample of writing whether these are different authors after all, who worked closely together. Of course it's best not to multiply entities needlessly, so at this point our preference would be for a single author.
Carson and Moo in their Introduction to the New Testament elaborate three main reasons given by some for the "need" then to multiply entities.
Some (and here we veer into theology) think that there are key thematic or theological differences that cannot be reconciled. Particularly, the eschatology of the Fourth Gospel and 1 John are thought too different. For example, the Anchor Yale Bible Dictionary elaborates:
The similarities between the Fourth Gospel and 1 John are the most impressive. But the differences are so significant as to weigh against the tradition that equates the fourth evangelist with the author of 1 John. For instance, the careful articulation of the present and future eschatologies of the Fourth Gospel is entirely lost in 1 Jn with its view of the imminent end (cf. G.5 below). It might be argued, however, that the evangelist wrote 1 Jn at a time when the author embraced a different eschatology.
A simpler solution seems to be that the author of 1 John was a student of the Fourth Gospel and lived in a community which cherished that gospel as its primary tradition. The community had, however, undergone changes since the writing of the gospel, so that its views were no longer identical to those articulated by the fourth evangelist. In particular it may be that the differences between 1 John and the Gospel of John could be the result of the influence of other Christian traditions on the community.3
He also lists a number of other differences between 1 John and the Gospel of John:
Examples of the uniqueness of 1 Jn when compared with the Fourth Gospel include the following themes: imminent “last hour” (2:18), expiation (2:1; 4:10), anointing of believers (2:20, 27), lust (2:16–17), antichrists (2:18; 22; 4:3), lawlessness (3:4), false prophets (4:1), spirit of error (4:6), day of judgment (4:17), mortal and nonmortal sins (5:16–17), and ethical considerations (3:4; 4:20).
Those (like Carson and Moo) who see the Epistle and Gospel as being from the same pen tend to downplay these differences. They view the eschatological differences as real, but see them as complementary rather than contradictory viewpoints.
As I mentioned above per Keener, another tack has been to list key vocabulary and terms missing in one text or the other. Carson and Moo state though that, "Today most scholars acknowledge that nothing decisive can be based on these lists. The divergent vocabularies enjoy greater similarity than those of, say, Luke and Acts, known to come from the same pen."4
Finally, the third strand of thought Carson and Moo mention is hinted at in the quote from the AYB above: that of a Johannine school or community. This is inferred at times from the "we" passages that testity to eye witness in the prologues of both the Gospel and First Epistle. There is not room to elaborate this topic here, but this theory (at least according to Köstenberger) seems to be on the wane in Johannine studies.5
My own view, then, is that they are penned by the same author, recognizing others may find the types of evidence offered above more compelling.
Keener, C. S. (2012). The Gospel of John: A Commentary & 2 (Vol. 1, p. 124). Grand Rapids, MI: Baker Academic.
Galatians 4:4 comes close here, but the form of the thought is different. Paul's statement is historical ("born of a woman") whereas in the Johannine writings, the sending idea is allowed to remain almost abstract.
Kysar, R. (1992). John, Epistles of. In (D. N. Freedman, Ed.)The Anchor Yale Bible Dictionary. New York: Doubleday.
Carson, D. A.; Moo, Douglas J. (2009-05-12). An Introduction to the New Testament (Kindle Locations 17293-17294). Zondervan. Kindle Edition.
Köstenberger, A. (2009). A Theology of John's Gospel and Letters. Grand Rapids, MI: Zondervan. See pages 55-59 for more detail.