A literalistic reading of the aforementioned passage in Mathew teaches that under a certain circumstance, namely "your eye causes you to sin," one most poke out his or her own eye. A literary (non-literal) reading of the passage sees the usage of hyperbole and a vivid visual image to communicate the horrifying and traumatic nature of sin. The former reading is basically didactic, the latter reading sees a teaching that is implied and indirect.
“He seemed to be of the idea that the Scripture could be read either literally word for word from beginning to end or just as poetry or allegory”
The Bible clearly has didactic content, but it's unreasonable and certainly wrongheaded to reduce the Bible to a book about what a person should do and when. If the Bible was intended as such, it would have been written in legalese. The Bible contains narrative history, poetry, philosophy and law and is wrought with a pervasive tension between the literal meaning of its words and its literary effectiveness.
[In real life, many people are unable to appreciate nuance or tolerate complexity and are unable to see the power and effectiveness of tension, ambiguity and mystery in the human experience. No great literature is so simple that it can be packaged as either being literal or allegorical and that is what makes literature such a powerful and important medium of communication. The culture of bullet point summaries and bite sized content is antithetical to a deep appreciation of the Bible or any great art.]
Usually intuition is sufficient to determine if a text was intended more for its literal content or for its literary content. For example, it's overwhelmingly evident that Mathew 18:9 should not be taken literally since:
- A literal rendering of the passage would be grossly inconsistent with much of scriptures
- The passage fails to be meaningful or informative when taken in its literal sense - "if your eye causes you to sin" is far to vague and ambiguous and “tearing out one's eye” is far to intense to have any practical import
- The added “and throw it away” contributes nothing to the literal obligation in the verse but is consistent with and complementary to its vivid and descriptive style
- The context in Mathew isn't legalistic at all
Not all cases are so clear cut (see this question) and these issues are taken seriously and are well debated within religious circles. Nevertheless, a precise and rich understanding of the Bible, like any other important text, can only come through experience, study, and a learned sensitivity to the thematic and philosophical content therein.