Authority and tradition are often at odds with one another, particularly when a new--or seemingly new--authority bursts on the scene. Sometimes the new authority (and for our purposes here, let us assume the authority is a single person, such as Jesus) puts forward a new paradigm, which is simply a new way of looking at something.
Today, we say this person has a way of "thinking outside the box." In other words, he or she provides a "new set of eyes" to the scene, and sees things
Now tradition can certainly be a good thing in virtually any area of endeavor, be it science, technology, medicine, or religion, and progress in any field owes a great deal to specialists and authorities from previous generations who have laid the foundation for subsequent generations.
When Jesus burst on the scene in the first third of the first century A.D., Judaism, a respected and honorable religion for almost two thousand years (assuming it had its nascency in Abraham), had many noble traditions in place, based on the Hebrew Scripture. In synagogue, Jesus Himself read from the scroll containing the book of Isaiah, for example. Both He and the teachers and scribes from perhaps the two biggest "denominations" within Judaism, the Pharisees and the Sadducees, were conversant in the Law of Moses, the Prophets, and the Writings, the three-fold division of the Tanakh, or "Old Testament."
In Jesus' day, however, many non-biblical traditions had become part and parcel of Judaism, and the number of laws, or commandments, in the Law of Moses, which numbered 613, according to one tradition, had increased significantly to perhaps thousands! Some of these "new" commandments had a clear and non-controversial basis is scriptures; others did not. Evangelist Mark commented on some of the accretions of tradition which had attached themselves to some of the original 613 commandments contained in the Torah.
"The Pharisees and some of the scribes gathered around [Jesus] when they had come from Jerusalem, and had seen that some of His disciples were eating their bread with impure hands, that is, unwashed. (For the Pharisees and all the Jews do not eat unless they carefully wash their hands, thus observing the traditions of the elders; and when they come from the market place, they do not eat unless they cleanse themselves; and there are many other things which they have received in order to observe, such as the washing of cups and pitchers and copper pots" (Mark 7:1-4 NIV).
In a subsequent verse, Jesus remarked in the Pharisees' hearing,
"'You have let go of the commands of God and are holding on to the traditions of men" (Mark 7:8 NIV)
In other words, the basis of God's commands in the Law of Moses was no longer just the Torah but also non-biblical traditions (the accretions over time) which in some cases took the front seat and pushed the former to the back seat.
In His typically direct and authoritative way, with irony, Jesus also said,
"'Rightly did Isaiah prophesy of you hypocrites, as it is written: THIS PEOPLE HONORS ME WITH THEIR LIPS, BUT THEIR HEART IS FAR AWAY FROM ME. BUT IN VAIN DO THEY WORSHIP ME, TEACHING AS DOCTRINES THE PRECEPTS OF MEN. Neglecting the commandment of God, you hold to the tradition of men. . . . . . You are experts at setting aside the commandment of God in order to keep your tradition'" (Mark 7:6-9).
What a stinging indictment. Jesus then went on to illustrate the ways in which God's commands were trumped by the accretions of tradition.
As you indicate in your question, Jesus did not advocate that His fellow Jews ignore the teaching they received from the "clergy" of their day. No. Notice, however, what Jesus said in the second part of the verse you quoted (viz., Matthew 23:3b):
"'. . . but do not do according to their deeds, for they say things and do not do them.'"
Put differently, Jesus was saying, in effect, "Obey only those non-biblical commands of the scribes and Pharisee which they themselves obey, not the ones they do not."
In another famous teaching, Jesus said,
"'Woe to you, teachers of the law and Pharisees, you hypocrites! You give a tenth of your spices--mint, dill and cumin. But you have neglected the more important matters of the law--justice, mercy and faithfulness. You should have practiced the latter, without neglecting the former'" (Matthew 23:23 NIV; cf. Luke 11:42 NIV).
Notice that Jesus did not condemn their tithing; He condemned their neglect of weightier things, such as justice, mercy, and faithfulness.
Jesus illustrated how this neglect revealed itself:
"'. . . but you say, 'If a man says to his father or his mother, whatever I have that would help you is Corban (that is to say, given to God),' you no longer permit him to do anything for his father or his mother ; thus invalidating the word of God by your tradition which you have handed down; and you do many things such as that" (Mark 7:11-13).
In conclusion, the common people in Jesus' day detected an authority in Jesus which went well beyond the authoritativeness of the men who were supposedly their spiritual leaders, but whom Jesus roundly condemned as hypocrites.
What Jesus said made eminently good sense to the common people. His teaching, while not ignoring the good traditions rooted firmly in the Holy Book, concerned itself primarily with the weightier issues which undergird the letter of the Law; such things as love, compassion, mercy, inner purity, humility, integrity, and so much more.
Jesus' reliance on God's word and His marginalizing the non-biblical traditions which had grown up around God's word, impressed the common people, and initially they flocked to Him in droves. Only when they realized that the spirit of the Law sounds great in theory but is difficult to put into practice, did they then start to fall away and stop following Jesus (see, for example, John 6:66).