Authority was very important to the men of the Sanhedrin, for they had the duty of calling prophets to account should there be error suspected in their activity. They considered themselves to be superior to those who they deemed to be speaking, or acting, without their idea of authority. And Jesus of Nazareth was certainly one whom they felt challenged their authority. This is where Matthew Henry's Commentary gives helpful details:
"They came to him when he was walking in the temple, not for his
diversion, but teaching the people, first one company and then
another. The Peripatetic philosophers were so called from the custom
they had of walking when they taught. The cloisters, or piazzas, in
the courts of the temple, were fitted for this purpose. The great men
were vexed to see him followed and heard with attention, and therefore
came to him with solemnity, and did as it were arraign him at the bar with this question, By what authority doest thou these things?"
[page 1435, 3rd column]
Now, that was a good question, and one they were entitled to ask. The trouble was, Jesus had already furnished them with proof a'plenty as to having authority from God in heaven, no less. He had performed miracles based on having forgiven peoples' sins, and the miracles happened. In front of their eyes. They knew that only God could forgive sins, so that if this Jesus of Nazareth falsely claimed to have authority to forgive men's sins, the miracles would not have followed his words of authority.
Because they could not fault the miracles, they hoped to trap Jesus with words. Now, Jesus knew what was in their hearts even as they asked that question. Because they had no excuse to continue disbelieving his God-ordained ministry, he used words to trip them up. He batted the ball straight back into their court. By asking them what authority they deemed John the Baptist to have had, Jesus knew they were defeated. They dare not reply one way or another and - defeated by the skill of Jesus' question - said "We do not know." Henry makes this telling point:
"Our Saviour intimates how near akin his doctrine and baptism were to
John's; they had the same original, and same design and tendency - to
introduce the gospel kingdom... 'Why did ye not believe [John] and
receive his baptism?' They could not bear that Christ should say
this, but they could bear it that their own consciences should say so,
because they had an art of stifling and silencing them... If they say,
'It is of men', he was not sent of God, but his doctrine and baptism were inventions of his own,' they expose themselves, the people will
be ready to do them a mischief... for all men counted John that he was
a prophet indeed, and therefore they could not bear that he should be
reflected on. ...They were confounded and baffled, and forced to make
a dishonourable retreat; to pretend ignorance (mortification enough to
those proud men)." [page 1436 1st column]
What Jesus did there was to "put to silence the ignorance of foolish men" (1 Pet. 2:15). They were dishonoured, while Christ remained honoured. Those men did not deserve to be answered because they were not searching for truth, but to attack Christ publicly, in the eyes of others. (He was already the object of their attack, in their own eyes.) Jesus defeated them with one honest question. Oh, yes, the questioners got his point all right. But nothing would ever get them exclaiming that yes, Jesus' ministry and authority was from God in heaven, any more than they would ever state publicly that John the Baptist's ministry and authority was also heavenly. Their stubborn disbelief in that source of authority remained, as proven a few days later by their manipulations to get Jesus put to death, just as those who hated John the Baptist's honest words got him killed.