John the Baptist knew the Hebraic prophecies about a forerunner to the promised Messiah. Some think he did not know what was said in Luke 1:17 (by an angel to his father, before he was even conceived.) However, given the way the Holy Spirit was 'with him' even while he was in Elizabeth's womb, with her obedience to the command not to drink alcohol, which had to be conveyed to him so he also would not drink alcohol, it's perfectly reasonable to suppose that all of the angel's words were told to him. And John's prophetic life-style, and what he said about one coming after him who would baptise with the Holy Spirit, show that he knew he was preparing Israel for the coming Messiah. But it was not until Jesus presented himself to John for baptism at the Jordan river the next day after he'd denied those questions, that John then identified this Messiah.
John knew what was said in Malachi 3:1 and 4:5 about a messenger, and a messenger of the covenant, and the coming of 'Elijah the prophet'. He knew he was not the messenger of the covenant (Messiah), nor was he Elijah (who was bone-dead); John pointed to Jesus as the one who was superior to him, for that one had come before him. John knew he was merely the messenger. But there's more to this query about "the prophet".
It was the Messiah, Jesus, who was "the prophet like Moses" that John's questioners had in mind when they asked if John was "the prophet". Consider Deuteronomy 18:15-19, which Peter says in Acts 3:22-26 was fulfilled in Jesus, not John the Baptist. Jesus was the prophet Moses pointed to, anyone not listening to that future prophet to be cut off.
This means that John rightly denied being the Messiah, the messenger of the covenant; he rightly denied being Elijah (for he knew he was no reincarnation of that ancient prophet); he rightly denied being "the prophet" foretold by Moses. He knew his role was to prepare the way for the Messiah, and that he had power from the Holy Spirit to do that. His authority to tell Israel to repent and to be baptized came from heaven, because that was a vital requirement for Israelites to receive Jesus as the Messiah. John knew his role would diminish once Christ appeared, while Christ's role would increase, and once he'd verbally identified Jesus as the Messiah the day after he'd denied being those three ones, immediately two of his own disciples left him to follow Jesus.
To clinch it, Jesus himself identified John the Baptist:
"This is he, of whom it is written, Behold I send my messenger
before thy face, which shall prepare thy way before thee." Luke 7:27
A.V. [Bold mine]
John's negative answers were all true. He was not the messenger of the covenant (that was Jesus); he was not Elijah (who was just a collection of bones); he was not the prophet like Moses (that was Jesus). He was merely God's messenger, sent to prepare the way before the messenger of the covenant, Jesus. Now, had John's questioners asked him if he was the messenger of Malachi, he might have said "Yes, that's who I am - a mere messenger."
P.S. you put as a quote, "‘Look, I am sending my messenger (Elijah) ahead of you, who will prepare your way before you.’ Luke 7:26-27 NET." But the word 'Elijah' is not in any Greek text. It can be misleading to think that 'my messenger' should read, 'Elijah'.
Further, Jesus spoke to an audience about J the B; "And if ye will receive it, this is Elias, which was to come" Mat.11:14. Then in John 5:46 he said, "For had ye believed Moses, ye would have believed me: for he wrote of me." 16 verses on, when an audience saw his miracles, they said, "This is of a truth that prophet that should come into the world." John the B came in the power and spirit of Elijah, in a prophetic role, and was a messenger ahead of the Messenger of the Covenant, the Messiah, who was "that Prophet" Moses foretold. The relevant Hebrew prophecies speak of two covenants, two messengers, and two particular prophets: John the Baptist was never in the new covenant as he died before Jesus inaugurated it; Jesus was the messenger of the covenant. John was A messenger. Jesus was the supreme prophet, with John the Baptist foretelling his coming, making him A prophet, but not THAT prophet Moses foretold.
Once these distinctions are clear, then John the Baptist answered those 3 questions honestly. But those who did not understand the scriptures clearly would be puzzled - until they discovered the supreme role of the Christ. Then it would all make sense.