Yes. You have hit upon the metaphoric meaning of מגבר as both 'speaking' and 'wilderness'.
The wilderness is a secondary metaphor. Like 'Waterloo', place is used for a metaphor of an activity that takes place there.
An interesting study would be to find any outliers: are there accounts of someone going to the 'wilderness' and not speaking to God in the literal or sensus plenior?
Looking at the formation of 'midbar' מדבר: