The most I was able to find from my preliminary research is a few lines in this book:
In true acrostics the alphabet used can vary: certain letters can be
omitted or transposed. Professor Williams Johnstone has argued that pe
replaced waw some stage in the Hebrew alphabet. Since Sirach 51 has
a waw verse and an extra pe verse at the end while Psalms 25 and
34 not have the waw verse (but do have the extra pe verse) then
the acrostic in Sirach 51 may mark a transitional stage and perhaps it
is older than the two Psalms.
Other sources simple explain that the pe was added to complete the number of 22, since the letter waw was omitted from the list (for reasons unknown), to compensate for this loss the letter pe was added in the end (perhaps because it was thought to be the replacement letter of the waw) to complete the 22 letters that were customary in the acrostic Psalms. For the significance of the number 22 see here.
The idea espoused by Johnstone that Sirach 51 marks a transitional stage is pretty audacious, since that would mean that Psalms 25 and 34 were composed no earlier than 200 BC, the time Sirach is thought to have been written down, and at that time the LXX may have already been completed. It is more likely that Sirach 51 marks a later development of the pe-waw replacement method, in which it became customary to end any acrostic psalm with a pe, even when a replacement for the waw wasn't anymore necessary. So although it started off as a replacement method for the letter waw as prof. Johnstone contends, at the time Sirach was composed this tradition morphed into something else and took on a life of its own. Perhaps at that point the letter pe came to symbolize something sacred, and had some other meaning that was completely lost to us.