Alas, generally speaking, single-page applications do not work well with the back button; as you might have noticed, when you right click a button to open in a new tab, that also does not work as intended.
There probably is a solution involving JavaScript that you can put into an HTML snippet on the pop-up page, but then you are moving away from the power of low-code/Mendix. Do check out the link, and let me know if you managed to make this work.
Best regards,
Wouter