Hi Bart,
I don't think this is related to mendix itself but the way your browser tries to render a web page and it's components. Try reducing the zoom levels in your browser 'Ctrl -' to 75-80% and try opening your pop-up again.
You can try various systems with different screen resolution.
If this is bugging you then I think reducing the height of your popup page can help. Share your findings and let us know! :)