Michael,
You should not open the form in popup but In content. At least that is what I always use. Note that this way you leave the Mendix application.
Regards,
Ronald
[EDIT]
In the URL redirector widget you can set the properties to open the URL in a new browser page or in the same browser page. So if you want to your Mendix application to stay open use the new browser page option. And closing the form with the microflow timer widget does work for me when the page is in content. Have not tried in a popup but this should work in my opinion.
Had the same issue. This solved my problem:
On the new form that is called by the microflow, you can place a HTMLSnippet widget. Set the Content Type to Javascript and add the content:
$(".someclassname").css("display", "none")
Also add the class someclassname to the form that is called by the microflow.