Then you have misconfigured Gluu. See the documentation here for open ID: https://gluu.org/docs/ce/admin-guide/openid-connect/ or here for SAML https://gluu.org/docs/ce/admin-guide/saml/
Look in the chart at step 4 for open ID or the part about IDP-initiated outbound SAML flow for SAML. The Gluu server should do the redirect back to your Mendix app.
TThis is the Idp configuration page where I have selected the MxModelReflection and SAML 2.0 I want to know whether I have done it correctly. Before doing this I have configured SP According to the process given as per the appstore documentation of the SAML 2.0 Module.
To configure the SP I have provided all the things according to the proccess given in the saml 2.0 of mendix app store.downloaded and uploaded the keystore SPMetaData.jks and the downloaded the sp_metadata.xml. Then coming to the configuration of idp Where I got stuck in providing the member name and type and want to know whether I am doing It correctly or not.