When you test you always have to set the lognode for SAML to trace. Otherwise you only see a generic error and not the real problem. And I always test with Firefox with the SAML tracer plugin. This way you can follow the handshakes etc.
And in the last screenshot I can not see if the principal has been found. If that is empty the user could not be found in Mendix.
Regards,
Ronald