YOu need to import the SSL on mendix server.
Alternatively, Try using secure database from cloud like azure or aws and see if facing the same issue.
Please try to use custom configuration under custom tab as I was also facing the same issue but with this configuration it started working for me.
Name: DatabaseJdbcUrl
Value: jdbc:sqlserver://localhost:1434;databaseName=MendixDB;integratedSecurity=false;encrypt=true;trustServerCertificate=true