Setting the compression quality to a lower value should help in reducing file size, but as I see from this post it does not always work https://community.mendix.com/link/space/app-development/questions/107786
If you need to ensure that the file size is always below a certain size you might need to implement a java action for this. A plain java implementation of this is explained here: https://www.youtube.com/watch?v=yQGjl675K7M
You will need to implement the code in a java action to use this in your app.