How to move file documents from mendix cloud to aws s3?

2
Hello,  We're trying to set up our file storage for a mendix app on amazon s3 and managed to configure this on a local mendix app with the custom settings. Now we would like to transfer all existing filedocuments from the mendix cloud to AWS S3. We're trying to figure out what the steps are to achieve this and would like to receive some help. Our Mendix app is in mendix 6.9.1 and we currently have almost 200GB (POD) of filedocuments in our production environment, so we need to have a robust and fast way to transfer all the documents.  How can we transfer all exiting file documents from the mendix cloud v3 to amazon s3? Do we need to build a process to migrate all our existing filedocuments or can we use a backup and restore process for this purpose? Is there migration path to transfer this huge amount?
asked
3 answers
5

You can do this, but please realize that you won't have any backups in this scenario. The Mendix Cloud v3 only backs up the database and local file storage. Also, restoring data to the test/accp environments will not work anymore. If a FileDocument is deleted, it will actually be gone with no means of recovery, so I advise against it.

 

edit: here is a python script that moves filedocuments from local filesystem to s3, no guarantees, you'll need s3cmd installed: https://gist.github.com/jtwaleson/5c00642165655febfaaf4c6b2867055a

answered
2

Hi Samet, I would suggest to build an asynchronous solution with the processqueue. Keep track of the files you successfully uploaded to a AWS bucket(use the AWS connector from the app store) and start with your oldest file. 

Maybe the structure of the filebackup zip is also useful but you have to create a script you run from AWS to download that backup zip and extract. 

Let us know what worked for you :)

 

--edit: based on the comments; we need some inside Mendix help .. maybe a support ticket helps Samet?

answered
0

Is there any solution to the addressed problem?

answered