Question

Cannot view or download files using CF amazon S3 storage

1

We configured a CF IBM bleumix environment using Amazon S# as file storage an Elephant DB as a database. We configure the environment variables and deployed the application on CF. Everything seems to work fine. We can login, enter data, upload files (which appear on the S# storage, all in root), etc.. The problem is that we cannot download or view uploaded files When viewing files using an image viewer we get a brokenfile image, when downloading the file a html file is returned (e.g. test.jpg.html). Opening this file returns a link error. Is this a bug or a configuration issue?

asked 2016-07-19

René Smit

5 answers

Remco Zomer · Answer 1 · 2016-07-22

Probably found the solution :-)

It is relevant to choose for bucket regions that support AWS Auth V2 !! Frankfurt is the only location (which I used as storage location) supporting V4 Authentication and is not backwards compatible which breaks downloading the file correctly. If you select another region like Ireland (supporting Amazon V2 and V4 authentification) the expense-app of the appstore behaves well related to files!

Remco Zomer · Answer 2 · 2016-07-22

FOR THE ARCHIVE, LESSONS LEARNED WITH S3 IF MOVING EXISTING MENDIX APPLICATION TO CLOUD FOUNDRY

1) all files in S3 need to be in root (so not in a tree structure with folders)

2) select an S3 bucket location supporting Amazon V2 authentification (currently not Frankfurt)

3) Connect a (read/write) policy to your S3 account / or bucket in Amazon

Remco Zomer · Answer 3 · 2016-07-22

In Amazon S3 we have a tech-user role, with 2 programmatic keys (acces key and secret access key) connected to a policy "AmazonS3FullAccess". I assume by this way read/write access to bucket(s) used is in place? In one of the spaces as trial we run the "company expenses" from the appstore. If I download an attachment connected to a task in this application nothing is added to the log. The file downloaded is not a binary (it was a picture, so binary expected) but the same-file name with "double extension" .jpg.html
Actually if you "open" this "html" its empty but has one error-description "'screenshot_01.jpg' could not be found.","result":560". Will see if I can check in the other application we test now in Bluemix if I can detect something (e.g. in log) related to pictures not shown by the image viewer (now "broken icons").

Remco Zomer · Answer 4 · 2016-07-22

Small update: in writing a file (attachment to a task) to Amazon using the Company-expenses from the Appstore I see that actually 3 files?? are entered in the bucket and the logs shows 2 errors:

1) com.amazonaws.services.s3.AmazonS3Client shouldRetryWithAuthParam 2) WARNING: Attempting to re-send the request to expenseadoptiq.s3-external-1.amazonaws.com with AWS V4 authentication. To avoid this warning in the future, please use region-specific endpoint to access buckets located in regions that require V4 signing.

Not sure if this has to do with issues related to downloading the file as next step and mentioned before.

Remco Zomer · Answer 5 · 2016-07-22

Small update now in testcase uploading files and showing it with image handler:

In writing the file same errors as mentioned before

in retrieving the image file actually 2 errors in the log: ERROR - Connector: The file 'logo_test.jpg' could not be found. 2016-07-22T12:18:31.620+0200[App/0]OUTERROR - Connector: com.amazonaws.services.s3.model.AmazonS3Exception: Bad Request (Service: Amazon S3; Status Code: 400; Error Code: 400 Bad Request; Request ID: 95C663463DA4DE19)