#StackBounty: #node.js #amazon-s3 #aws-sdk #aws-sdk-nodejs #wasabi-hot-cloud-storage Image corrupted from s3

Bounty: 50

This is my upload code (node)

// generate a uuid string
            const uuid = uuidv4(); // type string

            // upload to wasabi

            fs.readFile(file.tempFilePath, "utf8", (err, data) => {
                if (err) {
                    throw (err);
                }
                // upload the object
                s3.upload(
                    {
                        Body: data,
                        Key: uuid + "-" + file.name,
                        Bucket: "files",
                        Metadata: { 'ext': ".png" },
                        // ContentLength: file.size
                    }, (err, data) => {
                        if (err) {
                            console.log(err);
                            throw (err);
                        }
                        console.log(data); 
                });
            });

The object actually does get to the bucket. It has more than 0 bytes. It is about half as small the source image but I’m just assuming someone is compressing it (wasabi or something).
I am using express-fileupload and my config for it is:


app.use(fileUpload({

    createParentPath: true,

    limits: {
        fileSize: 2 * 1024 * 1024 * 1024 //2MB max file(s) size
    },

    useTempFiles: true,

    tempFileDir: "/tmp/",

}));

Eventually I download this directly from the wasabi web client, and try to open it in an image viewer and they all say there is an error in the file or that they don’t support the file type (which is .png, so it is supported).
The image is only ~150kb .
Essentially the file gets there but its corrupted.


Get this bounty!!!

#StackBounty: #amazon-web-services #aws-sdk #aws-batch #aws-sdk-cpp Why can I not override container variables for multi-node parallel …

Bounty: 250

I am working with AWS Batch. My goal is to create a multi-node parallel job through the AWS SDK for C++. For this, I have created a job definition as per the instructions here.

I am working with the AWS C++ SDK, and I noticed that when I try to override either environmental variables or commands, nothing is actually transferred to the job.

Interestingly, the same code works perfectly fine for a job which uses a normal job definition (as opposed to a multi-node one):

#include <aws/batch/BatchClient.h>
#include <aws/batch/model/ContainerOverrides.h>
#include <aws/batch/model/KeyValuePair.h>
#include <aws/batch/model/SubmitJobRequest.h>
#include <aws/core/Aws.h>
#include <aws/core/utils/Outcome.h>

int main(void)
{
    Aws::SDKOptions options;
    Aws::InitAPI(options);

    Aws::Batch::BatchClient batchClient;
    Aws::Batch::Model::SubmitJobRequest submitJobRequest;
    Aws::Batch::Model::SubmitJobOutcome submitJobOutcome;
    Aws::Batch::Model::ContainerOverrides containerOverrides;
    Aws::Batch::Model::KeyValuePair envVariable;

    envVariable.SetName("foo");
    envVariable.SetValue("bar");

    containerOverrides.AddEnvironment(envVariable); // This does nothing for a multi-node job definition.
    containerOverrides.AddCommand("foobarbaz"); // This does nothing for a multi-node job definition.

    submitJobRequest.SetJobName("myjob");
    submitJobRequest.SetJobDefinition("arn:aws:...."); // This string is an example. I have used the actual job definition ARN.
    submitJobRequest.SetJobQueue("arn:aws:...."); // This string is an exmaple. I have used the actual queue ARN.
    submitJobRequest.SetContainerOverrides(containerOverrides);

    submitJobOutcome = batchClient.SubmitJob(submitJobRequest);

    Aws::ShutdownAPI(options);

    return 0;

}

Should I be using a different API for multi-node parallel jobs?


Get this bounty!!!

#StackBounty: #php #amazon-web-services #aws-sdk #amazon-cognito MissingAuthenticationTokenException ("Missing Authentication Toke…

Bounty: 50

I have a working implementation of the AWS PHP SDK. Operations like $client->getUser() are working, but $client->adminCreateUser() and others are not working.

When I call $client->adminCreateUser([...]), it results in:

Error executing "AdminCreateUser" on "https://cognito-idp.ap-southeast-2.amazonaws.com"; AWS HTTP error: Client error: `POST https://cognito-idp.ap-southeast-2.amazonaws.com` resulted in a `400 Bad Request` response:
{"__type":"MissingAuthenticationTokenException","message":"Missing Authentication Token"}
 MissingAuthenticationTokenException (client): Missing Authentication Token - {"__type":"MissingAuthenticationTokenException","message":"Missing Authentication Token"}

Line 191 in /var/www/project/vendor/aws/aws-sdk-php/src/WrappedHttpHandler.php

Similar services evoked from CLI (e.g cognito-idp admin-create-user) with the exact same credentials are working.

What is causing this?


Example Details

My environment:

  • Ubuntu 18.04
  • Apache 2.4.29
  • PHP 7.3
  • aws/aws-sdk-php 3.92.3

.aws/credentials

[default]
aws_access_key_id=XXXX
aws_secret_access_key=XXXX

I am using my developer credentials

Example code:

$client = new CognitoIdentityProviderClient([
    'version' => 'latest',
    'region' => 'ap-southeast-2',
    'credentials' => false, // Set to false to allow roles provisioned to our EC2 instances
]);

$result = $client->adminCreateUser([
    'DesiredDeliveryMediums' => ['Email'],
    'MessageAction' => 'RESEND',
    'TemporaryPassword' => 'TemporaryPassword1234',
    'UserAttributes' => [
        ['Name' => 'email', 'Value' => 'mailbox@domain.tld'],
    ],
    'UserPoolId' => 'ap-southeast-2_XXXX',
    'Username' => 'mailbox@domain.tld',
]);


Get this bounty!!!

#StackBounty: #android #amazon-s3 #retrofit #retrofit2 #aws-sdk Retrofit2-AWS S3 Multipart image File corrupt issue

Bounty: 50

I am using Retrofit2 for handling API calls in my Android application. The application contains several image upload features. All the image uploading features are wired up to two API calls and will fire one after another.

  1. The first API will upload the image content to the server and the server will generate a dynamic signed AWS S3 image URL as a response.

  2. The second API is using the above-signed URL and PUT the data into AWS S3.

It was working perfectly for small images. For large images, the API has failed. The possible solution for this issue is to change the upload API as multipart. So I have changed the second API as “Retrofit2-multipart“. But the problem here is, After uploading the image, the file got corrupted in the S3.

So my question is, Is retrofit multipart is fit for ASW S3 upload?
Do we have any solution to solve this issue?.

Please check my Implementation

Retrofit API Interface

 @Multipart
    @PUT
    Call<Void> uploadFile(@Url String url,
                          @Part MultipartBody.Part file);

Retrofit service creator

 private ApiServicesList createService() {
        OkHttpClient okHttpClient = new OkHttpClient.Builder()
                .readTimeout(300, TimeUnit.SECONDS)
                .connectTimeout(360, TimeUnit.SECONDS)
                .cache(null)
                .addInterceptor(new NetworkInterceptor(context))
                .addInterceptor(createLoggingInterceptor())
                .addInterceptor(createSessionExpiryInterceptor())
                .addInterceptor(createContextHeaderInterceptor())
                .build();

        return new Retrofit.Builder()
                .baseUrl(FirebaseConfig.getInstance().getStagingBaseURl())
                .addConverterFactory(GsonConverterFactory.create())
                .client(okHttpClient)
                .build()
                .create(ApiServicesList.class);
    }

File Upload Class

 public void uploadFileTos3(String url, MultipartBody.Part file, Callback<Void> listener) {
        Call<Void> call = mAPIServices.uploadFile(url,file);
        call.enqueue(listener);
    }

Presenter API Call

networkServices.uploadFileTos3(credentials.getSignedUrl(), prepareRequestBody(file), new Callback<Void>() {
            @Override
            public void onFailure(Call<Void> call, Throwable t) {
               //App Action
            }

            @Override
            public void
            onResponse(Call<Void> call, Response<Void> response) {
               //App Action
            }
        });

Request Body Method

 public static MultipartBody.Part prepareRequestBody(File file) {
        RequestBody requestFile = RequestBody.create(MediaType.parse(CONTENT_TYPE), file); // Checked Both "image/jpeg" and "multipart/form-data"
        return  MultipartBody.Part.createFormData("image", file.getName(), requestFile);
    }

DDMS LOGS

2019-04-10 19:07:13.926 18761-18874/com.xxx D/OkHttp: --> PUT https:Signed URL(Removed actual URL)
2019-04-10 19:07:13.926 18761-18874/com.xxx D/OkHttp: Content-Type: multipart/form-data; boundary=60561d1c-ff3f-4a43-8022-ca2be3e8ec4e
2019-04-10 19:07:13.926 18761-18874/com.xxx D/OkHttp: Content-Length: 71154
2019-04-10 19:07:13.931 18761-18874/com.xxx D/OkHttp: --60561d1c-ff3f-4a43-8022-ca2be3e8ec4e
2019-04-10 19:07:13.931 18761-18874/com.xxx D/OkHttp: Content-Disposition: form-data; name="image"; filename="forest-trees-fog-foggy.jpg"
2019-04-10 19:07:13.931 18761-18874/com.xxx D/OkHttp: Content-Type: image/jpeg
2019-04-10 19:07:13.931 18761-18874/com.xxx D/OkHttp: Content-Length: 70934
2019-04-10 19:07:13.931 18761-18874/com.xxx D/OkHttp: Image Body
2019-04-10 19:07:13.942 18761-18874/com.xxx D/OkHttp: --60561d1c-ff3f-4a43-8022-ca2be3e8ec4e--
2019-04-10 19:07:13.942 18761-18874/com.xxx D/OkHttp: --> END PUT (71154-byte body)
2019-04-10 19:07:15.648 18761-18874/com.xxx D/OkHttp: <-- 200 OK


Get this bounty!!!