I need to upload ~500 jpeg images to S3, each image size is ~8MB. The requirements are uploading from browser, and the GUI should reflect the status of each image. The average upload speed of our users is 1MBS. My approach is requesting the server (node) for an array of presign URL's ,then upload from browser using http put. When I upload 1 - 3 images, all works fine, However attempting to upload more then 4 images have random results, some of the image are uploaded and some throw error of CORS. For those with the error, when I use the exact same url in postman it works fine, so there is a different error underneath.
The code:
Client (Angular):
initImagesUpload = (event) =>{ //<<<< result from html input file change
let imagesUrl = {
"images":[],
"userId": this.user.id
};
let lookforImageFile =[];
var formData = new FormData();
// gather the files names to an array
this.targetFiles = event.target.files;
for (let i = 0; i < event.target.files.length; i++) {
imagesUrl.images.push(event.target.files[i].name);
}
this.imagesService.postImagesforPresignURL(imagesUrl)
.subscribe(images =>{
this.imagesWithURL = images;
this.imagesWithURL.map((image,i)=>{
// image == {"image_name": "21.JPG","presign_url": "XXX"}
lookforImageFile = Object.values(this.targetFiles).filter((obj: any) => {return obj.name == image.image_name});
if(lookforImageFile.length > 0){
const p = new Promise((resolve,reject)=>{
formData = new FormData();
// asume there is only single image with the same name
formData.append("userfile",lookforImageFile[0]);
this.imagesService.UploadSingleImage(image.presign_url,formData).subscribe(uploadResult =>{
if(uploadResult.status == 200){
resolve(image.image_name + ' ' +uploadResult.status);
}
else{
reject(`error at image ${image.image_name}`)
}
});
});
p.then((status)=>{
setTimeout(function(){ console.log(`get following status ${status}`); }, 1000); // <<< a try, not sure if helps
console.log(status);
})
}
});
});
}
UploadSingleImage(image:any,formData:any): Observable<HttpResponse<any>>{
return this._http.put<any>(image ,formData,{observe: 'response'});
}
Server (Node):
GetAllImagesNamesKeys:(imagesNames,userId) =>{
let allImagesResult = [];
// use map
imagesNames.map((x,i) =>{
const params = {
Bucket: module.exports.GetImageBucket(userId),
Key: module.exports.GenerateImagePresignKey(x,userId),
//Expires: 15 * 60,
ContentType: 'image/jpeg'
};
// then get the presigned url
allImagesResult.push({
image_name: x,
presign_url: s3client.getSignedUrl('putObject', params)
});
});
return allImagesResult;
}
Bucket CORS config:
<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
<AllowedOrigin>*</AllowedOrigin>
<AllowedMethod>POST</AllowedMethod>
<AllowedMethod>GET</AllowedMethod>
<AllowedMethod>PUT</AllowedMethod>
<AllowedMethod>DELETE</AllowedMethod>
<AllowedMethod>HEAD</AllowedMethod>
<ExposeHeader>ETag</ExposeHeader>
<AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>
Example error:
Access to XMLHttpRequest at '' from origin 'http://localhost:4200' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
PUT net::ERR_FAILED
EDIT: needles to say, uploading 32 image size 110 KB is OK.
