0

I spent way too much time finding this bug. So here is my code. Apparently when i use push, the array ends up being complete, but when i use concat there is a estimated 50% chance that i will not get all concatenated items since the concats seem to run at the same time. I did not believe this to be possible, so could anyone explain to me why VERSION 1 works but version 2 does not.

let employments: any[] = [];
let companyIds = ['Id1', 'Id2']
await Promise.all(
    companyIds.map( async ( companyId ) => {
        const companies = await getCompaniesWithId(companyId);
        // VERSION 1
        employments.push( ...(await mergeWithOtherSource( companies )) );
        
        // VERSION 2
        employments = employments.concat( await mergeWithOtherSource( companies ));
    } )
);
// VERSION 1 always returns 3 expected items as are in the databases
// VERSION 2 RANDOMLY returns between 1 and 3 items
return employments.length
Jeremias Nater
  • 341
  • 1
  • 11
  • 1
    `companyIds.map` doesn't return anything. There is nothing to await. `.map` is meant to map an array to another. You are just pushing to an external array which is incorrect. – zero298 Aug 31 '21 at 15:19
  • 1
    @zero298 `companyIds.map` does return `Promise`s (`Promise[]`) – apple apple Aug 31 '21 at 15:22
  • @OP can you try move `await mergeWithOtherSource( companies )` outside of `concat`? – apple apple Aug 31 '21 at 15:30
  • `companyIds.map` does return promises: https://jsfiddle.net/4db6ecxg/1/ – Matt U Aug 31 '21 at 15:36
  • @appleapple Putting await outside of concat also resolves the issue. It seems that concat takes a reference of the original array before waiting for the promise. – Jeremias Nater Aug 31 '21 at 15:42
  • @JeremiasNater, I think I understand it now. I share my understanding in this answer. https://stackoverflow.com/a/69026768/14032355 Someone downvote me because I think I ask the similar question before, but I hope you can have a look of my answer. It may help. – ikhvjs Sep 02 '21 at 11:53

2 Answers2

1

This answer provide alternative (and imho simpler) way to do the same thing. (the possible reason is provided in my another answer)

let companyIds = ['Id1', 'Id2']

/// mix await and then (some people may dislike this)
let employments = [].concat(
   await Promise.all(companyIds.map(companyId =>
      getCompaniesWithId(companyId).then(mergeWithOtherSource)
   ))
)

/// with await
let employments = [].concat(
   await Promise.all(companyIds.map( async ( companyId ) => {
      const companies = await getCompaniesWithId(companyId)
      return mergeWithOtherSource( companies )
   }))
)

/// with 2-step
let companys = await Promise.all(companyIds.map(getCompaniesWithId)
let employments = [].concat(await Promise.all(companys.map(mergeWithOtherSource)))

// can also use flat instead of concat
let employments = (await Promise.all(...)).flat()
apple apple
  • 7,296
  • 1
  • 14
  • 35
0

It's probably because the reference of this is fetched before the evaluation of it's parameter

like the following

// VERSION 1 step by step
// employments.push( ...(await mergeWithOtherSource( companies )) );
let O = employments
let task = mergeWithOtherSource( companies )
let items = await task;
O.push(...value) // no problem, it's always the same object
    
// VERSION 2 step by step
// employments = employments.concat( await mergeWithOtherSource( companies ));
let O = employments // multiple async thread may get the same object
let task = mergeWithOtherSource( companies )
let items = await task;
let temp = O.concat(value) // may use outdate employments
employments = temp

so two async function may execute in the way

// VERSION 2: possible execution order
// "thread" in the sense of javascript `async`, which only one run at the same time, and only yield when `await`.

let O = employments // 1st thread
let task = mergeWithOtherSource( companies ) // 1st thread
let items = await task // 1st thread, waiting

let O = employments // 2nd thread
let task = mergeWithOtherSource( companies ) // 2nd thread
let items = await task // 2nd thread, waiting

let temp = O.concat(items) // 1st thread
employments = temp // 1st thread

let temp = O.concat(items) // 2nd thread, now using outdated employments!
employments = temp // 2nd thread, overwrite the object from thread 1


it may be easier to see if consider the statement

GetEmployments().concat(GetSomething())

GetEmployments() would be evaluate only once, to produce the O in above example.


step by step of a function call (roughly)

// step by step `x.func(y)`
let V = x
let F = V.func
let Arguments = [y]
F.apply(V,Arguments)
apple apple
  • 7,296
  • 1
  • 14
  • 35
  • I think that you are correct, but i dont understand how concat can do something before the parameters are ready. It seem the parameters are awaited outside of concat, so it could not make a reference to "this" because the function is only called when the parameters are awaited as far as i know. As i mentioned apparently the solution works though. – Jeremias Nater Aug 31 '21 at 16:52
  • @JeremiasNater if you think about it like calling `concat(this_object,items)` then it's obvious one of the parameter need to be evaluate first. (and in this case it appears that `this_object` is resolve first) – apple apple Aug 31 '21 at 17:03
  • @JeremiasNater or consider this simple expression `foo().concat(bar())` – apple apple Aug 31 '21 at 17:52
  • Well it wasnt me but your answer doesnt really read well as code since it is mostly pseudo, and im also not quite understanding your examples and why you are making them. The other example fully runs on https://npm.runkit.com/ but for him to be accepted he is missing the example we mentioned showing fully what happens – Jeremias Nater Aug 31 '21 at 20:38
  • @JeremiasNater well it does actually run and produce the same result (if you separate thread 1 and thread 2). It is a step by step explaining. – apple apple Sep 01 '21 at 03:17
  • @JeremiasNater in case you refer to `foo().concat(bar())`, it is to show `foo()` would be evaluate **only once** to produce a value (`O` in the answer), thus either `foo()` or `bar()` need to run first. – apple apple Sep 01 '21 at 03:20
  • @JeremiasNater as in the question `employments.concat(...)`, `employments` would also evaluate **only once**, either before or after it's parameter. – apple apple Sep 01 '21 at 03:22
  • @JeremiasNater I've update the answer, hope it's easier to understand now. – apple apple Sep 01 '21 at 03:36
  • @JeremiasNater continue on my [previous comment](https://stackoverflow.com/questions/69001273/crazy-simultaneous-javascript-array-concat-interferes-with-itself/69001781?noredirect=1#comment121960777_69001781) it does also run directly if you append every `let x` with `let x_thread1` or `let x_thread2` for what thread they're supposed to be in. – apple apple Sep 01 '21 at 04:14
  • As JS only have one thread, I think what matters is not because of the thread but closure I posted a related question about this. https://stackoverflow.com/questions/69009288/why-async-await-have-different-output-when-await-expression-is-a-argument-of-con – ikhvjs Sep 01 '21 at 07:58
  • @ikhvjs no it's not about closure. And yes my answer does include literally *"thread" in the sense of javascript `async`, which only one run at the same time, and only yield when `await`.* if you look at it. – apple apple Sep 01 '21 at 14:32
  • @ikhvjs it's simply an order of evaluation problem. and the javascript evaluate `employments` before it's parameter (`await ...`). (Which is reasonable) – apple apple Sep 01 '21 at 14:37
  • @ikhvjs you can actually observe similar situation without `await` or any closure involved. see this https://jsfiddle.net/vhf39a5c/ – apple apple Sep 01 '21 at 14:56
  • @appleapple, thanks for your reply. I can understand your fiddle and it is sync function. What I have difficulties with is the async function. – ikhvjs Sep 01 '21 at 14:56
  • @ikhvjs if the situation exist in sync function, why you think it would disappear in async function? (btw I updated the snippet a little, which you may missed) – apple apple Sep 01 '21 at 14:57
  • @appleapple, can you please help to answer this question? https://stackoverflow.com/questions/69015688/why-await-inline-on-variable-in-outer-scope-is-different-from-simple-await-assig?noredirect=1#69015907 – ikhvjs Sep 01 '21 at 15:06
  • @ikhvjs well the reason (as [both](https://stackoverflow.com/a/69010179/5980430) [answer](https://stackoverflow.com/a/69015907/5980430) to both of your question and as [my answer](https://stackoverflow.com/a/69001781/5980430) said), is that the expression before the dot is evaluate before the parameter. – apple apple Sep 01 '21 at 15:26
  • @ikhvjs or (the expression before the `+`) is evaluate before (the expression after the `+`) – apple apple Sep 01 '21 at 15:34
  • @appleapple, thanks for your comment. I post an answer about what I understand. You may have a look. https://stackoverflow.com/a/69026768/14032355 – ikhvjs Sep 02 '21 at 08:34