0

I am trying to do multiple delete operation on dynamodb table. However due to dynamodb limitation of 25 item per batch, I cannot delete more than 25 item per batch. I have list of deleteWriteOperation (batch of 25 each) and I am trying to run the batches parallelly. Any suggestion how can I avoid this or how do I add delay functionality so dynamodb autoscale while task wait.

Here is my code:

// batches is list of list holding DeleteWriteOperation (batch of 25) each list
var opts = new ParallelOptions { MaxDegreeOfParallelism = Convert.ToInt32(Math.Ceiling((Environment.ProcessorCount * 0.75) * 1.0)) }; // limiting number of concurrent threads
try
{
    Parallel.ForEach(
      batches,
      opts,
      async batch =>
      {
          await processDelete(batch, clientId);
      });
}
catch (Exception ex)
{
    _logger.LogDebug(e)
}

Here is the error that I received using the above code:

Amazon.DynamoDBv2.AmazonDynamoDBException: 'Throughput exceeds the current capacity for one or more global secondary indexes. DynamoDB is automatically scaling your index so please try again shortly.'

Peter Csala
  • 10,331
  • 15
  • 20
  • 47
akh
  • 95
  • 3
  • 11
  • `Parallel.ForEach` does not support async operation. In .NET 6 we have `Parallel.ForeachAsync`, prior that consider to use `Task.WhenAll` for async I/O operations – Peter Csala Apr 04 '22 at 07:17

0 Answers0