-1

what is the most optimum way to make 1000+ HTTP requests without worrying about socket/port exhaustion?

I'm facing performance & socket/port issues where for example if there are 1400 requests sometimes only half of it are getting executed. It's also affecting other scripts which are running in parallel looking for sockets/port for their own execution.

I'm calling my async Task method like this : -

//Inside a for loop which can be iterated 1000+ times based on condition
c8y_pushAlert(c8y_Alert_MSG).GetAwaiter().GetResult();
//Reason behind using above line is that I want each request to complete it's
//execution cycle of both get and post request inside async Task method before
//the next c8y_Alert_MSG is pushed.

Also to note that the above line is not inside any method, There's no main method & I cannot use it inside static async Task Main(string[] args) due to some reasons

Here's whats going on in my async Task method : -

public async Task c8y_pushAlert(string Alert_MSG)
{
//Doing bunch of stuff with Alert_MSG
using (var client = new HttpClient())
    {
        client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Basic","auth_val");
        HttpResponseMessage get_response = await client.GetAsync("https://myurl.com/"+controllerName);
        var get_responseString = await get_response.Content.ReadAsStringAsync();

        //doing bunch of stuff with get_responseString to use contents from it for Post request

        client.DefaultRequestHeaders.Add("Accept", "application/vnd.com.nsn.xyz.alarm+json");
        var stringContent = new StringContent(c8y_finalPayloadString, Encoding.UTF8, "application/json");
        stringContent.Headers.ContentType.CharSet = "";
        HttpResponseMessage response = await client.PostAsync("https://myurl.com", stringContent);

        if (response.IsSuccessStatusCode)
        {
            var responseString = await response.Content.ReadAsStringAsync();
            Console.WriteLine(responseString);
        }
  }
}
saurabh dhyani
  • 55
  • 5
  • 3
    Instead of creating a new HttpClient for each request, get a client from a pool of clients by using HttpClientFactory https://docs.microsoft.com/en-us/dotnet/architecture/microservices/implement-resilient-applications/use-httpclientfactory-to-implement-resilient-http-requests – Daniel Kiptoon Jun 02 '22 at 06:06
  • 3
    Well, first of all, you should really reuse your `HttpClient`, [straight from the horse's mouth](https://docs.microsoft.com/en-us/dotnet/api/system.net.http.httpclient?view=net-6.0#instancing) "HttpClient is intended to be instantiated once and reused throughout the life of an application [...] If you instantiate an HttpClient class for every request, the number of sockets available under heavy loads will be exhausted" – MindSwipe Jun 02 '22 at 06:07
  • Secondly, you should really try and avoid using `GetAwaiter().GetResult()` (more on why [here](https://stackoverflow.com/a/39007110/9363973)) and instead `await` your call to `c8y_pushAlert` – MindSwipe Jun 02 '22 at 06:09
  • @MindSwipe something like this : ```await c8y_pushAlert(c8y_Alert_MSG);``` ?? Will it wait for one cycle to complete before pushing another ```c8y_Alert_MSG```?? – saurabh dhyani Jun 02 '22 at 06:11
  • Any reason why my question is downvoted?? Wouldn't really make any sense to downvote without letting me know what's wrong. – saurabh dhyani Jun 02 '22 at 06:13
  • Again, [quoting Microsoft](https://docs.microsoft.com/en-us/dotnet/csharp/language-reference/operators/await) "The await operator suspends evaluation of the enclosing async method until the asynchronous operation represented by its operand completes", so, yes – MindSwipe Jun 02 '22 at 06:13
  • 1
    And btw, you're question is being downvoted because you didn't actually explain the problem you're having, and questions along the line of "What is the best way to do X?" will invariably lead to opinion based answers, which are specifically off-topic for Stack Overflow. You can check out [this](https://meta.stackoverflow.com/q/255468/9363973) meta Q&A to read more on *why* this decision was made – MindSwipe Jun 02 '22 at 06:16
  • @MindSwipe can ```await c8y_pushAlert(c8y_Alert_MSG);``` be called from inside a synchronous method or it must be inside async method?? Part of the reason why I went with ```.GetAwaiter().GetResult();``` is because I have to call it from inside sync method. I'm new to c# so have these doubts. – saurabh dhyani Jun 02 '22 at 06:17
  • `await` can only be used in an `async` context, if you want to do something asynchronously, then the entire call stack up to that point should probably (more likely has to be) be async. This is also why async/ await is often referred to as "contagious" or "viral" – MindSwipe Jun 02 '22 at 06:22

0 Answers0