Validate Correct URL With Http Status Code

Question

I am trying to validate URL which exists and which doesn't exist by status code.

here I have two URL like below:

Valid URL: https://stackoverflow.com/questions/ask

Invalid URL: https://stackoverflow.com/questions/ask/testquestion

Invalid URL has 404 status code like below:

Same thing I am trying to achieve using below code but I am not getting exact output. Instead 404 or 200 (if valid) I am getting 0. Here is my code:

function getUrlStatus(url) {
    var request = new XMLHttpRequest();
    request.onreadystatechange = function() {
        if (request.readyState === 4) {
            request.status;

            console.log(request.status);
        }
    };
    request.open("GET", url, true);
    request.send();
}

Is there any other why to achieve this? Like If URL is valid I am looking for 200 or others if invalid. This URL https://httpstatus.io/ has the great example.

Pascal Lamers · Accepted Answer · 2020-06-23T18:05:32.703

0

In general, to read a status code from a request depends on the library you use. The way you are doing it should by fine. However according to https://developer.mozilla.org/en-US/docs/Web/API/XMLHttpRequest/status Browsers also report a status of 0 in case of XMLHttpRequest errors.

Try listen for the error events on the XMLHttpRequest Object to get more insight in what's going on. That's your specific case, since you are trying to use XMLHttpRequest.

  request.addEventListener('error', function (e) {
     
  });

Also you can find lots of different suggestions for status 0 in this SO Question XMLHttpRequest status 0 (responseText is empty)

I also suggest looking into the more modern FETCH API which is widely supported : https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API .

fetch(myRequest)
  .then(response => {
     console.log(response.status);
     console.log(response.statusText);
     return response.json();
  })
  .then(data => {
      /* process your data further */
  })
  .catch(error => console.error(error));

Example

I added a quick example how to use Fetch API to get Status Code

https://codesandbox.io/s/sad-waterfall-c4k1r

The problem with https://stackoverflow.com/questions/ask/testquestion specifically is that the request fails du to CORS protection, hence you don't get a status code. You are simply not allowed to request the source. That's a complete other topic.

Problem

Due to reasons like Same Origin Policy, it is not always possible to get a valid status code back.

From Browser

In the Browser ? You can't. Not with simple requests like XMLHttpRequest or Fetch due to the issue I mentioned above. The example https://httpstatus.io/ shows that it works, but they are not performing the check within the browser. They are performing the check in their backend/server, somehow simulating a browser with a valid User Agent and showing you the result.

From Server

You can use server side librarys to perform the request for you. I tried the first package I found called got . Here is an example:

const got = require("got");
const url = "https://stackoverflow.com/";

(async () => {
    try {
        
        const response = await got(url, {
            throwHttpErrors : false
        });
        
        console.log("Status Code : " + response.statusCode);
       
    } catch (error) {
     
        console.error(error);
     
    }
})();

// "Status Code : 200"

const got = require("got");
const url = "https://stackoverflow.com/asdkljasdjklasd";

(async () => {
    try {
        
        const response = await got(url, {
            throwHttpErrors : false
        });
        
        console.log("Status Code : " + response.statusCode);
       
    } catch (error) {
     
        console.error(error);
     
    }
})();

// "Status Code : 404"

edited Jun 23 '20 at 18:05

answered Jun 18 '20 at 14:14

Pascal Lamers

2,909
2
13
39

If the response is empty how we are getting 404 in browser console? What would be the exact way to get the expected status code once the URl valid or Invalid? – Md Farid Uddin Kiron Jun 18 '20 at 14:30
1

@MdFaridUddinKiron — Open the console and read the error messages. The duplicate will explain them in more detail. – Quentin Jun 18 '20 at 14:30
I seen `status: 302` there but how do I get in my code that status? – Md Farid Uddin Kiron Jun 18 '20 at 14:36
@MdFaridUddinKiron I updated my answer - have you looked into using Fetch API ? – Pascal Lamers Jun 18 '20 at 14:45
Yes I did but doesn't work for me! Again trying yours let's see – Md Farid Uddin Kiron Jun 18 '20 at 14:46
How do I test your `fetch` block? – Md Farid Uddin Kiron Jun 18 '20 at 14:47
I added a quick example in my answer . – Pascal Lamers Jun 18 '20 at 14:58
"You are simply not allowed to request the source. That's a complete other topic." — No, that's the exact problem and why it is reporting status 0. – Quentin Jun 18 '20 at 14:59
Because you do not get a status code back. The request fails due to CORS protection. If you try the same thing with e.g. http://httpstat.us/500 you will get a status code back. The question was how to validate a URL with Status Code. – Pascal Lamers Jun 18 '20 at 15:01
Hey I already left, will have a look tomorrow then let you know the update. – Md Farid Uddin Kiron Jun 18 '20 at 18:23
I have tried but still it doesn't fill well. I cannot get exact status code of a response that can easily given from server. – Md Farid Uddin Kiron Jun 22 '20 at 14:45
Unless the website returns CORS headers in the response (which is incredibly unlikey for a client-facing site) then what you're trying to do will not be possible in JS due to the [Same Origin Policy](https://en.wikipedia.org/wiki/Same-origin_policy). You will have to make the request server side, using cURL for example. I already said that in my answer. – Pascal Lamers Jun 23 '20 at 17:44
@MdFaridUddinKiron I updated my complete answer again to show a server side solution to this problem. Please let me know if this answer helped you and accept if possible. – Pascal Lamers Jun 23 '20 at 18:06

Validate Correct URL With Http Status Code

1 Answers1

Example

Problem

From Browser

From Server