Why I'm getting 500 error when using file_get_contents(), but works in a browser?

Question

$html = file_get_contents("https://www.[URL].com"); 
echo $html;

produces this in the error logs:

PHP Warning: file_get_contents(https://www.[URL].com) [function.file-get-contents]: failed to open stream: HTTP request failed! HTTP/1.1 500 Internal Server Error in /Applications/MAMP/htdocs/test.php on line 13";

However, the site works fine in a browser.

I tried using cURL as well. I don't get any errors in the log file, but $html now echoes:

Server Error in '/' Application.
Object reference not set to an instance of an object.

...some more debugging info

Any ideas how to work around this?

Have you considered asking whoever hosts the other website? – Ignacio Vazquez-Abrams May 09 '12 at 22:05 — Ignacio Vazquez-Abrams, May 09 '12 at 22:05

score 78 · Accepted Answer · edited May 10 '12 at 19:00

78

Try this workaround:

$opts = array('http'=>array('header' => "User-Agent:MyAgent/1.0\r\n"));
$context = stream_context_create($opts);
$header = file_get_contents('https://www.example.com',false,$context);

If this doesn't work, maybe you cant read from https?

edited May 10 '12 at 19:00

Gilles 'SO- stop being evil'

98,216
36
202
244

answered May 09 '12 at 22:08

blang

2,050
2
18
17

Interesting solution. However, when I change the user agent to Chrome, I now get a 400 error. Using the "MyAgent" string works perfectly though. – remarsh Nov 27 '14 at 16:42
3

Thank you - it works, but I'm intrigued why this works. Any specific reasoning? – Dan Smart Sep 21 '15 at 16:10
1

@DanSmart PHP's default user agent (which is most probably just a blank string) is blocked by the web server you are requesting web page from. That's why you may need to set a fake user agent. – modu Oct 02 '16 at 18:56
The reason is that the web application goes wrong while dealing with User-Agent header. For example, their database goes wrong with NULL User-Agent insert. – Nick Tsai Jan 25 '18 at 09:01
@modu is there a way to unblock that by default? because allow_url_open in php.ini is not working, thanks – Fernando Torres Jan 01 '20 at 07:24
@FernandoUrban you can try using curl. – modu Jan 02 '20 at 12:03

score 5 · Answer 2 · answered May 19 '15 at 13:05

I had to enter more data into the header:

$opts = array('http' => array(
    'method' => "GET",
    'header' => "User-Agent Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Firefox/24.0\r\n"
    . "Accept:text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\n"
    . "Accept-Encoding:gzip, deflate\r\n"
    . "Accept-Language:cs,en-us;q=0.7,en;q=0.3\r\n"
    . "Connection:keep-alive\r\n"
    . "Host:your.domain.com\r\n"
    ));
$context = stream_context_create($opts);
$html = file_get_contents($sap_url, FALSE, $context);

Why I'm getting 500 error when using file_get_contents(), but works in a browser?

2 Answers2

Linked

Related