I am making a SEO project using PHP where I need to crawl every pages/directories of a website. But for this, I have to know or list all the directories of that website. Is it possible? Can we do it with PHP?
Asked
Active
Viewed 76 times
1 Answers
3
No. HTTP provides no way for a client (regardless of the programming language it is written in) to ask for directory listings.
This is why search engines crawl links and make use of sitemaps.
A PHP program could inspect the directory structure of the file system of the computer it is running on, but even that wouldn't give you a good view in the general case as most websites are not simply a bunch of files served up by mapping URLs directly onto a filesystem (for example, the Front Controller design pattern is quite common).
Quentin
- 857,932
- 118
- 1,152
- 1,264