Solution for large scale site maps

Question

I'm working on a nationwide retail web site with thousands of categories and product pages and my client wants to move from a single sitemap to multiple sitemaps based on page categorization. Here is how the new XML sitemaps for URLs would be constructed:

XML Sitemap for Category Pages – all category pages can fall into a single sitemap
XML Sitemap for Family Pages – if family pages are < 40,000 they can fall into a single sitemap
XML Sitemap for Product Pages – when product pages reach 40,000 a new XML sitemap would be created.

. Is there a free/professional solution for generating sitemaps based on such rules? Thanks

Nothing will come out of the box and do exactly what you need, but you can easily generate xml sitemaps with PHP using your item database. If you'd like I can post a small php script I use to generate a sitemap on a large site. — Josh Mountain, Feb 28 '13 at 22:12

Josh Mountain · Accepted Answer · 2014-07-07T20:49:41.700

I have a large site that I generate a sitemap with (well actually several sitemaps in batches of 40k URLs at a time). Here is a basic version of a php sitemap generator I wrote:

<?php

/* Do some stuff */
/* Connect to mysql etc */

/* create a dom document with encoding utf8 */
    $domtree = new DOMDocument('1.0', 'UTF-8');

/* create the root element of the xml tree */
    $xmlRoot = $domtree->createElement("urlset");
    $xmlRoot -> appendChild(new DomAttr('xmlns', 'http://www.sitemaps.org/schemas/sitemap/0.9'));
    $xmlRoot -> appendChild(new DomAttr('xmlns:xsi', 'http://www.w3.org/2001/XMLSchema-instance'));
    $xmlRoot -> appendChild(new DomAttr('xsi:schemaLocation', 'http://www.sitemaps.org/schemas/sitemap/0.9 http://www.sitemaps.org/schemas/sitemap/0.9/sitemap.xsd'));

/* append it to the document created */
    $xmlRoot = $domtree->appendChild($xmlRoot);

/* Loop through our database results and create URL entries for each page */
    while ($row = mysql_fetch_array($result)){
        $currentTrack = $domtree->createElement("url");
        $currentTrack = $xmlRoot->appendChild($currentTrack);
        $currentTrack->appendChild($domtree->createElement('loc','https://mysite.com/viewer.php?file='.$row['filename']));
        $currentTrack->appendChild($domtree->createElement('changefreq','weekly'));
        $currentTrack->appendChild($domtree->createElement('priority','1.0'));
    }

/* save the xml sitemap */
    $domtree->formatOutput = true;
    $domtree->preserveWhitespace = false;
    $domtree->save('sitemap.xml');

Obviously your milage will vary and you will need to do some things to make this work for you, but you could run a loop like this to generate a sitemap of all the categories, then all the products etc.

This is pretty much what I was looking for...thanks for the sample code. — Mukul, Mar 01 '13 at 19:58

Solution for large scale site maps

1 Answers1