Rand Fishkin interviews Vanessa Fox of Google’s Webmaster Central - SearchMasters highlights Vanessa’s mention of XML Sitemaps. What are XML Sitemaps, and how can XML Sitemaps be used to help search engines spidering of your websites.
The main search engines Google, MSN, Yahoo and Ask have agreed on a standard and have a joint website www.sitemaps.org regarding xml sitemaps.
XML sitemaps are lists of all the url’s of your website. You can include attributes such as how often the content of the pages change, how often you want them spidered, and the relative priority that you want pages spidered. This is a great way of letting the Search Engines know about new pages on your website.
The sitemaps protocol includes xml as note above, lists of links separated by line breaks, and even RSS feeds. RSS are another great way of letting the search engines know about new content. RSS feeds will typically have say the last 15 threads on your forum or blog etc.
You can include the xml sitemap file name in your robots.txt file - as an indication to all the se’s about the existence of your xml sitemap
ie Sitemap: http://www.searchmasters.co.nz
You can also ping the search engines to let them know they need to check your sitemap.
Best that you add the sitemap url to your Google Webmaster Tools page. The page gives you details of the last time that the xml sitemap was downloaded, and any errors that were found. Google will be offering sitemaps in 18 different languages.