Use a Visual Sitemap Creator to Make Two Sitemaps

Get Your Ecommerce Site Noticed by Robots, Crawlers and Spiders

Sketch sitemap with wireframes
SpiffyJ/DigitalVision Vectors/Getty Images

A sitemap is navigational tool that lists every accessible page in the site according to the site hierarchy. There are two basic kinds of sitemaps: One that is designed to be read by humans, and the other designed for visiting robots, crawlers and spiders.

Every ecommerce site should have both kinds of sitemaps, and they should be updated whenever new content or products are added to the store. If the site's ecommerce platform doesn't automatically generate sitemaps, they can be made manually.

HTML Sitemaps (for Humans)

The HTML sitemap is like a table of contents and is usually reached from a footer on the site. It can be formatted any number of ways. For example, if the ecommerce site is relatively small, a list of every category, with category headings linking to their respective pages, would be enough. For larger sites with multiple categories and sub-level categories, the sitemap should break them down. A full-service clothing shop, for example, might have general categories, such as Women, Men and Children, and then list various clothing types within each (e.g. Coats, Pants, Tops, Sweaters). The products sold on the site will dictate the hierarchy of the sitemap. 

XML Sitemaps (for Spiders)

XML sitemaps are designed to guide visiting robots, crawlers and spiders. The sitemap tells search engines how often a site's pages are updated, the last time each page was modified, and the relative priority of each page.

The sitemap file (sitemap.xml) is placed at the root (or topmost) directory of your site where it can be easily found whenever your site is crawled. XML sitemaps are one of the most effective ways to ensure that a site is listed in the major search engines.

Its companion file, robots.txt, contains additional instructions for the visiting robot.

For example, if you don't want every part of your site indexed, you can say, "Don't bother to look in this folder, and stay out of that folder over there." This is not to be mistaken for a security measure in any way, and will keep out only those robots that follow the rules.

After an XML sitemap is in place, it's necessary to notify Google through a Webmaster Tools account. Those without such an account should set one up right away. But don't stop there. It's also worth checking in with the other major search engines, including Yahoo, Ask and Bing.

Sitemap Generators

Although sitemaps can be created manually, it's much easier to use a program that automates the process. There's a wide range of software available, much of it for free. A good place to start is Rage Software's Sitemap Automator, which is easy to use and is available for both the Mac and PC. A free trial version can be downloaded from their site, and the standard version can be purchased for $29.95.

XML-Sitemaps.com is an online service that will generate a sitemap up to 500 pages at no charge. It takes about eight minutes to generate a 300-page sitemap. When the code is complete, it can be copied and pasted to Notepad, saved as sitemap.xml, uploaded to an FTP and downloaded to the server.

At this point, it's necessary to notify Google that the sitemap has been updated and that the site is ready to be crawled. The unlimited version of Sitemap Generator can be purchased for $29.95. It supports larger sites and can be run as a chron job, which means you can set it up to run on schedule.

Visual Sitemaps

PowerMapper is the Cadillac of them all, and generates a wide range of visual sitemaps -- in addition to the standard text-based HTML and XML. As it scans the site, it captures meta data and generates a thumbnail of each page, and then uses the information to create a map. Each map is fully hyperlinked, so you can drill down and jump to a specific page.

A visual sitemap is a great planning tool for anyone who designs or manages a website. It sure beats crawling around on the floor with Post-It Notes!

(That's not what is usually meant by "crawlers," but just ask any information architect who has been around for awhile.)

PowerMapper makes it easy to apply different mapping styles once the data is in place. For example, if you want to illustrate the site hierarchy, you can choose from a number of thumbnail map styles. However, to emphasize the site structure and content clusters, you might prefer the Skyscraper style where the height of each block indicates its relative distance from the home page. Sitemaps can also be exported as graphic images (PNG) for use in presentations and reports.

PowerMapper Standard includes seven map styles and exports XML sitemaps. The price is $149 for a single-user license. PowerMapper Pro includes thirteen map styles, and lets you overlay analytics/SEO data and notation. The price is $349 for a single-user license.

Special Note for Magento Users: PowerMapper needs some additional instructions to find its way around a Magento site. If left to its own devices, it will map every link it finds, which can easily turn into 16,000 or more. (You don't even want to go there!) One issue is that the Compare Products links cause it to produce more and more pages with new combinations of products. There may be similar issues on sites that allow you to apply filters to lists of products, such as Clothes - Trousers - Blue. So be sure to add the following lines to the blocked links box before you map your site. Here's how:

On the View menu, click Options. On the Blocks tab, type the following code into the Block Links box. Then, click OK to save the instructions.

*/customer/account/login*
*/checkout/cart*
*/catalog/product_compare*
*?*

This code has been tested with a Magento site and it reduced what had previously been over 16,000 links down to 800 or so. It took about an hour to map the site. And, of course, you can experiment with different instructions to see what works best. You might use the Block instructions to exclude certain areas so you can focus on a particular part of the site.

No matter what platform an ecommerce site is built on, creating a complete and accurate sitemap and broadcasting it to Google and other search engines is more than worth the time, effort and money.