GSiteCrawler Review

February 27, 2009

I’ve found this great tool for creating a Sitemap, quickly (well depending on the number of URLs you have), and fairly easily (after overcoming the learning curve) with a good amount of options.

Unlike online Sitemap tools which usually limit you to 500 or so pages, which can be not enough especially if you have a blog with many posts, GSiteCrawler is downloadable and runs off your computer, sending a network of spiders to fetch your site information.

You can customize settings, such as URLs not to crawl, parameters to drop, etc. and the options for outputting files after finishing a crawl is quite extravagant (many, many different options including CSV files and GZip files).  There are other programs you can buy, but this one is definitely getting the job done. 

You can download a copy free (opensource) at http://gsitecrawler.com/en/download/.  Happy crawling!