rawler 0.0.6

Rawler is a Ruby library that crawls your website and checks the status code for each of your links. Useful for finding dead links. Rawler will only parse pages with content type 'text/html', but it will check for the response code of every link.