govuk_seed_crawler 3.0.0
Retrieves a list of URLs to seed the crawler by publishing them to a RabbitMQ exchange.
Gemfile:
=
install:
=
Runtime Dependencies (6):
bunny
~> 1.3
crack
= 0.4.4
nokogiri
~> 1.6.0
public_suffix
~> 1.4.6
sitemap-parser
~> 0.3.0
slop
~> 3.6.0