parallel588_polipus 0.4.1
An easy to use distributed web-crawler framework based on Redis
Gemfile:
=
install:
=
Runtime Dependencies (6):
hiredis
>= 0.4.5, ~> 0.5
http-cookie
>= 1.0.1, ~> 1.0
nokogiri
>= 1.6.0, ~> 1.6
redis
>= 3.0.4, ~> 3.0
redis-bloomfilter
>= 0.0.3, ~> 0.0
redis-queue
>= 0.0.4, ~> 0.0