arachnid 0.2.2

Arachnid is a web crawler that relies on Bloom Filters to efficiently store visited urls and Typhoeus to avoid the overhead of Mechanize when crawling every page on a domain.

Gemfile:
=

install:
=

Versions:

0.4.1 January 17, 2014 (5 KB)
0.4.0 January 16, 2014 (4.5 KB)
0.3.2 January 16, 2014 (5 KB)
0.3.1 January 16, 2014 (5 KB)
0.3.0 January 16, 2014 (4.5 KB)
0.2.2 November 29, 2011 (4.5 KB)

Show all versions (12 total)

Runtime Dependencies (4):

bloomfilter-rb = 2.1.1

domainatrix = 0.0.10

nokogiri = 1.5.0

typhoeus = 0.3.2

Show all transitive dependencies

Owners:

Authors:

dchuk

arachnid 0.2.2

Gemfile:
=

install:
=

Versions:

Runtime Dependencies (4):

Owners:

Authors:

SHA 256 checksum:

Total downloads 50,085

For this version 4,789

Version Released:
November 29, 2011 9:49pm

Licenses:
N/A

Required Ruby Version: None

Links:

arachnid 0.2.2

Gemfile: =

install: =

Versions:

Runtime Dependencies (4):

Owners:

Authors:

SHA 256 checksum:

Total downloads 50,085

For this version 4,789

Version Released: November 29, 2011 9:49pm

Licenses: N/A

Required Ruby Version: None

Links:

Gemfile:
=

install:
=

Version Released:
November 29, 2011 9:49pm

Licenses:
N/A