Skip to main content

Scrapinghub Hubstorage Collection scanner.

Project description

High level hubstorage collection scanner

  • Provides convenient way to scan a collection in batches

  • Allows to merge data from multiple collections

  • Accepts endts and startts in many string formats (as accepted by dateparser lib) or standard HS epoch in millisecs

  • Accepts excluded prefixes

  • Adds stopbefore feature (analogous to startafter but the inverse)

  • Provides method for arbitrary prefix aggregation counting

  • Supports partitioned collections

  • Provides a suite for testing hs collection code.

Up to version 0.1.6: Python2 only Starting version 0.2: Python3 only

See usage instructions at scanner.py docstring.

Instalation

pip install collection-scanner

Project details


Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

collection_scanner-0.5.1.tar.gz (9.1 kB view hashes)

Uploaded Source

Built Distribution

collection_scanner-0.5.1-py3-none-any.whl (10.5 kB view hashes)

Uploaded Python 3

Supported by

AWS AWS Cloud computing and Security Sponsor Datadog Datadog Monitoring Fastly Fastly CDN Google Google Download Analytics Microsoft Microsoft PSF Sponsor Pingdom Pingdom Monitoring Sentry Sentry Error logging StatusPage StatusPage Status page