file-fetcher

0.8.4 • Public • Published

I don't want to read the README

Just show me how it works as fast as possible!

npm i -g file-fetcher
mkdir /tmp/file-fetcher-test
cd !$
echo '[
  {
    "url": "https://dummyimage.com/200x100/ddd/777.jpg&text=file-fetcher+test",
    "path": "img",
    "name": "#.jpg",
    "delay": 1
  }
]' > config.json
file-fetcher config.json

The images are being downloaded to /tmp/file-fetcher-test/img every minute.

File fetcher

npm version GitHub version

file-fetcher is a simple tool that lets you download files recurrently.

Installation

$ npm install -g file-fetcher

Usage

  • Create a config file with your configuration (See documentation and example below)
  • Quick use:
$ file-fetcher <config file>
  • Pro use:
$ file-fetcher
    [ --config | -c ]      <config file>
    [ --destination | -d   <destination folder>  ]

Default destination folder is cwd.

Config file properties

The config file must contain an array of objects. Each object corresponds to a file and can/must have the following properties:

Property Type Required? Default Description
url String The complete URL from which to download the file
path String The relative local path where to save the file
name String The name pattern of the destination file (e.g.: my-file-number-#.jpg)
description String name The string used to represent the file in log messages
firstIndex Number 1 The first number used as index (#) in name pattern
digits Number 3 The minimum number of digits for the index (#) in name pattern. Smaller numbers will be padded with 0s on the left.
cron String ✓ or delay The cron pattern that determines the download frequence
delay Number ✓ or cron The delay in minutes that determines the download frequence

File name pattern

The file name needs to be a pattern in order to not overwrite the same file after each download.

Pattern substitutions:

String Replaced by
# Index of the download iteration
{date} A string representing the current time

delay vs cron

To determine the download frequence of each file, you can use either the delay property or the cron property.

  • delay is simply a number of minutes between each download
    N.B.: due to the cron implementation, using a number like 29 will trigger a download at 00:00, 00:29, 00:58, 01:00, 01:29, ...
  • cron is a cron expression

Config example

[
  {
    "url": "http://www.my-site.com/path/to/file.jpg",
    "name": "file_nb_#({date}).jpg",
    "path": "relative/path/to/file",
    "cron": "0 0/2 7-18 * * *",
    "description": "my file"
  },
  {
    "url": "http://www.my-site.com/path/to/other/file.jpg",
    "name": "file_nb_#.jpg",
    "firstIndex": 42,
    "digits": 4,
    "path": "relative/path/to/other/file",
    "delay": 3,
    "description": "my other file"
  }
]

A functional .sample.config.json file is provided in the sources.

Package Sidebar

Install

npm i file-fetcher

Weekly Downloads

0

Version

0.8.4

License

AGPL-3.0

Unpacked Size

19.6 kB

Total Files

8

Last publish

Collaborators

  • pdonias