I don't want to read the README
Just show me how it works as fast as possible!
npm i -g file-fetchermkdir /tmp/file-fetcher-testcd !$echo '[ { "url": "https://dummyimage.com/200x100/ddd/777.jpg&text=file-fetcher+test", "path": "img", "name": "#.jpg", "delay": 1 }]' > config.jsonfile-fetcher config.json
The images are being downloaded to /tmp/file-fetcher-test/img
every minute.
File fetcher
file-fetcher is a simple tool that lets you download files recurrently.
Installation
$ npm install -g file-fetcher
Usage
- Create a config file with your configuration (See documentation and example below)
- Quick use:
$ file-fetcher <config file>
- Pro use:
$ file-fetcher [ --config | -c ] <config file> [ --destination | -d <destination folder> ]
Default destination folder is cwd.
Config file properties
The config file must contain an array of objects. Each object corresponds to a file and can/must have the following properties:
Property | Type | Required? | Default | Description |
---|---|---|---|---|
url | String | ✓ | The complete URL from which to download the file | |
path | String | ✓ | The relative local path where to save the file | |
name | String | ✓ | The name pattern of the destination file (e.g.: my-file-number-#.jpg) | |
description | String | name |
The string used to represent the file in log messages | |
firstIndex | Number | 1 | The first number used as index (#) in name pattern | |
digits | Number | 3 | The minimum number of digits for the index (#) in name pattern. Smaller numbers will be padded with 0s on the left. | |
cron | String | ✓ or delay |
The cron pattern that determines the download frequence | |
delay | Number | ✓ or cron |
The delay in minutes that determines the download frequence |
File name pattern
The file name needs to be a pattern in order to not overwrite the same file after each download.
Pattern substitutions:
String | Replaced by |
---|---|
# | Index of the download iteration |
{date} | A string representing the current time |
delay
vs cron
To determine the download frequence of each file, you can use either the delay
property or the cron
property.
delay
is simply a number of minutes between each download
N.B.: due to the cron implementation, using a number like 29 will trigger a download at 00:00, 00:29, 00:58, 01:00, 01:29, ...cron
is a cron expression
Config example
A functional .sample.config.json
file is provided in the sources.