webhdfs

1.2.0 • Public • Published

node-webhdfs

Build Status NPM version

Hadoop WebHDFS REST API (2.2.0) client library for node.js with fs module like (asynchronous) interface.

Examples

Writing to the remote file:

var WebHDFS = require('webhdfs');
var hdfs = WebHDFS.createClient();
 
var localFileStream = fs.createReadStream('/path/to/local/file');
var remoteFileStream = hdfs.createWriteStream('/path/to/remote/file');
 
localFileStream.pipe(remoteFileStream);
 
remoteFileStream.on('error', function onError (err) {
  // Do something with the error
});
 
remoteFileStream.on('finish', function onFinish () {
  // Upload is done
});

Reading from the remote file:

var WebHDFS = require('webhdfs');
var hdfs = WebHDFS.createClient({
  user: 'webuser',
  host: 'localhost',
  port: 80,
  path: '/webhdfs/v1'
});
 
var remoteFileStream = hdfs.createReadStream('/path/to/remote/file');
 
remoteFileStream.on('error', function onError (err) {
  // Do something with the error
});
 
remoteFileStream.on('data', function onChunk (chunk) {
  // Do something with the data chunk
});
 
remoteFileStream.on('finish', function onFinish () {
  // Upload is done
});

TODO

  • Implement all calls (GETCONTENTSUMMARY, GETFILECHECKSUM, GETHOMEDIRECTORY, GETDELEGATIONTOKEN, RENEWDELEGATIONTOKEN, CANCELDELEGATIONTOKEN, SETREPLICATION, SETTIMES)
  • Improve documentation
  • More examples

Tests

Running tests:

npm test

License

MIT

Package Sidebar

Install

npm i webhdfs

Weekly Downloads

364

Version

1.2.0

License

MIT

Unpacked Size

28.4 kB

Total Files

6

Last publish

Collaborators

  • harrisiirak