It seems there are conditions where teraslice can no longer write to a file. This has been seen under conditions of high cluster load when there were many teraslice workers writing at a very high rate. All of the other workers are appending without issue while one worker continues to error on every slice with an error that looks like this:

Error sending data to file: /elasticsearch/myData-2018.01.16/host1.86

The file remains the same in each error and while the job is running and the file remains "open" it will fail an FSCK.

To fix this, I propose that teraslice should stop the worker after some default but configurable number of identical failures. Then it should restart a worker to replace the failed one. This will allow data to continue to be written.

It's possible that this will leave an unreadable file, but perhaps not since I've seen the file become readable after the job was stopped.

Supporting the 'connection' parameter should be a standard feature of all modules that read or store data.

The configuration schema does not define all the parameters used by the module so needs to be filled out.

• Add Travis-CI Automation.
• Add tests.
• Report Code Coverage to CodeCov, or a similar service.
• Use ESLint and the latest changes from teraslice.
• NPM Publish using the name teraslice-hdfs-append.

If the user configures a path with a trailing slash on it we tack on the filename also with a slash creating a double slash in the path. This causes the file create to fail and the HDFS library will throw an error with the entire content of the slice included which then ends up in the log. We need to better validate the incoming config and truncate any slashes.