vauvenal5 / pieshare Goto Github PK

Improve the ITs to have a small but efficient set of tools which allows the developer to easily start a number of clients and configure them what and when to do.

Implement a possibility to send a Message to exactly one user in the cloud, and also to get the information about the sender on the receivers side.

There is a bug with empty files. They won't get shared right to other nodes.

setup nightly build enviroment

Add encryption to all communication.

Extract the executor from piePlate to pieTools executor service

Is it necessary for the executor service to be able to handle more then one task per event? Or is this not needed for the time being?

If yes: Ordered or unordered execution? Have the tasks to be independent?

Read startup configuration.

Design sharing between users. This is the task for the LAN based sharing!

clean up the database code and add further tests

• Fix Lock: when sharing a file it will get locked and won't be usable
• Stop unneeded threads
• Restart share when files requested

integrate a code coverage tool to work with the current setup

When connecting to a LAN I don't want somebody else to be able to use my cloud or to get my data. So instances owned by should be able to authenticate to each other.

clean up rest of the code and add where necessary additional tests

It is to be expected that files bigger than 1GB will take a considerable amount of time to be encrypted and decrypted. Therefor it will be preferable to perform these operations parallel for big files.

One idea is to write ParallelCipherInput/OutputStream-Classes analog to the Java CipherInput/OutputStream-Classes which perform this process in parallel. On of the obstacles there is the ordering of the different encrypted parts so they are in the right order at the end. The problem there is that the whole process will be again slowed down by this step of synchronization. One possible solution is to simply not order them. By using an appropriated amount of bytes at the end or beginning of each block we could simply add the order to the file and write the blocks on the sending side unordered to the file.
On the receiver side all the blocks are read from the file decrypted (again multiple blocks at the same time) and written back into the right order. Main advantage here the encrypted file can be meanwhile still used for sharing with the other nodes.

Additionally it is to consider to do the whole thing not on the CPU but on the GPU. (to evaluate)

Before starting the actual sync of a requested file the node which requested the file needs to send a MetaCommit to inform all other nodes who has been chosen as tracker. This is necessary so all torrent clients work over the same tracker for this file together.

Important is also that every MetaCommit is handled and for every received commit a sharing instance is started. This is needed for the very rare case where two nodes actually accomplish sending a MetaCommit for the same file at the same time which is theoretically possible but should be very rare. This solution to the problem should be given the rarity of this case the most efficient and avoids unnecessary difficult negotiations between the nodes which are requesting.

Also the case where a client which is in process of receiving a file gets a second MetaCommit for the same file should be handled somehow.

Why should not one client simply inform the other about the already ongoing sync?
Because we cant guaranty consistency between the nodes. If the two Commits are send at the exact same time it can happen that different nodes get different Commits first and start sharing before anybody realizes that there is a sync already initialized somewhere else.

We of course could implement some kind of reply voting algorithm. Maybe something like every possible tracker replies with a "ok" or "nok" to every MetaCommit and the one with more "ok"'s gets initialized while the other has to switch the meta file but this seems unnecessarily complicated and is A not needed now and B can be easily added later on top of this simple commit strategy if needed.

Another case which has to considered is the reaction of FileRequests while a MetaCommit is still not everywhere. Here again I tend to implementing a best effort strategy where nodes which allready have received the MetaCommit send a according response about an already ongoing sync.

The resulting downside is that it could happen that there are two sync circles in the same cluster for the same file. However the big benefit is that every node can still work without the need to rely on response from specific nodes.

Fix the PiePlate Integration Tests to work in separate JVMs.

Create an file Watcher to listen to file changes an system

The ShareService works at the time with a max timeout of 30s thishas to be fixed.
The idea is to set up a rerequest mechanism or to work without a timeout. However in the second case there should be some kind of global interrupt for the share process.

When changing the working dir we musst copy all the fils in the working dir to the new one.

Add first primitive File Service and File Merger to syn the file lists between all clients in one cloud (just sync list, do not copy the files)

Implement LAN based sharing between different pieShare users based on protocol.

Decide which technology to use for coping files.

clean up the existing cluster code and add further tests