ser94mor / cloudtieringfs Goto Github PK

Create a mechanism for moving data out. In this task implement only basic "move" policy which moves out data which had no accesses within, say, 10 minutes.

When user accesses "cold" file, it should be pulled back from object store to local store. Depends on #1 and #2.

• currently, there are no tests for ops.c file;
• existing tests may be expanded as well.

Utilize Linux extended attributes to store information about object in cloud storage that corresponds to file stub.

Take ideas from prototype tier-engine.c. Make multi-threaded application that maintains in and out queues of files to be uploaded/downloaded to/from cloud store. Queue should filled based on policies of file transfer between tiers.

License: GNU GPL version 3

Here is the report produced by pjd-fstest POSIX compliance test suite:

Test Summary Report
-------------------
/home/morozs/Workspace/cloudtiering/src/tst/pjd-fstest/tests/chmod/00.t   (Wstat: 0 Tests: 58 Failed: 1)
  Failed test:  39
/home/morozs/Workspace/cloudtiering/src/tst/pjd-fstest/tests/chown/00.t   (Wstat: 0 Tests: 171 Failed: 7)
  Failed tests:  84, 88, 141, 145, 149, 153, 157
/home/morozs/Workspace/cloudtiering/src/tst/pjd-fstest/tests/link/00.t    (Wstat: 0 Tests: 82 Failed: 1)
  Failed test:  71
/home/morozs/Workspace/cloudtiering/src/tst/pjd-fstest/tests/rename/00.t  (Wstat: 0 Tests: 79 Failed: 1)
  Failed test:  65
Files=184, Tests=1957, 127 wallclock secs ( 0.95 usr  0.21 sys + 14.11 cusr  2.76 csys = 18.03 CPU)
Result: FAIL

Issues should be analyzed and fixed.

Sergey,

Thankyou for you detailed email reply. My libs3 seem to be the version you suggest.

steve@steve:~/CloudTieringFS$ ls -l /usr/lib/libs3.so*
lrwxrwxrwx 1 root root    10 Sep 27  2017 /usr/lib/libs3.so -> libs3.so.2
lrwxrwxrwx 1 root root    12 Sep 27  2017 /usr/lib/libs3.so.2 -> libs3.so.2.0
-rw-r--r-- 1 root root 72344 Sep 27  2017 /usr/lib/libs3.so.2.0
steve@steve:~/CloudTieringFS$ 

Original email conversation is in the attachment.
Inbox _ [email protected] _ ProtonMail.pdf

"Cold" files should be evicted to cloud object storage. Depends on #1 .

Need to separate code:

• application code (something that is started from command line)
• shared library (majority of functions should be implemented there)
• test code (tests for application and library)

Hint: use processor time, not real time; measurements should have nanosecond precision.