paddlesteamer / cloudstash Goto Github PK
View Code? Open in Web Editor NEWPrivacy wrapper around cloud storage(s)
License: GNU General Public License v3.0
Privacy wrapper around cloud storage(s)
License: GNU General Public License v3.0
A validation is required before adding files, creating directories.
and get rid of warnings about extended attributes on macOS. A way to do this to create another table with key-value columns and connect it to the files
table with a foreign key. Or another way is to add a new xAttr
column to the files
table and use JSON encoding/decoding.
Now StatFS
barely returns information about the filesystem:
Lines 474 to 480 in 1e5acea
'not found' error implementation is needed in drive/drive.go for the sake of error checks like below:
also, the dropbox implementation should be converted to return 'not found' error where necessary.
check if the file is being downloaded already on fuse read
Remove directory doesn't work recursively:
cloudstash/internal/manager/manager.go
Lines 239 to 241 in 51107f3
Deleted files don't get deleted in the remote drive.
When a file is deleted, tracker isn't notified
The following calls to os.Remove()
are not being checked for errors:
cloudstash/cmd/cloudstash/main.go
Line 125 in 033e404
cloudstash/cmd/cloudstash/main.go
Line 134 in 033e404
cloudstash/cmd/cloudstash/main.go
Line 140 in 033e404
cloudstash/cmd/cloudstash/main.go
Line 176 in 033e404
cloudstash/cmd/cloudstash/main.go
Line 182 in 033e404
Implement a method that checks changes in cached files.
Ideas:
In the first run, if there is a database file, check if it is a valid database (does it have all tables?)
Add requirements and how to run section
Now with encryption, it's not easy to check changes with content hash since it's hash of encrypted content.
Need implementation using drive.Drive interface. Also drive.Drive interface may change due to differences between Dropbox and GDrive APIs.
Possible problems:
Change default log level to info
and add --debug
parameter for debug logs. When cloudstash is run with --debug
option:
debug
-d
optionNo need to download file everytime fs.Read is called
Go over the error messages and
CBC
CTR
If a file is opened and then immediately flushed it's not cached but updateMetadataFromCache()
is called on it. So we need to keep track of opened files and if any write operation is done on them
Should we change it to Standard Go Project Layout or should we continue with this old-school, c-like layout?
If the requested item expires between get and set, the file would be unreachable.
https://github.com/paddlesteamer/hdn-drv/blob/b7e487b49f40fe1a2c5839ee85f46a4d313f9c3a/internal/manager/manager.go#L211-L222
Low probability but can happen. Unfortunately go-cache doesn't support atomic operation for this. There is a PR though:
patrickmn/go-cache#96 but it doesn't look completely safe.
So we can implement that ourselves or risk it and use the code in the PR.
If the remote and local databases are changed at the same time, attempt to merge databases.
There should be rwlock in drive.Drive for network operations. For example, an upload operation shouldn't start when there is a download operation processing.
or don't
don't put plain text passphrase into config file
keep history of hmacs of files against old version attacks
There are two incomplete go libraries onedrive-go and go-onedrive so maybe implementing HTTP requests directly from API docs may be easier.
Onedrive API is very straightforward so it won't be a problem to implement limited subset of the API calls:
cloudstash/internal/drive/drive.go
Lines 15 to 45 in 285954f
Cleanup in handler:
nautilus (and probably other file explorers) create thumbnails by reading contents of image files which causes the download of all files. it may be fine but also there may be a more efficient solution to this.
before uploading local database file, check whether the remote file is changed. this should also apply for cached files.
If updated database upload fails, remove inserted lines and return fuse.EIO
A rwlock is needed for each file opened.
The package log
is very limited. Only has Print
, Fatal
, Panic
methods and I think the usage of Fatal
and Panic
should be avoided.
With fmt
, we get to separate stdin
and stderr
outputs but still, it would be nice to have more indicators on logs.
So I suggest using one of the logging libraries here. It would be nice to have debug
, info
, warning
and error
levels.
debug
is for verbose messages like the ones in the fs.go
:
Lines 27 to 29 in 928046f
info
should be used for information obviously :D. We don't have those in the code right now but I think we can use it for the messages like 'file x is uploaded to y'.
warning
should be used for handled errors. For example, the error here is handled:
cloudstash/internal/manager/manager.go
Lines 345 to 349 in 928046f
warning
where it returned.
error
is for fatal errors that shouldn't happen. There we can log as error
and do a clean exit (unmount, wait for ongoing network operations, etc.)
Add comments above exported functions
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.