Code Monkey home page Code Monkey logo

logseq-git-sync-101's Introduction

Logseq-Git-Sync-101

Description

This repo aims to help Logseq users to sync their data via Git and GitHub.

Credits

Credits to @danieltomasz1, @Sawhney2, @Caps3, @Abulafia4 @qwxlea5 for the workflow build up. And credits to @danzu6 for the incredicable diagram!

FYI, those IDs are IDs in Logseq Discord Server.


Why use Git + GitHub to sync Logseq graph?

Self Managed Sync Diagram, by @danzu6

From the above diagram, it's pretty obvious that Git is the most robust way to sync your graph. iCloud is slow and problematic, and Syncthing is not available on iOS/iPadOS.

However, Git is quite scary for non-programmers, so this doc is here to help!


Prerequisite

  1. GitHub account.
  2. Git, for computer users.
  3. Working Copy, for iOS/iPadOS users.
  4. Termux, for Andriod users.
What is Git? Git is a free and open source distributed version control system created by Linus Torvalds in 2005.
What is GitHub? GitHub is a for-profit company that offers a cloud-based Git repository hosting service. Essentially, it makes it a lot easier for individuals and teams to use Git for version control and collaboration.
How does Git works?


⚠️ Limitations of GitHub

  1. Recommend repositories remain small, ideally less than 1 GB, and less than 5 GB is strongly recommended.
  2. GitHub blocks pushes that exceed 100 MB, and you will receive a warning form Git when you attempt to add or update a file larger than 50 MB.
  3. Use Git Large File Storage (Git LFS), if you really need to track those big files with Git.

⚠️ Things you should avoid

  1. Open multiple instances of Logseq at a time, e.g. open Logseq on your computer and on your phone at the same time. This will cause conflict in your repo.
  2. Put large files in assets folder, like images, videos, etc.

💥 Rejected push & Conflicts handling

What is Rejected push?

Rejected push happens when the remote (eg, GitHub) contains modifications that you have yet to download. This can happen if you forgot to do a pull in your local version before commitng new changes to it.

What is Git Conflict?

Git conflict happens when you have two commits, one local and the other remote, which modify the same file in the same lines.

Why do I need to know how to solve rejected push & Git conflict?

Rejected push and Git conflict are something every Git users will meet eventually. It's important to know how to solve them.

Case study

For example, You type "I'm faithful to Logseq." in your journal on your pc, but you also type "Na, I also use other note-taking tools." in your journal on your phone at the same time. GitHub will accept the first commit you push to it. But when you push the second commit, Git will say something like:

error: failed to push some refs to 'github.com:{your-username}/{your-reponame}.git'
hint: Updates were rejected because the tip of your current branch is behind
hint: its remote counterpart. Integrate the remote changes (e.g.
hint: 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

which basicly means: "WTF Bruh, I can't believe you just lied to me. How can I trust you again?"

So you type git pull in your Git Bash/iTerms/Termux to gain GitHub's trust again, like what Git suggested in hint:. And that's how to resolve most of the rejected push.

However, sometimes that's not enough. After using git pull to update your local repo, Git will say:

Auto-merging {the conflict file} #e.g. journals/2022_04_23.md
CONFLICT (content): Merge conflict in {the conflict file}
Automatic merge failed; fix conflicts and then commit the result.

which means you have to manually delete which part you don't want.

So you open {the conflict file} with whatever text editor you like and fix it.

In the text editor:

<<<<<<< HEAD
- I’m faithful to Logseq. # Ok, I choose to keep this part.
=======
- Na, I also use other note-taking tools.
>>>>>>> 744f5cf94a46da43f5b318dab74c0f672bae31e2

After deletion:

- I’m faithful to Logseq.

Open Logseq and that let Logseq do the rest (Logseq will commit and push due to git hooks setting.) for you, and the Git conflict should be resolved. Now you should only see the chosen part remain in your GitHub repo. And you are again a happy Logseq user now! 😍

※ Noted: For Android users, you have to manually commit and push the changes since your workflow does not contain git hooks I provided.

※ Noted: Sometimes, it's the logseq/metadata.edn or logseq/pages-metadata.edn having Git conflits. This is trickier because you cannot tell easily which part is the one you need to keep. In this case I would suggest simply remove logseq/metadata.edn or logseq/pages-metadata.edn and do a git pull again, that will restore it from the one coming from GitHub. After pulling logseq/metadata.edn or logseq/pages-metadata.edn from GitHub, Re-index and Refresh Logseq are advised.


🪜 Workflow

Set up Git

This is like typing in the account/passward, so make sure you follow ALL the steps in set-up-git.

※ Noted: SSH-keys is prefered to use in this workflow, PLEASE set it properly, PLEASE. Connecting over SSH Section Link

Create a private repo

This is like creating a folder in GitHub, follow create-a-repo to create a private repo.

※ Noted: If you create a public repo, that means everyone on GitHub can see what you put in there. You don't want that, do you?

Add .gitignore in your repo (Optional)

In your repo page > Add file > Create new file > Name your file: .gitignore. Type those:

logseq/bak/
logseq/.recycle

Download this repo

Click the green Code button and Download ZIP, then unzip it for later use.

For Windows users

After Git is set and a private repo is created:

  1. Go to your local drive (for example: D:\) and right-click on it.
  2. You should see Git Bash Here, click it.
  3. Type git clone [email protected]:{your-username}/{your-reponame}.git and hit enter.
  4. If it's your first time git clone something from GitHub, it will probably ask if you agree to authorized the connection, just type Yes and hit enter.
  5. After it's done, you should see a new folder with your repo name.
  6. Open the folder, there should have a hidden folder named .git.
  7. Copy&paste the post-commit and pre-commit in .git/hooks. (Note: this folder could be hidden)
  8. Open Logseq and add the folder with .git as your new graph.
  9. Open Logseq > Settings > Version control > toggle on "Enable Git auto commit".
  10. Type something and wait few minutes to see if what you typed has also appear in GitHub.
  11. If nothiong goes wrong, you are a happy Logseq user!

For MacOS users

After Git is set and a private repo is created:

  1. Go to your local drive and right-click on it.

  2. You should see New Terminal at folder, click it. If you don't see this option, check this link to enable it.

  3. Type git clone [email protected]:{your-username}/{your-reponame}.git and hit enter.

  4. If it's your first time git clone something from GitHub, it will probably ask if you agree to authorized the connection, just type Yes and hit enter.

  5. After it's done, you should see a new folder with your repo name.

  6. Open the folder, there should have a hidden folder named .git.

  7. Copy&paste the post-commit and pre-commit in .git/hooks.

  8. Right click in .git/hooks and click New Terminal at folder, type

    chmod +x ./pre-commit && chmod +x ./post-commit

    to make those files executable by MacOS.

  9. Open Logseq and add the folder with .git as your new graph.

  10. Open Logseq > Settings > Version control > toggle on "Enable Git auto commit".

  11. Type something and wait few minutes to see if what you typed has also appear in GitHub.

  12. If nothing goes wrong, you are a happy Logseq user!

For iOS/iPadOS users

  1. Pay for the Pro version of the App Working Copy, it is cheap since you will never bother by iCloud crashing your Logseq anymore.
  2. Open Working Copy > Seetings(⚙️) > Hosting Providers > Click Test > Sign-in to your GitHub account.
  3. Open Working Copy > click + > Clone repository > Choose your repo.
  4. When download is done, long-press on the repo > Share > Link Repository to Folder > My iPhone/iPad (I would suggest don't choose iCloud) > Logseq-icon folder > create a new sub-folder under the Logseq-icon one (I would suggest don't name the folder with the same name as your repo.) > click Done
  5. Open Logseq and add new graph to that folder.
  6. Open ShortCuts > Automation > Create Personal Automation:
    • On App Open:
      • App > choose Logseq > click Next
      • Search actions > Pull Repository - Repo: your Logseq repo
      • Turn off ask before running (or leave it on up to you)
    • On App Close:
      • App > choose Logseq > click Next
      • Search actions >
        • Stage for Commit - Path: *, Repo: your Logseq repo
        • Commit Repository - Repo: your Logseq repo, Message: Auto-commit from iOS/iPadOS
        • Noted: In Commit Repository, you can toggle off Fail when nothing to Commit if you find the failure notification annoying. Although I prefer to keep it toggle on to prevent empty commit and empty push.
        • Push Repository - Repo: your Logseq repo
      • Turn off ask before running (or leave it on up to you)
  7. Type something, exit Logseq, and wait few minutes to see if what you typed has also appear in GitHub.
  8. If nothiong goes wrong, you are a happy Logseq user!

※ Noted: If you are a student, you can sign up github student pack and apply for free Working Copy Pro.

For Android users, by @Abulafia4

Initial steps to install git and link it with logseq

  1. Install Termux in Android, using the F-droid app. Follow instructions here.

  2. Open Termux and type pkg install git to install Git.

  3. Configure Git username and email:

    git config --global user.name "John Doe"
    git config --global user.email [email protected]`

    ※ Noted: This is the name and email which will go into commits, it can be anything, not necessary related to your GitHub account.

  4. In the Termux terminal, create a folder for your graph, for example cd documents; mkdir MyGraph and clone your repo there with the command git clone [email protected]:{your-username}/{your-reponame}.git ~/documents/MyGraph

  5. Open Logseq and add a graph on that folder. Check that all looks ok.

Install shortcuts to synchronize the repository

  1. Install Termux:Widgets from F-droid.

  2. In Termux, type cd ~/.shortucts and edit a file named, for example, pull-graph containing:

    #!/usr/bin/bash
    source bin/source-ssh-agent
    cd {your repo location}  # eg: cd ~/documents/MyGraph
    git pull

    ※ Noted: If you use password-free private keys for Git you don't need the ssh-agent line. You can include it to avoid typing the private key passphrase each time.

  3. In the same ~/.shortcuts folder create a second one named, for example, pull-graph contaning:

    #!/usr/bin/bash
    source bin/source-ssh-agent
    cd {your repo location}
    git add -A
    git commit -m "sync from android"
    git push
  4. Add a widget for Termux Widget app to the Android screen, which will show the scripts available in ~/.shortcuts (in our example, it will show pull-graph and push-graph). Taping on them, they will be executed by Termux.

Android-Termux-Workflow

  • Tap pull-graph to download the most recent version of your graph from GitHub.
  • Open Logseq and admire the latest version of your notes.
  • Edit/add notes, work on them.
  • Exit Logseq and tap push-graph to upload your changes to GitHub.

For Linux users

Linux users click here If you are a Linux user, why are you still watching this repo? Shooo. 😎

References

Footnotes

  1. danieltomasz#5748 2

  2. Sawhney#7320 2

  3. Caps#0219 2

  4. Abulafia#3734 2 3

  5. qwxlea#3490 2

  6. danzu#6567 2

logseq-git-sync-101's People

Contributors

abul4fia avatar charleschiugit avatar hdansou avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.