Code Monkey home page Code Monkey logo

dotdrop's Introduction

dotdrop



GitHub release (latest by date) License: GPL v3

Tests Status Doc Status codecov CodeQL

Python

PyPI Homebrew version AUR Snap

Donate

Save your dotfiles once, deploy them everywhere

Dotdrop makes the management of dotfiles between different hosts easy. It allows you to store your dotfiles in Git and automagically deploy different versions of the same file on different setups.

It also allows to manage different sets of dotfiles. For example, you can have a set of dotfiles for your home laptop and a different set for your office desktop. Those sets may overlap, and different versions of the same dotfiles can be deployed using different predefined profiles. Or you may have a main set of dotfiles for your everyday host and a subset you only need to deploy to temporary hosts (cloud VM etc.) that may be using a slightly different version of some of the dotfiles.

Features:

  • Sync once every dotfile in Git for different usages
  • Allow dotfile templating
  • Dynamically generated dotfile contents with pre-defined variables
  • Comparison between deployed and stored dotfiles
  • Handling multiple profiles with different sets of dotfiles
  • Easily import and update dotfiles
  • Handle files and directories
  • Support symlinking of dotfiles
  • Associate actions to the deployment of specific dotfiles
  • Associate transformations for storing encrypted/compressed dotfiles
  • Provide solutions for handling dotfiles containing sensitive information

Check the example, the documentation or how people are using dotdrop for more.

Quick start:

## using dotdrop as a submodule
mkdir dotfiles && cd dotfiles
git init
git submodule add https://github.com/deadc0de6/dotdrop.git
pip3 install -r dotdrop/requirements.txt --user
./dotdrop/bootstrap.sh
./dotdrop.sh --help

A mirror of this repository is available on GitLab under https://gitlab.com/deadc0de6/dotdrop.

Why dotdrop?

There exist many tools to manage dotfiles; however, not many allow to deploy different versions of the same dotfile on different hosts. Moreover, dotdrop allows to specify the set of dotfiles that need to be deployed for a specific profile.

See the example for a concrete example of why dotdrop rocks.


Table of Contents

Installation

See the installation instructions.

Dotdrop is available on:

Getting started

Create a new repository to store your dotfiles with dotdrop. Init or clone that new repository and install dotdrop.

Then import any dotfiles (files or directories) you want to manage with dotdrop. You can either use the default profile (which resolves to the hostname of the host you are running dotdrop on) or provide it explicitly using the switch -p/--profile.

Import dotfiles on host home:

$ dotdrop import ~/.vimrc ~/.xinitrc ~/.config/polybar

Dotdrop does two things:

  • Copy the dotfiles to the dotpath directory (defined in config.yaml, defaults to dotfiles)
  • Create the associated entries in the config.yaml file (in the dotfiles and profiles entries)

Your config file will look like something similar to this:

config:
  backup: true
  banner: true
  create: true
  dotpath: dotfiles
  ignoreempty: false
  keepdot: false
  longkey: false
  showdiff: false
  workdir: ~/.config/dotdrop
dotfiles:
  d_polybar:
    dst: ~/.config/polybar
    src: config/polybar
  f_vimrc:
    dst: ~/.vimrc
    src: vimrc
  f_xinitrc:
    dst: ~/.xinitrc
    src: xinitrc
profiles:
  home:
    dotfiles:
    - f_vimrc
    - f_xinitrc
    - d_polybar

For a description of the different fields and their use, see the config doc.

Commit and push your changes with git.

Then go to another host where your dotfiles need to be managed as well, clone the previously set up repository, and compare the local dotfiles with the ones stored in dotdrop:

$ dotdrop compare --profile=home

Now you might want to adapt the config.yaml file to your liking on that second host. Let's say, for example, that you only want d_polybar and f_xinitrc to be deployed on that second host. You would then change your config to something like this (assuming that the second host's hostname is office):


profiles:
  home:
    dotfiles:
    - f_vimrc
    - f_xinitrc
    - d_polybar
  office:
    dotfiles:
    - f_xinitrc
    - d_polybar

Then adapt any dotfile using the templating feature (if needed). For example, you might want different fonts sizes in Polybar for each host.

Edit <dotpath>/config/polybar/config:

…
{%@@ if profile == "home" @@%}
font0 = sans:size=10;0
{%@@ elif profile == "office" @@%}
font0 = sans:size=14;0
{%@@ endif @@%}
font1 = "Material Design Icons:style=Regular:size=14;0"
font2 = "unifont:size=6;0"

You also want to have the correct interface set on the wireless network in the Polybar config.

Add a variable to the config file (In the below example, home gets the default wlan0 value for the variable wifi while office gets wlp2s0):


variables:
  wifi: "wlan0"

profiles:
  home:
    dotfiles:
    - f_vimrc
    - f_xinitrc
    - d_polybar
  office:
    dotfiles:
    - f_xinitrc
    - d_polybar
    variables:
      wifi: "wlp2s0"

Then you can adapt the Polybar config file so that the variable wifi gets correctly replaced during installation:

[module/wireless-network]
type = internal/network
interface = {{@@ wifi @@}}

Also, the home computer is running awesomeWM, and the office computer bspwm. The ~/.xinitrc file will therefore be different while still sharing some lines.

Edit <dotpath>/xinitrc:

#!/bin/bash

# load Xresources
userresources=$HOME/.Xresources
if [ -f "$userresources" ]; then
      xrdb -merge "$userresources" &
fi

# launch the wm
{%@@ if profile == "home" @@%}
exec awesome
{%@@ elif profile == "office" @@%}
exec bspwm
{%@@ endif @@%}

Finally you want everything installed with the office profile to be logged; you thus add an action to the config file:


actions:
  loginstall: "echo {{@@ _dotfile_abs_src @@}} installed to {{@@ _dotfile_abs_dst @@}} >> {0}"

profiles:
  home:
    dotfiles:
    - f_vimrc
    - f_xinitrc
    - d_polybar
  office:
    dotfiles:
    - f_xinitrc
    - d_polybar
    variables:
      wifi: "wlp2s0"
    actions:
      - loginstall "/tmp/dotdrop-installation.log"

When done, you can install your dotfiles using:

$ dotdrop install

If you are unsure, you can always run dotdrop compare to see how your local dotfiles would be updated by dotdrop before running install or you could run install with --dry.

That's it, a single repository with all your dotfiles for your different hosts.

For more, see the docs:

Documentation

Dotdrop's documentation is hosted on Read the Docs.

Thank you

If you like dotdrop, buy me a coffee.

Contribution

If you are having trouble installing or using dotdrop, open an issue.

If you want to contribute, feel free to do a PR (please follow PEP8). Have a look at the contribution guidelines.

License

This project is licensed under the terms of the GPLv3 license.

dotdrop's People

Contributors

belugame avatar brokenpip3 avatar cat-master21 avatar davla avatar deadc0de6 avatar dotiful avatar japorized avatar jtt9340 avatar jwodder avatar laggardkernel avatar lelgenio avatar lyze237 avatar maelan avatar maetveis avatar marcelrobitaille avatar marczz avatar maxchaos avatar moyiz avatar open-dynamix avatar parona-source avatar ranjanashish avatar richboss avatar roachsinai avatar s0undt3ch avatar samueloph avatar samwhitlock avatar sclu1034 avatar severen avatar sighery avatar ziirish avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dotdrop's Issues

The second alias in the README for using env vars does not work

Actually there are two issues. Just tell me if I should open them separately.

  1. When trying to execute dotdrop after eval, the shell just hangs. I've tested this with bash and zsh. It seems to be some kind of recursion problem, but I didn't look into it a lot. When calling the alias differently (e.g. dotdrop2) it works without an issue. I was able to work around this using the full path (/usr/bin/dotdrop). Obviously this is not an issue when using dotdrop as a submodule.

  2. ~ is not resolved when using the --cfg argument.

$ dotdrop --cfg=~/dotfiles/config.yaml compare
[ERR] error: config file does not exist

My working alias looks like this now:

alias dotdrop='eval $(cat ~/dotfiles/env.defaults ~/dotfiles/.env | grep -v "^#") /usr/bin/dotdrop --cfg=${HOME}/dotfiles/config.yaml'

Feature: Action Argument(s)

Hey!
First thanks for this great tool with its awesome features.
I'd just found this here and being around to switch over. I like the idea to use this for installing the respective packages as well. So for example installing NeoVim as pre action for the nvim dotfiles. This lead to a big bunch of actions when have to add so for each package with (a) dotfile(s). But most of them are pretty the same, cause just the package name changes. So for example if I'm working with ArchLinux and like to install nvim and nmutt I do it this way:

actions:
  pre:
    install_nvim: pacman -Sy neovim
    install_nmutt: pacman -Sy neomutt
...
dotfiles:
  f_nvim:
    actions:
      - install_nvim
    ...
  f_nmutt:
    actions:
      - install_nmutt
    ...

I think the "problem" is obvious. So what about the idea using arguments for actions. I'm not aware about the implementation but I imagine it in a way like this:

actions:
  pre:
    install_package: pacman -Sy ${0}
...
dotfiles:
  f_nvim:
    actions:
      - install_package neovim
    ...
  f_nmutt:
    actions:
      - install_package neomutt
    ...

That would lead to a cleaner configuration and make changes easier cause of the single point of adjustment.
Thanks for listening!

[Feature request] Git integration

Right now I have a couple of zsh aliases to speed up using git on my dotdrop repository:

alias dotdrop='~/git/dotfiles/dotdrop.sh'
alias dotgit='git -C ~/git/dotfiles'
alias dotsync='dotgit pull; dotgit add -A; dotgit commit; dotgit push; dotdrop install'

Might not be the best naming scheme but the idea is that when I edit a dotfile on any of my pc's, I can run dotsync on the originating pc and write a commit message. Then I can just run dotsync again everywhere else and it automagically syncs and installs everything with a grand total of two commands.

I think end users would really benefit from git integration like this. I have no idea how I would start implementing something like this but I'm just putting this on the table for the contributors to consider.

Wildcard hostname?

In addition to my physical machines, I have a lot of VM's which I have to periodically work on. It would be nice to have a wildcard hostname entry in the configuration, so that if I'm trying to install on a machine that doesn't have a known hostname, it would install only my most basic dotfiles like vimrc and zshrc. Is this possible right now?

[Feature request] Way to deactivate the banner

Hi,

imho it would be great to have a way to deactivate the banner on every execution of the script. Especially at the beginning when you start to use dotdrop it's quite noisy ;)

Maybe just add a config flag which controls whether the banner gets drawn or not?

Best regards,
Felix

Symlinked dotfiles with templating are directly linked

The README states that symlinked dotfiles with templating are first installed into ~/.config/dotdrop and then linked from there. Instead dotdrop seems to just link the files directly without templating. Looking through the source code, it seems like some steps are definitely missing (e.g workdir is never used as far as I can tell).

Running `dotdrop compare` after import is confused by Windows line endings

I have some historical files in my homedir that are from my Windows days (don't judge me :-)) and they still have line-feed + carriage-return line endings.

For reason, after dotdrop import and then a dotdrop compare with zero changes, all of my files that had LF+CR line endings are shown as dirty, with the diff including the entire "all lines with LF deleted" + "all lines with LF+CR added".

[feature request] check for file existence

Hi :)

I was wanting to check for a file existence, in order to source a profile in my zshrc, such as:

{%@@ if_exists[$(which kubectl)] @@%}                                                                                                                                                   
source <(kubectl completion zsh)
{%@@ endif @@%}                                                                                                                                                   

As I understand, jinja2 templating does not have this capability.

Anyways, just a shot in the dark ;-)

Thanks for the tool! Loving it!

longkey option and space in path

Hey

Let's see if this is something real ;-)

Wanting to dotdrop my VS Studio settings.json file. Works fine with longkeyto false. But fails with the below if longkeyis true. Is space the culprit?

$ dotdrop import $HOME/Library/Application\ Support/Code/User/settings.json                                                                                                                            INSERT    22:20:12 
     _       _      _
  __| | ___ | |_ __| |_ __ ___  _ __
 / _` |/ _ \| __/ _` | '__/ _ \| '_ |
 \__,_|\___/ \__\__,_|_|  \___/| .__/  v0.23.0
                               |_|

[ERR] "cp -R -L /Users/fblaise/Library/Application Support/Code/User/settings.json /Users/fblaise/gitrepos/dotfiles/dotfiles/Library/Application Support/Code/User/settings.json" returned non zero (1): cp: /Users/fblaise/Library/Application Support/Code/User/settings.json: No such file or directory
[ERR] importing "/Users/fblaise/Library/Application Support/Code/User/settings.json" failed!

0 file(s) imported.

Error when updating files with a tilde in its name

I've just installed dotdrop today and found an error on my system when trying to update a file that was registered with a tilde in its path.

I'm using Ubuntu 18.04 LTS and the default python3 (3.6). This is what it happens when trying to update:

$ python --version
Python 3.6.5

$ ./dotdrop.sh update ~/.bashrc
     _       _      _
  __| | ___ | |_ __| |_ __ ___  _ __
 / _` |/ _ \| __/ _` | '__/ _ \| '_ |
 \__,_|\___/ \__\__,_|_|  \___/| .__/  v0.19.1
                               |_|

Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/enriqueg/dotfiles/dotdrop/dotdrop/dotdrop.py", line 385, in <module>
    if main():
  File "/home/enriqueg/dotfiles/dotdrop/dotdrop/dotdrop.py", line 375, in main
    update(opts, conf, args['<paths>'])
  File "/home/enriqueg/dotfiles/dotdrop/dotdrop/dotdrop.py", line 236, in update
    updater.update(path, opts['profile'])
  File "/home/enriqueg/dotfiles/dotdrop/dotdrop/updater.py", line 73, in update
    return self._handle_file(left, right)
  File "/home/enriqueg/dotfiles/dotdrop/dotdrop/updater.py", line 87, in _handle_file
    if compare and filecmp.cmp(left, right, shallow=True):
  File "/usr/lib/python3.6/filecmp.py", line 51, in cmp
    s1 = _sig(os.stat(f1))
FileNotFoundError: [Errno 2] No such file or directory: '~/.bashrc'

I've managed to solve it by expanding the tildes before comparing the files in the update() function of updater.py:

    def update(self, path, profile):
        """update the dotfile installed on path"""
        if not os.path.lexists(path):
            self.log.err('\"{}\" does not exist!'.format(path))
            return False
        left = self._normalize(path)
        dotfile = self._get_dotfile(left, profile)
        if not dotfile:
            return False
        if self.debug:
            self.log.dbg('updating {} from {}'.format(dotfile, path))

        right = os.path.join(self.conf.abs_dotpath(self.dotpath), dotfile.src)
        ## These two lines fix the error
        ## ---- BEGIN ----
        left = os.path.expanduser(left)
        right = os.path.expanduser(right)
        ## ---- END ----
        # go through all files and update
        if os.path.isdir(path):
            return self._handle_dir(left, right)
        return self._handle_file(left, right)

Symlink function on macos broken

When I set a file or folder to use links like this:

f_config.fish:
     dst: ~/.config/fish/config.fish
     src: config/fish/config.fish
     link: true

And I install it normally ./dotdrop.sh --profile=mac

It generates a symlink to ./dotfiles/config/fish/config.sh instead of the actual path <DotfilesFolder>/dotfiles/config/fish/config.sh

Templating a file inside a symlinked parent folder

I have a folder with files all related to configuring my mutt with multiple accounts. The general rc, contacts, signatures, theme, and several account files that also hold the passwords etc.

Now, initially I used dotdrop to link the whole folder and then tried to use env vars to replace the credentials in the nested account files.. but that does not work as it is already symlinked by the parent folder...

This won't work:

~/.mutt:
    src: mutt
    dst: ~/.mutt
    link: true
~/.mutt/accounts/myaccount.offlineimap:
    src: mutt/accounts/myaccount.offlineimap
    dst: ~/.mutt/accounts/myaccount.offlineimap
    link: true

Now I could link all file by file, but is there a smarter way to link the whole bunch but template a few?

Managing the same dotfile location but for different profiles

I find myself doing the following in config.yml

dotfiles:
  f_zshrc_host1:
    dst: ~/.zshrc
    src: zshrc_host1
  f_zshrc_host2:
    dst: ~/.zshrc
    src: zshrc_host2
profiles:
  host1:
    dotfiles:
    - f_zshrc_host1
  host2:
    dotfiles:
    - f_zshrc_host2

So in essence, I wanted to manage the same location for my .zshrc, but for different hosts and by extension different configurations.

When I did dotdrop import -p host1 ~/.zshrc, it creates a dotfile with the name zshrc. Then when I did the same for my other host (dotdrop import -p host2 ~/.zshrc), it will simply default to sharing that file for both hosts. This is not what I want.

To accomplish what I wanted, I have created separate files for both hosts and changed the config accordingly. Am I missing a command for doing this, or do I have to do this manually each time? I feel like this should be part of the basic functionality.

I have no problem doing it manually seeing as it still works when I install the files, but I was wondering if I am missing a built-in way for doing this.

Thanks

dotfiles execution in reverse order

Is there a reason why the dotfiles are returned in reverse order?
I was configuring my dotfiles in the scheme XX_f_filename where XX is a two-digit-number, now the highest value gets executed first, which is a very uncommon behaviour.
I fixed the issue for myself. Wouldn't it be logic to make this beaviour default?
I can submit my change as a pull-request if you want.

Running an action before a dotfile is installed

Hi, I'm struggling to use dotdrop for my emacs setup, I use the following configuration helper, which is required for my setup: https://github.com/bbatsov/prelude.

The installer for that helper does the following:

  1. Move ~/.emacs.d/ (the relevant dotfile dir) to ~/.emacs.d.bak/
  2. Install a whole bunch of new files in ~/.emacs.d/

Obviously this means that any dotfiles dotdrop attempts to install are overwritten immediately when it runs an install action for this.

Copying files into this directory immediately after the installer ran would solve this issue, is this possible, or would you accept a PR for this?

Updating a directory copies destination to wrong path

Hi there.
I have been using dotdrop for quite some time now and I really love it.

Since last month's commit, I have started tracking whole directories and noticed that using dotdrop to update a directory whose source's and destination's base-names are different will result in a new copy inside the dotfiles directory.

For example, let's say that I track the directory ~/foo.d which is stored in ~/.dotfiles/bar.d (according to my config.yaml). Running:
dotdrop.sh update ~/foo.d
will copy that directory to ~/.dotfiles/foo.d instead of ~/.dotfiles/bar.d.
The reason for this seems to be the current implementation of update which actually executes:
['cp', '-R', '-L', '/home/maxchaos/foo.d', '/home/maxchaos/.dotfiles/foo.d/..']

I guess an easy fix to this issue would be passing the option -T to cp like that:
['cp', '-R', '-L', '-T', '/home/maxchaos/foo.d', '/home/maxchaos/.dotfiles/bar.d']

Any ideas?
Thanks in advance.

Consider downgrading log level of "link exists" on `install`

When running dotdrop.sh install, I have ~200 files, all currently symlinks, and so I get a lot of output like:

  -> ignoring "/home/stephen/.local/share/fonts/SourceCodePro-Black.ttf", link exists
  -> ignoring "/home/stephen/.local/share/fonts/SourceCodePro-Bold.ttf", link exists
  -> ignoring "/home/stephen/.local/share/fonts/SourceCodePro-ExtraLight.ttf", link exists
  -> ignoring "/home/stephen/.local/share/fonts/SourceCodePro-Light.ttf", link exists
  -> ignoring "/home/stephen/.local/share/fonts/SourceCodePro-Medium.ttf", link exists
  -> ignoring "/home/stephen/.local/share/fonts/SourceCodePro-Regular.ttf", link exists
  -> ignoring "/home/stephen/.local/share/fonts/SourceCodePro-Semibold.ttf", link exists

Given this is a no-op, seems like this could be downgraded to a debug / verbose-only statement.

Should I always use absolute path when import dotfiles?

Today when I install dotfiles I get error:

➜  dotfiles git:(master) ✗ /usr/bin/dotdrop --cfg=~/dotfiles/config.yaml install  
     _       _      _
  __| | ___ | |_ __| |_ __ ___  _ __
 / _` |/ _ \| __/ _` | '__/ _ \| '_ |
 \__,_|\___/ \__\__,_|_|  \___/| .__/  v0.20.1
                               |_|

Traceback (most recent call last):
  File "/usr/bin/dotdrop", line 11, in <module>
    load_entry_point('dotdrop==0.20.1', 'console_scripts', 'dotdrop')()
  File "/usr/lib/python3.7/site-packages/dotdrop/__init__.py", line 11, in main
    if dotdrop.dotdrop.main():
  File "/usr/lib/python3.7/site-packages/dotdrop/dotdrop.py", line 369, in main
    ret = install(opts, conf, temporary=args['--temp'])
  File "/usr/lib/python3.7/site-packages/dotdrop/dotdrop.py", line 116, in install
    r = inst.install(t, src, dotfile.dst)
  File "/usr/lib/python3.7/site-packages/dotdrop/installer.py", line 50, in install
    return self._handle_file(templater, src, dst)
  File "/usr/lib/python3.7/site-packages/dotdrop/installer.py", line 116, in _handle_file
    ret = self._write(dst, content, st.st_mode)
  File "/usr/lib/python3.7/site-packages/dotdrop/installer.py", line 172, in _write
    if not self._create_dirs(base):
  File "/usr/lib/python3.7/site-packages/dotdrop/installer.py", line 197, in _create_dirs
    os.makedirs(directory)
  File "/usr/lib/python3.7/os.py", line 221, in makedirs
    mkdir(name, mode)
FileNotFoundError: [Errno 2] No such file or directory: ''

But, if I use /usr/bin/dotdrop --cfg=~/dotfiles/config.yaml install -t they are good, no errors.

Finally, I trace it use ipdb and find I import a file name like .to_import.sh directory,

➜  ~ ls | grep import
.to_import.sh
➜ dotdrop import .to_import.sh
...

and then when I install dotfiles I get above error when I'm not in dir ~.


There are two palces found absolute in README.md. But, it doesn't specific that I have to use absolute path when import dotfiles(I thought this great tool ,dotdrop, should do it before, lol). Your import examples is good.

So I think it's good to specific it at README.md.

Enable profiles which are extensions of other profiles

Hi, this is a feature suggestion, not an issue.
Could it be useful to have a feature where a user can define a profile as an extension of another. For example, imagine that I define a "basic" profile with the config files that I need everywhere, and then I define a "work" profile, with all the config files of basic, plus some extra ones.

Right now I would have to maintain two separate and partially duplicate lists in the config.yaml files. With such an approach the redundancy would be removed.

Thanks for the awesome program! It is so convenient!

provide the ability to use long keys

provide users with the ability to use long keys (mirroring the abs path to the dotfile, instead of the shorten version).

  • entry in the config file
  • ability to rename existing short keys

see #38

Support uninstall

Currently, dotdrop works very well! However, I think it would be better if uninstall is supported.

Thanks!

Ignoring files in the configuration file

Hello,

i have a few use cases where folders I imported in dotdrop contain temporary files. Those temporary files show up, every time I use "dotdrop compare"

I would be happy to have a configuration option to specify some files and folders to exclude in the diff output. Basically exactly what the switch "-i --ignore=" does but with more comfort.
Yes, I could make an alias specifying the ignore switch. But it would be a lot better in my opinion to keep everything a the same place: the config.yaml.

A little example: I use persist ssh session to log into ssh faster. This leads to socket files inside a specified directory. They show up like this:

~> dotdrop compare
     _       _      _
  __| | ___ | |_ __| |_ __ ___  _ __
 / _` |/ _ \| __/ _` | '__/ _ \| '_ |
 \__,_|\___/ \__\__,_|_|  \___/| .__/  v0.19.2
                               |_|

diffing "d_sockets" VS "~/.ssh/sockets"
only in right: "user@server"
only in right: "user@server"
only in right: "user@server"
only in right: "user@server"
...

Thanks and Regards

Richard

Installing to a non-existing directory throws a stacktrace

While installing a dotfile as a symlink from ~/.config on a fresh machine without that directory, I got a stacktrace with a FileNotFoundError. The cause seems to have been the missing ~/.config directory.

I think a user would expect either of:

  1. A nice error message telling me that I should create the parent directories of any dotfiles I'm installing.
  2. Dotdrop creates the relevant directory for me, automagically.

Install doesn't create directories for symlinks

Hello!

I get the following crash when I tried to install my dotfiles on a new machine:

~/t/dotfiles OvO/"/ ./dotdrop.sh install                                                                                                        master\"\^v^/"/?+
     _       _      _
  __| | ___ | |_ __| |_ __ ___  _ __
 / _` |/ _ \| __/ _` | '__/ _ \| '_ |
 \__,_|\___/ \__\__,_|_|  \___/| .__/  v0.10.4
                               |_|

Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.6.4_3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/Cellar/python/3.6.4_3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/wml/tmp/dotfiles/dotdrop/dotdrop/dotdrop.py", line 346, in <module>
    if main():
  File "/Users/wml/tmp/dotfiles/dotdrop/dotdrop/dotdrop.py", line 318, in main
    ret = install(opts, conf)
  File "/Users/wml/tmp/dotfiles/dotdrop/dotdrop/dotdrop.py", line 90, in install
    r = inst.link(dotfile.src, dotfile.dst)
  File "/Users/wml/tmp/dotfiles/dotdrop/dotdrop/installer.py", line 62, in link
    os.symlink(src, dst)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/wml/tmp/dotfiles/dotfiles/tmp/config/test' -> '/Users/wml/tmp/config/test'
~/t/dotfiles OvO/"/ ./dotdrop.sh install --dry                                                                                                  master\"\^v^/"/?+
     _       _      _
  __| | ___ | |_ __| |_ __ ___  _ __
 / _` |/ _ \| __/ _` | '__/ _ \| '_ |
 \__,_|\___/ \__\__,_|_|  \___/| .__/  v0.10.4
                               |_|

[DRY] would link /Users/wml/tmp/config/test to /Users/wml/tmp/dotfiles/dotfiles/tmp/config/test 

0 dotfile(s) installed.

Here's the way I did it:

~ OvO/"/ mkdir ~/tmp
~ OvO/"/ cd ~/tmp
~/tmp OvO/"/ mkdir dotfiles
~/tmp OvO/"/ cd dotfiles/
~/t/dotfiles OvO/"/ git init
Initialized empty Git repository in /Users/wml/tmp/dotfiles/.git/
~/t/dotfiles OvO/"/ git submodule add https://github.com/deadc0de6/dotdrop.git                                                                       master\"\^v^
Cloning into '/Users/wml/tmp/dotfiles/dotdrop'...
remote: Counting objects: 954, done.
remote: Compressing objects: 100% (42/42), done.
remote: Total 954 (delta 29), reused 30 (delta 14), pack-reused 898
Receiving objects: 100% (954/954), 184.85 KiB | 267.00 KiB/s, done.
Resolving deltas: 100% (621/621), done.
~/t/dotfiles OvO/"/ ./dotdrop/bootstrap.sh                                                                                                       master\"\^v^/"/+
~/t/dotfiles OvO/"/ mkdir ../config                                                                                                             master\"\^v^/"/?+
~/t/dotfiles OvO/"/ echo hi > ../config/test                                                                                                    master\"\^v^/"/?+
~/t/dotfiles OvO/"/ ./dotdrop.sh import -l ~/tmp/config/test                                                                                    master\"\^v^/"/?+
     _       _      _
  __| | ___ | |_ __| |_ __ ___  _ __
 / _` |/ _ \| __/ _` | '__/ _ \| '_ |
 \__,_|\___/ \__\__,_|_|  \___/| .__/  v0.10.4
                               |_|

        -> "/Users/wml/tmp/config/test" imported

1 file(s) imported.
~/t/dotfiles OvO/"/ rm -r ../config   #simulate new install                                                                         master\"\^v^/"/?+
~/t/dotfiles OvO/"/ ./dotdrop.sh install                                                                                                        master\"\^v^/"/?+
     _       _      _
  __| | ___ | |_ __| |_ __ ___  _ __
 / _` |/ _ \| __/ _` | '__/ _ \| '_ |
 \__,_|\___/ \__\__,_|_|  \___/| .__/  v0.10.4
                               |_|

Traceback (most recent call last):
  File "/usr/local/Cellar/python/3.6.4_3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/Cellar/python/3.6.4_3/Frameworks/Python.framework/Versions/3.6/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/wml/tmp/dotfiles/dotdrop/dotdrop/dotdrop.py", line 346, in <module>
    if main():
  File "/Users/wml/tmp/dotfiles/dotdrop/dotdrop/dotdrop.py", line 318, in main
    ret = install(opts, conf)
  File "/Users/wml/tmp/dotfiles/dotdrop/dotdrop/dotdrop.py", line 90, in install
    r = inst.link(dotfile.src, dotfile.dst)
  File "/Users/wml/tmp/dotfiles/dotdrop/dotdrop/installer.py", line 62, in link
    os.symlink(src, dst)
FileNotFoundError: [Errno 2] No such file or directory: '/Users/wml/tmp/dotfiles/dotfiles/tmp/config/test' -> '/Users/wml/tmp/config/test'

And here's the test yaml it generated:

~/t/dotfiles OvO/"/ cat config.yaml                                                                                                             master\"\^v^/"/?+
config:
  backup: true
  create: true
  dotpath: dotfiles
dotfiles:
  f_test:
    dst: ~/tmp/config/test
    link: true
    src: tmp/config/test
profiles:
  3dboyss-MBP:
    dotfiles:
    - f_test

I tried it with: antergos, macos high sierra


Here's the line where it crashes: https://github.com/deadc0de6/dotdrop/blob/master/dotdrop/installer.py#L62

        if self.dry:
            self.log.dry('would link %s to %s' % (dst, src))
            return []
        os.symlink(src, dst)
        self.log.sub('linked %s to %s' % (dst, src))

How to update tracked dotfiles?

Thanks for the tool first of all!

Now, my question is about how to sync updated files with dotdrop. Say I have a profile, 'laptop', and I locally change, say, '~/.vimrc'. There doesn't seem to be a way to update the dotfiles managed by dotdrop. How do I sync them? I can't call import again on them again, but there is no update command as far as I can tell.

Pass options to diff

Would be nice to be able to pass options to diff when using the compare command.

Allow not dropping `.` prefix when importing

I have an existing homedir with files like .vimrc and I'd like to run dotdrop import .vimrc to populate the config.yml file. (I'm technically running git ls-files | xargs dotdrop import to mass import my existing repo.)

Since I have an existing homedir, I want the import to be somewhat no-op, e.g. I don't need it to actually copy the .vimrc file over, I just need the config.yml entry made.

This works really well, except right now import wants to rename the file vimrc, so my existing homedir working copy ends up with both .vimrc (which I've had checked in for awhile) and also the new vimrc.

I made a hacky change to dotdrop.py which changes src = src.lstrip('.' + os.sep) to just src = src.lstrip(os.sep) and I get the "preserve dots on import" behavior that I want.

It seems like this would be a nice/easy feature to support, especially for people coming from existing-with-dot-prefix setups.

update failed, do manually: [Errno 21] Is a directory

Hi, I successfully installed this wonderful tool using submodule method.

It was okay until I try to update my dotfiles like bellow:

>  dotfiles [master] ⚡ dotdrop update ~/.zshrc.d
     _       _      _
  __| | ___ | |_ __| |_ __ ___  _ __
 / _` |/ _ \| __/ _` | '__/ _ \| '_ |
 \__,_|\___/ \__\__,_|_|  \___/| .__/  v0.22.0
                               |_|

Overwrite "/home/dim/dotfiles/dotfiles/zshrc.d" with "/home/dim/.zshrc.d"? [y/N] ? y
[WARN] /home/dim/.zshrc.d update failed, do manually: [Errno 21] Is a directory: '/home/dim/.zshrc.d'

Here are my config.yml file content:

config:
  backup: true
  banner: true
  create: true
  dotpath: dotfiles
  keepdot: false
  link_by_default: false
  longkey: false
  showdiff: false
  workdir: ~/.config/dotdrop
dotfiles:
  d_zshrc.d:
    dst: ~/.zshrc.d
    src: zshrc.d
  f_tmux.conf:
    dst: ~/.tmux.conf
    src: tmux.conf
  f_tmux.conf.local:
    dst: ~/.tmux.conf.local
    src: tmux.conf.local
  f_vimrc:
    dst: ~/.vimrc
    src: vimrc
  f_xinitrc:
    dst: ~/.xinitrc
    src: xinitrc
  f_xmodmap:
    dst: ~/.Xmodmap
    src: Xmodmap
  f_xprofile:
    dst: ~/.xprofile
    src: xprofile
  f_zpreztorc:
    dst: ~/.zpreztorc
    src: zpreztorc
  f_zshrc:
    dst: ~/.zshrc
    src: zshrc
profiles:
  lappie:
    dotfiles:
    - f_vimrc
    - f_zpreztorc
    - d_zshrc.d
    - f_zshrc
    - f_tmux.conf.local
    - f_tmux.conf
    - f_xinitrc
    - f_xprofile
    - f_xmodmap

Where did I make my mistake?

Hidden directory breaks compare

It seems as though dotdrop compare crashes if you try to use dotdrop to manage a hidden directory.

     _       _      _
  __| | ___ | |_ __| |_ __ ___  _ __
 / _` |/ _ \| __/ _` | '__/ _ \| '_ |
 \__,_|\___/ \__\__,_|_|  \___/| .__/  v0.2
                               |_|

diffing "f_zshrc" VS "~/.zshrc"
same file
diffing "f_xresources" VS "~/.Xresources"
same file
diffing "f_xmodmap" VS "~/.Xmodmap"
same file
diffing "f_xinitrc" VS "~/.xinitrc"
same file
diffing "f_xbindkeysrc" VS "~/.xbindkeysrc"
same file
diffing "f_vimrc" VS "~/.vimrc"
same file
diffing "f_i3blocks.conf" VS "~/.i3blocks.conf"
70c70
< instance=/mSATA
---
> instance=/media/mSATA
diffing "f_conkyrc" VS "~/.conkyrc"
same file
diffing "f_compton.conf" VS "~/.config/compton.conf"
same file
diffing "d_i3" VS "~/.i3"
Traceback (most recent call last):
  File "/home/matthew/dotfiles/dotdrop/dotdrop/dotdrop.py", line 182, in <module>
    if compare(opts, conf, tmp):
  File "/home/matthew/dotfiles/dotdrop/dotdrop/dotdrop.py", line 91, in compare
    inst.compare(t, tmp, opts['profile'], dotfile.src, dotfile.dst)
  File "/home/matthew/dotfiles/dotdrop/dotdrop/installer.py", line 168, in compare
    tmpfolder)
  File "/home/matthew/dotfiles/dotdrop/dotdrop/installer.py", line 151, in _install_to_temp
    return self.install(templater, profile, src, tmpdst), tmpdst
  File "/home/matthew/dotfiles/dotdrop/dotdrop/installer.py", line 32, in install
    return self._handle_dir(templater, profile, src, dst)
  File "/home/matthew/dotfiles/dotdrop/dotdrop/installer.py", line 91, in _handle_dir
    templater, profile, f, os.path.join(dst, entry))
  File "/home/matthew/dotfiles/dotdrop/dotdrop/installer.py", line 66, in _handle_file
    content = templater.generate(src, profile)
  File "/home/matthew/dotfiles/dotdrop/dotdrop/templategen.py", line 35, in generate
    return self._handle_file(src, profile)
  File "/home/matthew/dotfiles/dotdrop/dotdrop/templategen.py", line 42, in _handle_file
    return self._handle_bin_file(src, profile)
  File "/home/matthew/dotfiles/dotdrop/dotdrop/templategen.py", line 62, in _handle_bin_file
    with open(src, 'rb') as f:
FileNotFoundError: [Errno 2] No such file or directory: '/home/matthew/dotfiles/dotfiles/i3/.i3'

Switch/mirror to gitlab

I wanted to update my submodule but didn't find the project in gitlab. Is it planned to switch/mirror?

😗

Removing files after importing

Hi there, thanks for making dotdrop! Absolutely love the idea behind it.

How does dotdrop go about removing files after importing, say you may have accidentally imported something that's in a directory that you only come to realize later on. Will it work if I just remove the files from the dotfiles directory and from the config file?

script short-to-long-key.py is not working

Dotdrop has been updated to be included in pypi and
the way it needs to be called has slightly changed.

See https://github.com/deadc0de6/dotdrop/wiki/migrate-from-submodule

AUR package for 0.10 results in a package with the version 0.9.2

Thanks for also providing an AUR package!

The version of dotdrop seems to be right, but the package claims to be 0.9.2:

$ dotdrop -v
0.10
$ pacman -Qs dotdrop               
local/dotdrop 0.9.2-1
    Save your dotfiles once, deploy them everywhere

The problem seems to be:

$ git tag -l | tail -1 | sed 's/^v//g'
0.9.2
$ git tag -l
v0.10
v0.6
v0.7
v0.7.1
v0.7.2
v0.8.1
v0.8.2
v0.9
v0.9.1
v0.9.2

Feature request: specifying files to ignore when defining a directory dotfile

The compare feature is very useful, but it would be even more useful for me if it would be possible to specify files to ignore in that comparison.
Use case: I want to link the ~/.config directory to a config directory in my dotfiles repo. However, if there are files in my ~/.config directory which are not in the dotfiles repo, they will show up in the comparison.

Thanks!

add command to check status of dotfiles

add a command (for example check) that can be run quickly (smallest change check) from a bashrc or similar in order to warn the user in case some dotfiles haven't be synced.

Dotdrop does not work with macOS provided readlink

GNU's readlink has the -f flag, but this option is not supported by every implementation, like the macOS implementation of readlink. Currently dotdrop uses readlink -f to get the directory where dotdrop.sh lives, meaning that the script does not start on macOS. It would be nice if dotdrop would not depend on this behaviour, so it also works on systems that do not have GNU readlink, but the BSD variants.

Error installing dotfiles from home subfolder

Hi, when I try to import in my dotfiles these two config files:

~/.zprezto/runcoms/zshrc
~/.zprezto/runcoms/zpreztorc

dotdrop creates the entries in config.yaml as well as the subfolders in the dotfiles subfolder. However it seems not to be copying the files themselves, which I had to do manually.

compare functionality dies with special character

Hey there,

Kudos on the nifty project! Quick one: the compare functionality gave me a UnicodeDecodeError when it hit a special character in my viminfo file. Here's the full exception:

diffing "f_viminfo" VS "~/.viminfo"
Traceback (most recent call last):
File "/home/inf0junki3/dotfiles/dotdrop/dotdrop/dotdrop.py", line 165, in
if compare(opts, conf, tmp):
File "/home/inf0junki3/dotfiles/dotdrop/dotdrop/dotdrop.py", line 86, in compare
inst.compare(t, tmp, opts['profile'], dotfile.src, dotfile.dst)
File "/home/inf0junki3/dotfiles/dotdrop/dotdrop/installer.py", line 131, in compare
templater, profile, src, dst, tmpfolder)
File "/home/inf0junki3/dotfiles/dotdrop/dotdrop/installer.py", line 118, in _install_to_temp
return self.install(templater, profile, src, tmpdst), tmpdst
File "/home/inf0junki3/dotfiles/dotdrop/dotdrop/installer.py", line 31, in install
return self._handle_file(templater, profile, src, dst)
File "/home/inf0junki3/dotfiles/dotdrop/dotdrop/installer.py", line 38, in _handle_file
content = templater.generate(src, profile)
File "/home/inf0junki3/dotfiles/dotdrop/dotdrop/templategen.py", line 34, in generate
return self._handle_file(src, profile)
File "/home/inf0junki3/dotfiles/dotdrop/dotdrop/templategen.py", line 42, in _handle_file
return self._handle_text_file(src, profile)
File "/home/inf0junki3/dotfiles/dotdrop/dotdrop/templategen.py", line 46, in _handle_text_file
template = self.env.get_template(src[l:])
File "/usr/local/lib/python3.5/dist-packages/jinja2/environment.py", line 812, in get_template
return self._load_template(name, self.make_globals(globals))
File "/usr/local/lib/python3.5/dist-packages/jinja2/environment.py", line 774, in _load_template
cache_key = self.loader.get_source(self, name)[1]
File "/usr/local/lib/python3.5/dist-packages/jinja2/loaders.py", line 175, in get_source
contents = f.read().decode(self.encoding)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 1807: invalid start byte

Error while updating directory

Hey there,
i just started using dotdrop, love it so far.

but if i want to update whole dir i get an error.

situation:

>> dotdrop import ~/.config/rofi/rofi-scripts-collection

results in the following config.yam:

... 
d_rofi-scripts-collection:
    dst: ~/.config/rofi/rofi-scripts-collection
    src: config/rofi/rofi-scripts-collection
...

Try to update the whole dir results in:

>> dotdrop update ~/.config/rofi/rofi-scripts-collection 
Traceback (most recent call last):
  File "/usr/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/XXXXX/dotfiles/dotdrop/dotdrop/dotdrop.py", line 355, in <module>
    if main():
  File "/home/XXXXX/dotfiles/dotdrop/dotdrop/dotdrop.py", line 345, in main
    update(opts, conf, args['<path>'])
  File "/home/felix/Data/dotfiles/dotdrop/dotdrop/dotdrop.py", line 200, in update
    if Templategen.get_marker() in open(src, 'r').read():
IsADirectoryError: [Errno 21] Is a directory: '/home/XXXXX/dotfiles/dotfiles/config/rofi/rofi-scripts-collection'

Seems that dotdrop thinks its a single file, but the import is correct?

regards!

When de-linking a file, first remove the symlink

I started out with a file, e.g. .Xresources as a symlink, and later wanted to template-ize it, so I can have per-machine DPIs.

I removed the "link: true" from the config.yml, added my template syntax to .homedir/.Xresources, and run dotdrop install.

I was correctly prompted to overwrite the previous symlink:

Overwrite "/home/stephen/.Xresources" [y/N] ? y
        -> copied "/home/stephen/.homedir/./.Xresources" to "/home/stephen/.Xresources"

However, it looks like dotdrop did not first remove the symlink, as the post-template files where written to ~/.Xresources, which still being symlinked, means it also overwrote the ~/.homedir/.Xresources file as well.

I re-made my template changes to ~/.homedir/.Xresources, and then manually deleted the ~/.Xresources file, and then everything works as expected.

Automatic update of modified dotfile

Because I'm a boring and lazy user ...

Related to #4, it seems I'm not the only one looking for this feature that would be great to have.

Templated files sound tricky, but as first step, what about updating "flat" files automatically and requesting the user to manually update files containing Jinja syntax?

Access environment variables inside templates

First of all: Thank you very much for dotdrop!

To make it really usable for me, I need access to environment variables inside the templates.

That way - instead af having {%@@ if profile == "PROFILENAME" @@%} all over the place - I can just say {{@@ env['MY_VAR'] @@}} to set host-specific properties. I have an .env-file in my directory, that is not tracked by git and I invoke dotdrop with the following alias:

alias dotdrop='eval $(cat ~/dotfiles/.env | egrep -v "^#") ~/dotfiles/dotdrop.sh'

This also gives me the possibility to use dotdrop for files that need plain-text credentials or other information I don't want to share with the public.

You can see how I use it in my dotfiles (beware: I should spend some time to clean it up a bit 😉).

Since it's just two lines, that needs to be changed, for the time being, I've just edited them locally. If there is an interest in this, I can make a PR, when I find the time.

Possible to make Windows compatible (Git bash)?

I regularly use bash on Windows 7, as well as Linux and Mac. I was installing your script and it started to look like no consideration was given to the possibility of using it on Windows.

First, it looks for "python3" which was easy enough to fix by a "python3.exe" symlink to "python.exe". The next thing I ran into is this when I tried $ ./dotdrop.sh compare

Traceback (most recent call last):  
  File "C:\Python37\lib\runpy.py", line 193, in _run_module_as_main  
    "__main__", mod_spec)  
  File "C:\Python37\lib\runpy.py", line 85, in _run_code  
    exec(code, run_globals)  
  File "C:\Users\nick\dotfiles\dotdrop\dotdrop\dotdrop.py", line 26, in <module>  
    HOSTNAME = os.uname()[1]  
AttributeError: module 'os' has no attribute 'uname'

Python recommends using socket.gethostname() instead of os.uname() (unix-only). After I made this change to dotdrop.py, everything seemed to go smoothly.

I don't know if that's all that will be needed, but I thought I'd raise the issue and see if anyone else has used this with bash on Windows.

Thanks!
Nick

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.