Code Monkey home page Code Monkey logo

sitegeist.magicwand's Introduction

Sitegeist.MagicWand

Tools that make the Flow/Neos development easier

This package is intended to be used on development systems and should NEVER be installed on production servers. Please add this package to the require-dev section of your composer.json.

Authors & Sponsors

The development and the public-releases of this package is generously sponsored by our employer https://www.sitegeist.de.

Easy and fast cloning of Flow and Neos Installations

The CLI commands clone:list, clone:preset to help to clone a remote Flow/Neos setup into the local Flow/Neos installation that executes the command.

Attention: These commands will empty the local database and resources of your local Flow installation. The data is replaced with the information from the remote host. Make sure you understand that before actually using the commands.

CLI-Examples

# show all available presets
./flow clone:list

# clone from remote host with the information stored in the master preset
./flow clone:preset master

Settings.yaml

The presets that are defined in the configuration path. Sitegeist.MagicWand.clonePresets

Sitegeist:
  MagicWand:
    flowCommand: './flow'

    # preset which is used by the clone:default command
    defaultPreset: 'master'

    # available presets
    clonePresets:

       # the name of the preset for referencing on the clone:preset command
      master:
        # hostname or ip of the server to clone from
        host: ~
        # ssh username
        user: ~
        # ssh port
        port: ~
        # ssh options
        sshOptions: ~
        # path on the remote server
        path: ~
        # flow-context on the remote server
        context: Production

        # the flow cli command on the remote server
        # default is the main flowCommand-Setting
        flowCommand: ~

        # options to adjust the clone process
        clone: []

        # commands to execute after cloning like ./flow user:create ...
        postClone: []

        # informations to access the resources of the cloned setup via http
        # if this is configured the rsync of the persistent resources is skipped
        # and instead resources are fetched and imported on the fly once read
        resourceProxy:
          baseUri: http://vour.server.tld
          # define an optional subDirectory (defaults to: '_Resources/Persistent/', trailing slash is required!)
          subDirectory: '_Resources/Persistent/'
          # define wether or not the remote uses subdivideHashPathSegments
          subdivideHashPathSegment: false
          # curl options
          curlOptions:
            CURLOPT_USERPWD: very:secure

The settings should be added to the global Settings.yaml of the project, so that every developer with SSH-access to the remote server can easily clone the setup.

Quick backup and restore mechanisms for persistent data

Sometimes it's useful to quickly backup an integral persistent state of an application, to then perform some risky change operations and restore the data in case of failure. The stash:create,stash:restore,stash:list and stash:clear commands of this package allow for a flawless backup-try-restore workflow.

Attention: These commands will empty the database and resources of your local Flow installation. The data is replaced with the information from the stash. Make sure you understand that before actually using the commands.

CLI-Examples

# Create a backup of the entire database and the directory `Data/Persistent` ("stash entry") under the given name
./flow stash:create --name=name

# Lists all named stash entries
./flow stash:list

# Restores a stash entry
./flow stash:restore --name=name

# Removes all stash entries
./flow stash:clear

Note: Use this command on a regular basis, because your stash tends to grow very large.

Resource proxies

While cloning the database to your local dev system is manageable even for larger projects, downloading all the assets is often not an option.

For this case the package offers the concept of resource proxies. Once activated, only the resources that are actually used are downloaded just at the moment they are rendered. This is done by custom implementations of WritableFileSystemStorage and ProxyAwareFileSystemSymlinkTarget and works out of the box if you use this storage and target in you local development environment. If you use other local storages, for example a local S3 storage, you can easily build your own proxy aware versions implementing the interfaces ProxyAwareStorageInterface and ProxyAwareTargetInterfaceof this package.

Installation

Sitegeist.Magicwand is available via packagist. Just add "sitegeist/magicwand" : "~1.0" to the require-dev section of the composer.json or run composer require --dev sitegeist/magicwand. We use semantic-versioning so every breaking change will increase the major-version number.

Contribution

We will gladly accept contributions especially to improve the rsync, and ssh-options for a specific preset. Please send us pull requests.

We will NOT add the following features to the main-repository

  • Windows support: We rely on a unix-shell and a filesystem that is capable of hard-links.
  • SSH with username/password: We consider this unsafe and recommend the use of public- and private-keys.

sitegeist.magicwand's People

Contributors

daniellienert avatar drillsergeant avatar drprofesq avatar fnkr avatar freesh avatar grebaldi avatar jonnitto avatar kabarakh avatar mficzel avatar mstruebing avatar namiol avatar nikdro avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sitegeist.magicwand's Issues

Escaping parameters for remote shell execution

If e.g. the password of the db user on the remote server contains special characters as e.g. '$' the mysqldump command fails (mysql connection cannot be established).

In the setup where I experienced difficulties with this, we are using a Jumphost (passed by ProxyCommand sshOption) and I had to double escape the special character in the password in order to make a successful clone.

[Enhancement] Extend flow command with placeholder

Hi and thanks for this great package!

What do you think about adding a placeholder for the flow command execution. Something like this:

Sitegeist:
  MagicWand:
    flowCommand: '<FLOW_CONTEXT> ./flow <FLOW_COMMAND>'

If you are using Docker you can tweak your command for running in a container. I think that would be simple to achrive and really useful too :)

Default preset

Would it be a good idea to introduce a default-preset which is used when no parameter is given?

i.e. the live server is the default target and in this way it would be possible to only type ./flow clone:preset to clone this one. This way it wouldn't be needed to do a ./flow clone:list before to look up which preset should be cloned - I want to clone the live server in 99% of the cases anyway.

I would do a PR if this idea is welcome.

Alternative database sources

I like the package for its conveniences, however in some cases I can't use it because access to the flow commands to fetch the "real" database is not intended/restricted/...
I would imagine an alternative way of downloading a database dump from a different source and then proceed as before with resource proxies and stash options.

Is there any way I could do this with this package or would this exceed the scope of this package?

skip `Persistent` paremeter

Maybe it would be a good idea to add a feature which lets you skip the transfer of the Data/Persistent directory.
This would be useful if you only need a fresh database instead of the whole project data, especially in large projects.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.