Code Monkey home page Code Monkey logo

ops's Introduction

Ops

Version 0.16.4

  • A local development environment focused on PHP-based projects.
  • Create a new host by making a directory.
  • Local HTTPS with self-signed certs.
  • Shared services: MariaDB, PostgreSQL, Redis, Mailhog, Adminer, and more.
  • Sync remote databases and filesystem paths to local projects.
  • Built on Docker and Traefik. Configure project-specific services when required.

Ops Dashboard

Prerequisites

Ops supports Linux, Mac, and Windows Subshell Linux (WSL)

bash, npm, docker, docker-compose, ssh, and rsync are required.

Linux Installation Instructions

Mac Installation Instructions

Windows Subshell for Linux

For HTTPS support in Firefox on Windows, go to about:config and change security.enterprise_roots.enabled to true

How To Install

When installing global npm scripts, ensure you can do this as a non-superuser through one of these solutions.

Install/Update ops:

npm install -g git+https://[email protected]/imarc/ops
ops system install

On first install, You will be asked for your sudo/system password in order to install the self-signed certs for HTTPS support

Create Your First Project

To start ops services, run ops start

To add a new project, create a directory within $HOME/Sites. Your local project will then be available at https://{directory}.imarc.io.

The project directory name can only contain letters, numbers, and dashes ([A-Za-z0-9-]+).

By default the document root is set to a public directory within your project.

To stop ops, run ops stop

The Dashboard

The dashboard (https://ops.imarc.io) will show you all projects and let you manage services.

Project Configuration

Within your project's .env file you can set the following options:

# Set the application backend
#
# Valid values are the OPS_BACKENDS global option
#
# Default: value of OPS_DEFAULT_BACKEND global option
OPS_PROJECT_BACKEND="apache-php82"

# set the project document root
#
# Default: value of OPS_DEFAULT_DOCROOT global option
OPS_PROJECT_DOCROOT="public"

# enable basic auth for project
#
# Default: 0
OPS_PROJECT_BASIC_AUTH=1

# sets the .htpasswd file to use for basic auth
# looks for this file within the project path
#
# Default: .htpasswd
OPS_PROJECT_BASIC_AUTH_FILE=".htpasswd"

# allow the use of subdomains.
# may require 'ops system regenerate-certs' for HTTPS
#
# Default: 0
OPS_PROJECT_ALLOW_SUBDOMAINS=1

Global Configuration

The following options can be set within your $HOME/.ops/config file.

# A space separated list of application backends that
# should be enabled. Available options are:
#  apache-php72, apache-php73, apache-php74, apache-php80
#
# Default: apache-php82
OPS_BACKENDS="apache-php82 apache-php83"

# A linux user id that a backend process should attempt to run as.
#
# Recommended on linux systems. Ensures that files written to the filesystem
# by the backend process map to the correct owner on the host system.
#
# Docker Desktop for mac or windows already resolves this behavior.
#
# Default: (none)
OPS_DOCKER_UID=""

# A linux group id that a backend process should attempt to run as.
#
# See OPS_DOCKER_UID for description.
#
# Default: (none)
OPS_DOCKER_GID=""

# The root domain to extend with custom project subdomains
#
# For development, the dns should be configured as a wildcard and to point
# back to 127.0.0.1
#
# Default: "imarc.io"
OPS_DOMAIN="imarc.io"

# Configure domain aliases for a particular project
#
# Set up space separated key value pairs to use a real domain
# Also requires a host override in /etc/hosts to 127.0.0.1
#
# Example:
#   OPS_DOMAIN_ALIASES="myrealdomain.com:project"
#
# Default: (none)
OPS_DOMAIN_ALIASES=""

# Set the diretory where projects live.
#
# Default: $HOME/Sites
OPS_SITES_DIR="$HOME/Sites"

# If a backend is not specified in a project's .env
# file, this backend will be used.
#
# Default: apache-php73
OPS_DEFAULT_BACKEND="apache-php73"

# If a document root is not specified in a project's .env
# file, this document root will be used.
#
# Default: public
OPS_DEFAULT_DOCROOT="public"

# A localtunnel (localtunnel.me) host. If you run your own server, configure it here.
#
# Default: https://localtunnel.me
OPS_LOCALTUNNEL_HOST="https://localtunnel.me"

# Command to launch a browser for 'ops www'. You can override the browser here.
#
# Default (mac): open
# Default (linux): xdg-open
# Default (WSL): explorer.exe
OPS_BROWSER=firefox

# Enable XDebug extension for PHP containers
#
# Requires restart to take effect. Set to 1 to enable.
#
# Default: 0
OPS_PHP_XDEBUG="0"

Connect to Services

You can connect your app to the following shared services:

MariaDB 10.3
hostname: mariadb
username: root
password: none
port: 3306

PostgreSQL 9.6
hostname: postgres
username: postgres
password: none
port: 5432

Redis 3.2
hostname: redis
port: 6379

Memcached 1.4
hostname: memcached
port: 11211

Minio
hostname: minio
root user: minio-user
root password: minio-password

Mailhog
SMTP Config:
hostname: mailhog
port: 1025

ChromeDriver
hostname: chrome
port: 9515

Remote to Local Syncing

With a few settings in a project's .env file, you can sync the project's database and filesystem storage from a remote server. Ideally, these settings should be checked into version control within a .env.example file to be shared amongst the development team.

Currently ops sync assumes the following:

  • SSH access is enabled to the remote web and/or DB servers.
  • remote server has the following database tools available to the SSH user: mysqldump, pg_dump.
  • OPS_PROJECT_SYNC_DIRS, OPS_PROJECT_REMOTE_HOST and OPS_PROJECT_REMOTE_PATH are all defined.

The following settings can be placed in the project's .env

# the local database type: mariadb, psql
# default: none
OPS_PROJECT_DB_TYPE=""

# the local database name.
# default: the project name (which is the directory name)
OPS_PROJECT_DB_NAME="${OPS_PROJECT_NAME}"

# space separated relative dirs to sync down
# default: none
OPS_PROJECT_SYNC_DIRS=""

# disable remote database sync. 1 or 0
# default: 0
OPS_PROJECT_SYNC_NODB="0"

# space separated rsync exclude patterns
# default: none
OPS_PROJECT_SYNC_EXCLUDES=""

# rsync max filesize to sync
# default: 500M
OPS_PROJECT_SYNC_MAXSIZE="500M"

# the remote hostname for the filesystem sync
# default: the project name (which is the directory name)
OPS_PROJECT_REMOTE_HOST="${OPS_PROJECT_NAME}"

# the remote SSH user for the filesystem sync
# default: none
OPS_PROJECT_REMOTE_USER=""

# the remote project path for the filesystem sync
# default: none
OPS_PROJECT_REMOTE_PATH=""

# the remote hostname for the database sync
# default: the remote hostname for the filesystem sync
OPS_PROJECT_REMOTE_DB_HOST=""

# the remote database type
# default: the local database type
OPS_PROJECT_REMOTE_DB_TYPE="${OPS_PROJECT_DB_TYPE-$OPS_PROJECT_DB_TYPE}"

# the remote database name
# default: the local database name
OPS_PROJECT_REMOTE_DB_NAME="${$OPS_PROJECT_DB_NAME}"

# the remote database user
# default: the remote SSH user
OPS_PROJECT_REMOTE_DB_USER="${OPS_PROJECT_REMOTE_USER}"

# the remote database password
#
# !!! best practice is to not use this and instead place a .my.cnf file
# !!! in your remote home directory. that is more secure as it won't
# !!! leak your password in cli history or process lists
#
# default: none
OPS_PROJECT_REMOTE_DB_PASSWORD=""

# the remote database port
# default: 3306 for mariadb, 5432 for psql
OPS_PROJECT_REMOTE_DB_PORT=""

Custom Project Container

This is for advanced users and it would be best to have a solid understanding of docker and docker-compose files before moving forward with implementing this solution

There are times where you want to configura a custom container to run your project. This could be due to:

  • Your project needs a special PHP configuration or extension.
  • Your project need to lock down the PHP version or use a specific image.
  • You are using a language that isn't PHP.

With a little configuration, Ops allows you to run a custom container beside the shared services. Ops' project subcommands deal with project specific compose commands. Running ops link within a project directory will load a ops-compose.yml file.

By linking a project, its containers will respond to with ops start and ops stop commands.

To unlink a project and remove its container run ops unlink within a project directory.

Here is a generic ops-compose.yml file:

version: '2'

services:
  craft:
    image: imarcagency/ops-apache-php72:${OPS_VERSION}

    labels:
      - "ops.project=${OPS_PROJECT_NAME}"
      - "ops.hostname=${OPS_PROJECT_NAME}.${OPS_DOMAIN}"
      - "traefik.enable=true"
      - "traefik.docker.network=ops_gateway"
      - "traefik.frontend.rule=Host:${OPS_PROJECT_NAME}.${OPS_DOMAIN}"
      - "traefik.port=80"
    environment:
      - "APACHE_UID=${OPS_DOCKER_UID}"
      - "APACHE_GID=${OPS_DOCKER_GID}"
      - "APACHE_ROOT=/var/www/public"
    volumes:
      - "./:/var/www/:cached"
    networks:
      - default
      - ops_gateway
      - ops_backend

networks:
  ops_gateway:
    external: true
  ops_backend:
    external: true

The most important things are the labels and the networks. Those settings are required for the proxy to function or for your app to conenct to shared services. Everything else can be customized to whatever your app requires.

Contributing

If you are contributing to ops, debugging, or want to try out bleeding edge features, It is recommended you install like so:

# clone into a local dir and enter dir
git clone [email protected]:imarc/ops.git
cd ops

# create 'ops' symlink to your repo
npm install -g .

This means your global ops script will point directly to the repo and you can make live changes.

License

MIT License

Copyright (c) 2021 Imarc

ops's People

Contributors

chantron avatar ellierider avatar jeffturcotte avatar jfe-imarc avatar khamer avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

ops's Issues

Allow for mapping of additional subdomains to a site

Use case: building an site having a codebase that serves multiple subdomains (like an intranet). Would like to be able to set up something like:

  • marketing.my-company.imarc.io
  • finance.my-company.imarc.io
  • hr.my-company.imarc.io
    etc...

If it simplifies things, the subdomains could be hardcoded into the .env rather than served via wildcards.

Ops command error

When typing the ops command to see what arguments are available, this is the result.

Version: 0.10.0

$: ops
awk: syntax error at source line 1
 context is
	 >>>  <<<
	missing )
awk: bailing out at source line 1
awk: extra ) at source line 1
 context is
	/--/{next;} >>>  /^ops-/"") <<<
awk: syntax error at source line 1
	extra )
awk: bailing out at source line 1

Usage: ops <command>

Available commands:

ops version 0.10.0

Missing write access to /usr/local/lib/node_modules

When doing a fresh install on macOS 11.0.1 (Big Sur), a permissions error is encountered writing to /usr/local/lib/node_modules (which is owned by root).

I found this note but am unsure of how to find or modify the Dockerfile:
https://github.com/nodejs/docker-node/blob/master/docs/BestPractices.md#global-npm-dependencies

Global npm dependencies
If you need to install global npm dependencies, it is recommended to place those dependencies in the non-root user directory. To achieve this, add the following line to your Dockerfile

ENV NPM_CONFIG_PREFIX=/home/node/.npm-global

ENV PATH=$PATH:/home/node/.npm-global/bin # optionally if you want to run npm global bin without specifying path

% npm install -g git+https://[email protected]/imarc/ops

npm WARN checkPermissions Missing write access to /usr/local/lib/node_modules
npm ERR! code EACCES
npm ERR! syscall access
npm ERR! path /usr/local/lib/node_modules
npm ERR! errno -13
npm ERR! Error: EACCES: permission denied, access '/usr/local/lib/node_modules'
npm ERR!  [Error: EACCES: permission denied, access '/usr/local/lib/node_modules'] {
npm ERR!   errno: -13,
npm ERR!   code: 'EACCES',
npm ERR!   syscall: 'access',
npm ERR!   path: '/usr/local/lib/node_modules'
npm ERR! }
npm ERR!
npm ERR! The operation was rejected by your operating system.
npm ERR! It is likely you do not have the permissions to access this file as the current user
npm ERR!
npm ERR! If you believe this might be a permissions issue, please double-check the
npm ERR! permissions of the file and its containing directories, or try running
npm ERR! the command again as root/Administrator.

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/kyle/.npm/_logs/2020-11-17T02_42_43_711Z-debug.log

Issue with local user to container user mapping on Linux

User/group mapping isn't working properly on Linux (at least Ubuntu).

My username/group is chandler/chandler, the container uses www-data/www-data.
User/group of files is -rw-rw-r-- 1 1000 1000 1360 Apr 15 14:56 index.php

docker-credential-desktop.exe not installed or not available in PATH

Windows info

Edition	Windows 10 Home
Version	20H2
Installed on	‎9/‎1/‎2020
OS build	19042.630
Experience	Windows Feature Experience Pack 120.2212.31.0

Debian

PRETTY_NAME="Debian GNU/Linux 10 (buster)"
NAME="Debian GNU/Linux"
VERSION_ID="10"
VERSION="10 (buster)"
VERSION_CODENAME=buster
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

Docker info

Docker version 19.03.13, build 4484c46d9d

Issue

OPS installs ok locally on the WSL machine. However, I am running into issues when attempting to start OPS locally.

I keep running into the following error:

samfonseca@xps13:~/Sites$ ops start
Starting ops services...

Pulling dnsmasq (jpillora/dnsmasq:1.1.0)...
Traceback (most recent call last):
  File "bin/docker-compose", line 3, in <module>
  File "compose/cli/main.py", line 67, in main
  File "compose/cli/main.py", line 126, in perform_command
  File "compose/cli/main.py", line 1070, in up
  File "compose/cli/main.py", line 1066, in up
  File "compose/project.py", line 615, in up
  File "compose/service.py", line 356, in ensure_image_exists
  File "compose/service.py", line 1267, in pull
  File "compose/progress_stream.py", line 99, in get_digest_from_pull
  File "compose/service.py", line 1234, in _do_pull
  File "docker/api/image.py", line 396, in pull
  File "docker/auth.py", line 48, in get_config_header
  File "docker/auth.py", line 324, in resolve_authconfig
  File "docker/auth.py", line 235, in resolve_authconfig
  File "docker/auth.py", line 262, in _resolve_authconfig_credstore
  File "docker/auth.py", line 287, in _get_store_instance
  File "docker/credentials/store.py", line 25, in __init__
docker.credentials.errors.InitializationError: docker-credential-desktop.exe not installed or not available in PATH
[13120] Failed to execute script docker-compose

Need to install mkcert manually

Installing on a new Mac today, I saw this output:

Warning: "certutil" is not available, so the CA can't be automatically installed in Firefox! ⚠️
Install "certutil" with "brew install nss" and re-run "mkcert -install" 👈

I followed the instructions but then saw:

% mkcert -install
zsh: command not found: mkcert

I tried a new session, same results.

Then I ran brew install mkcert and reran mkcert -install and it worked.

Should I update the instructions to include installing mkcert alongside nss?

Can't install as root

Either show a warning to stop people from installing as root, or figure out why the OPS_HOME dir copy doesn't work as root.

mariadb import fails when current path has a space

When trying to import an SQL file into mariadb, the process will fail if your current path has a space in it.

Example:
$ cd ~/MySQL Dumps
$ ops mariadb import my_database my_file.sql
cat: my_file.sql: No such file or directory

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.