beemojs / beemo Goto Github PK

View Code? Open in Web Editor NEW

143.0 4.0 9.0 20.17 MB

🤖 Centralized configuration layer for dev tools. Beep boop.

Home Page: https://beemo.dev

License: MIT License

JavaScript 3.12% TypeScript 96.09% CSS 0.77% EJS 0.02%

developer-tools development-environment configuration-management configuration-files dotfiles beemo

beemo's Introduction

🤖 Beemo

Beemo is no longer in development and will not be maintained or upgraded! We suggest migrating to a more thorough solution, like moon, a Rust based build system.

Manage dev and build tools, their configuration, and commands in a single centralized repository. Beemo aims to solve the multi-project maintenance fatigue by removing the following burdens across all projects: config and dotfile management, multiple config patterns, up-to-date development dependencies, continuous copy and paste, and more.

Features

Manage dev tools and configurations in a single repository.
Configure supported dev tools using .js or .ts files.
Customize and alter config at runtime with CLI options.
Pass custom CLI options to dev tool commands without failure.
Automatically expand glob patterns (a better alternative to bash).
Listen to and act upon events.
Easily share config between dev tools.
Avoid relative config or extends paths.
Automatic config file cleanup.
Custom scripts with CLI options.
Scaffolding and template generation.
Workspaces (monorepo) support.
Automated TypeScript project references.
Parallel, pooled, and prioritized builds.
Integrated watch/live mode.
And much more.

Requirements

Node 12.17+
GitHub, Bitbucket, or another VCS

Documentation

https://beemo.dev

beemo's People

Stargazers

Watchers

Forkers

hayes domoritz alexxnica sshyran niieani morni-andoka qasura-h

beemo's Issues

Drivers shouldnt have context property

Causes too many circular refs. Pass to events instead.

Script tasks lose access to class methods when ran

Script tasks are piped to the parent routine when the script is ran, but this pipe process re-binds the this of the task, so local script methods are lost.

This has to be fixed upstream in boost.

Add filtering to dotfile syncing

Command failed with exit code is not useful

When a driver fails, execa will throw an error with the message "Command failed with exit code". The error looks like the following:

{ Error: Command failed with exit code 1: babel --clean --copy-files --extensions .ts,.tsx,.js,.jsx src --out-dir lib
       at makeError (/Users/Miles/Projects/emojibase/node_modules/@boost/core/node_modules/execa/lib/error.js:56:11)
       at handlePromise (/Users/Miles/Projects/emojibase/node_modules/@boost/core/node_modules/execa/index.js:114:26)
       at process._tickCallback (internal/process/next_tick.js:68:7)
     command:
      'babel --clean --copy-files --extensions .ts,.tsx,.js,.jsx src --out-dir lib',
     exitCode: 1,
     signal: undefined,
     signalDescription: undefined,
     stdout: '',
     stderr: 'error: unknown option \'--clean\'',
     failed: true,
     timedOut: false,
     isCanceled: false,
     killed: false,
     cmd:
      'babel --clean --copy-files --extensions .ts,.tsx,.js,.jsx src --out-dir lib' }

The actual cause of failure is in stderr. This will be on a driver basis but should be doable here: https://github.com/beemojs/beemo/blob/master/packages/core/src/routines/driver/ExecuteCommandRoutine.ts#L370

Failed driver executions should persist exit code

Right now we exit with code 1, even if the underlying driver process is a different exit code.

Remove configs that return functions

They're just very problematic, especially when using "reference" strategy. Rely on process.beemo instead.

TypeScript config uses enums

We reference ts.CompilerOptions instead of duplicating those types, but they use enums instead of string unions, which don't map to what's actually in the config file.

Prettier config typings don't allow overrides

Prettier allows overrides but the beemo typings don't allow it.

import {PrettierConfig} from '@beemo/driver-prettier';

const config: PrettierConfig = {
  printWidth: 120,
  proseWrap: 'never',
  overrides: [
    {
      files: '*.{js,jsx,ts,tsx}',
      options: {
        bracketSpacing: false,
        singleQuote: true,
      },
    },
  ],
  ignore: ['.eslintrc.js', 'prettier.config.js', '.prettierrc.js'],
};

export default config;

Use a real dependency graph for priority sort

At the moment, if 2 dependencies have the same number of dependents, but one needs to run before the other, it may fail based on how packages were loaded from the file system. For example

foo
bar - depends on foo
baz - depends on bar

Ideally our chain would be foo -> bar -> baz, but since foo and bar both have 1 dependent, foo may come after bar, as such bar -> foo -> baz, which breaks for projects that require explicit order (like TS).

Windows support to do

Remove extension from binary: https://github.com/beemojs/beemo/blob/master/packages/cli/src/CLI.ts#L15
Normalize Beemo and driver paths: #66
Boost updates: milesj/boost#68

Prettier driver adds double quotes

My configs/prettier.js file only uses ' but the generated prettier.config.js has ". Can the driver be configured to maintain the original quoting or use what is specified in eslint or prettier?

Add --live arg for drivers/scripts

So consumers can see output live and not on completion.

Support copying config files instead of generating them

For example, Babel should support both, as some Babel configs can return a function in which we don't want to break.

Glob is buggy

Try fast-glob instead.

Provide hooks into the event system

Beemo emits events for every action that occurs. It would be nice to allow the configuration module to listen to these events and act accordingly. Some examples:

Inject CLI args automatically for a driver.
Modify the configuration file before it's written.
Trigger functionality before and after a command execution.

Show buffered driver output when ctrl + c is triggered

Otherwise we need to wait for the process to finish to see any output.

Set current beemo tool and args as a global

Something like BEEMO_ARGS and BEEMO_TOOL. This allows drivers like Webpack to hook into them still.

Beemo binary name must be camelCase while docs use a kebab-case name in examples

Hey Miles 👋
Thank you for the excellent work on Beemo! 😄

When running through the examples, I have noticed that following them exactly and using dev-tools as a binary name, an error is thrown due to the binName in the Beemo constructor being invalid.

dev-tools
❯ ./packages/dev-tools/bin/dev-tools
/Users/matan/work/dev-tools/node_modules/optimal/lib/index.js:198
        throw new Error(prefix + ". " + error);
        ^

Error: Invalid Beemo field "configName". Config file name must be camel case without extension.
    at StringPredicate.invariant (/Users/matan/work/dev-tools/node_modules/optimal/lib/index.js:198:15)
    at StringPredicate.<anonymous> (/Users/matan/work/dev-tools/node_modules/optimal/lib/index.js:137:18)
    at /Users/matan/work/dev-tools/node_modules/optimal/lib/index.js:333:28
    at Array.forEach (<anonymous>)
    at StringPredicate.validate (/Users/matan/work/dev-tools/node_modules/optimal/lib/index.js:332:17)
    at StringPredicate.run (/Users/matan/work/dev-tools/node_modules/optimal/lib/index.js:315:23)
    at /Users/matan/work/dev-tools/node_modules/optimal/lib/index.js:520:39
    at Array.forEach (<anonymous>)
    at Schema.build (/Users/matan/work/dev-tools/node_modules/optimal/lib/index.js:511:33)
    at Object.optimal [as default] (/Users/matan/work/dev-tools/node_modules/optimal/lib/index.js:575:17)
/Users/matan/work/dev-tools/node_modules/optimal/lib/index.js:198
        throw new Error(prefix + ". " + error);

This traces back to the Tool definition which Beemo extends from @boost/core, found here: https://github.com/milesj/boost/blob/master/packages/core/src/Tool.ts#L135-L139

I don't know the context of why that is the case in @boost/core, so I'll hold off on making a PR there.

ESLint swallowing warning output

When ran through Beemo:

[1/2] CONFIG Generating config files
[2/2] DRIVER Executing ESLint v5.16.0 driver (failed)

ESLint found too many warnings (maximum: 0).

Failed to execute pipeline. The following errors have occurred:

Command failed: eslint --ext .ts,.tsx,.js,.jsx --report-unused-disable-directives --color --max-warnings 0 src/index.ts
    at ExecuteDriverRoutine.formatAndThrowErrors (/Users/miles_johnson/Sites/airlab/repos/nova/node_modules/@beemo/core/lib/execute/BaseRoutine.js:61:15)
    at ExecuteDriverRoutine.execute (/Users/miles_johnson/Sites/airlab/repos/nova/node_modules/@beemo/core/lib/execute/BaseRoutine.js:47:18)
    at process._tickCallback (internal/process/next_tick.js:68:7)

When ran directly.

// npx eslint --ext .ts,.tsx,.js,.jsx --report-unused-disable-directives --color --max-warnings 0 src/index.ts

/src/index.ts
  27:1  warning  Unexpected console statement  no-console

✖ 1 problem (0 errors, 1 warning)

ESLint found too many warnings (maximum: 0).

This may be mixed stdout/stderr.

ENV should be passed to config creation

Instead of just the driver binary being execute.

create-config command doesnt create dep configs

Array config merging is broken

For example, if ESLint defines the following:

module.exports = {
  ignore: ['foo', 'bar', 'baz'],
};

And a beemo.eslint.ignore is defined in package.json:

{
  "beemo": {
    "eslint": {
      "ignore": ["qux"]
    }
  }
}

The result becomes ['qux', 'bar', 'baz'] instead of ['foo', 'bar', 'baz', 'qux'].

Jest outputting to stderr

Looks to be a bug in Jest: jestjs/jest#5064

Remove driver-babel as dependency of driver-jest

Hey Miles! 👋

Working on a Jest Beemo config, I have noticed that driver-babel is depended on by driver-jest. I see this making sense if the config is going to be using Babel to transpile TypeScript, but in my own use case, I use ts-jest and have no use for a Babel driver or config.

I'm currently using a fork of driver-jest with the dependency removed, and it appears to work perfectly.

What is the purpose of that dependency?
Do you think it would be worthwhile to make Babel optional here?

scaffold project dotfiles fails every time

npx nimbus scaffold project dotfiles

Console output:

npx nimbus scaffold project dotfiles --debug
  beemo:reporter-loader Sorting by priority +0ms
  beemo:reporter-loader Bootstrapping with tool environment +0ms
  beemo:driver-loader Locating driver module babel +0ms
  beemo:driver-loader Resolving in order: @beemo/driver-babel, beemo-driver-babel +0ms
  beemo:driver-loader Found with module @beemo/driver-babel +137ms
  beemo:driver-loader Locating driver module eslint +0ms
  beemo:driver-loader Resolving in order: @beemo/driver-eslint, beemo-driver-eslint +0ms
  beemo:driver-loader Found with module @beemo/driver-eslint +45ms
  beemo:driver-loader Locating driver module jest +0ms
  beemo:driver-loader Resolving in order: @beemo/driver-jest, beemo-driver-jest +0ms
  beemo:driver-loader Found with module @beemo/driver-jest +44ms
  beemo:driver-loader Locating driver module prettier +0ms
  beemo:driver-loader Resolving in order: @beemo/driver-prettier, beemo-driver-prettier +0ms
  beemo:driver-loader Found with module @beemo/driver-prettier +51ms
  beemo:driver-loader Locating driver module typescript +1ms
  beemo:driver-loader Resolving in order: @beemo/driver-typescript, beemo-driver-typescript +0ms
  beemo:driver-loader Found with module @beemo/driver-typescript +78ms
  beemo:driver-loader Locating driver module webpack +0ms
  beemo:driver-loader Resolving in order: @beemo/driver-webpack, beemo-driver-webpack +0ms
  beemo:driver-loader Found with module @beemo/driver-webpack +42ms
  beemo:driver-loader Sorting by priority +0ms
  beemo:driver-loader Bootstrapping with tool environment +0ms
  beemo:script-loader Sorting by priority +0ms
  beemo:script-loader Bootstrapping with tool environment +0ms
  beemo:reporter-loader Using default boost reporter +505ms
  beemo:core Bootstrapping configuration module +0ms
  beemo:core Executing bootstrap function: Found +451ms
  beemo:core Locating configuration module root +16ms
  beemo:core Found configuration module root path: /mnt/c/git/asdf/node_modules/@airbnb/nimbus/lib/index.js +8ms
  beemo:core Running scaffold command +2ms
  beemo:core Instantiating pipeline +0ms
  beemo:routine:scaffold Bootstrapping routine +0ms
  beemo:core Running pipeline +1ms
  beemo:console Starting console render loop +0ms
  beemo:console Wrapping `stderr` stream +0ms
  beemo:console Wrapping `stdout` stream +0ms
  beemo:serial-executor Instantiating task executor +0ms
  beemo:serial-executor Serializing 1 tasks +0ms
  beemo:serial-executor Instantiating task executor +0ms
  beemo:serial-executor Serializing 1 tasks +0ms
  beemo:console Stopping console with an error +70ms
  beemo:console Rendering final console output +0ms
[1/1] SCAFFOLD Generating from templates (failed)

Loaded templates: node_modules/@airbnb/nimbus/lib/index.js/templates

Failed to find scaffolding templates "project/dotfiles/*.ejs" in configuration module.
    at ScaffoldRoutine.runGenerator (/mnt/c/git/asdf/node_modules/@beemo/core/lib/routines/ScaffoldRoutine.js:74:23)

  beemo:console Unwrapping `stderr` stream +1ms
  beemo:console Unwrapping `stdout` stream +1ms

It does look like it's looking in wrong folder.
node_modules/@airbnb/nimbus/lib/index.js/templates which doesn't make sense.

Creating a config outside of executing a driver breaks because of context

TypeError: Cannot read property 'addConfigPath' of undefined
at PrettierDriver.handleCreateIgnoreFile

Referenced configs dont have access to tool instance

Make a note of this!

Dont run priority when not using workspaces

Custom scripts/bins

Provide a mechanism in which the configuration module can define scripts, like scripts/init.js, which could then be executed with Beemo, like beemo run-script init.

Priority builds aren't running parallels

DRIVER       Executing driver [19/22, 15.32s]
 ICONS       └ babel --workspaces=* --clean --copy-files --extensions .ts,.tsx,.js,.jsx ./src --out-dir ./lib --cjs [5/5, 2.82s]
 CORE        └ babel --workspaces=* --clean --copy-files --extensions .ts,.tsx,.js,.jsx ./src --out-dir ./lib --cjs [5/5, 3.60s]
 PLATFORM    └ babel --workspaces=* --clean --copy-files --extensions .ts,.tsx,.js,.jsx ./src --out-dir ./lib --cjs [5/5, 1.10s]
 APP-SHELL   └ babel --workspaces=* --clean --copy-files --extensions .ts,.tsx,.js,.jsx ./src --out-dir ./lib --cjs [5/5, 3.74s]
 APP-SHELL   └ babel --workspaces=* --clean --copy-files --extensions .ts,.tsx,.js,.jsx ./src --out-dir ./lib --esm [5/5, 3.75s]
 FORMS       └ babel --workspaces=* --clean --copy-files --extensions .ts,.tsx,.js,.jsx ./src --out-dir ./lib --cjs [5/5, 4.71s]
 FORMS       └ babel --workspaces=* --clean --copy-files --extensions .ts,.tsx,.js,.jsx ./src --out-dir ./lib --esm [5/5, 4.68s]
 METRICS     └ babel --workspaces=* --clean --copy-files --extensions .ts,.tsx,.js,.jsx ./src --out-dir ./lib --cjs [5/5, 3.39s]
 METRICS     └ babel --workspaces=* --clean --copy-files --extensions .ts,.tsx,.js,.jsx ./src --out-dir ./lib --esm [5/5, 3.37s]

Support TypeScript composite/reference projects

This should replace the --workspaces option.

https://blogs.msdn.microsoft.com/typescript/2018/07/30/announcing-typescript-3-0/#project-references

https://github.com/RyanCavanaugh/project-references-demo

Remove emojis in CI

Since they dont work anyways.

Support .eslintignore

ESLint supports a secondary config file, the .eslintignore file. Ideally this should also be generated and cleaned up alongside .eslintrc.

STRATEGY_REFERENCE does not work on windows.

Using @airbnb/nimbus with webpack option doesn't work with create-config webpack --silent.
this will create config path with looks like this:
module.exports = require('./node_modules\@airbnb\nimbus\configs\webpack.js');.
In wsl it works great.
This can be due to this line. Possible fix can be with path.normilize.

Support tools that utilize JS config files (Webpack, Rollup, etc)

It's quite difficult to generate these JS files on demand. Copy as is? Modify with an AST?

eslint formatting/color gets lost

Is I run eslint, I get errors like

With beemo, I get

It would be nice if coloring was preserved.

Support `executeCommand` in Script?

This is a great method for executing binaries, but since script extends Plugin and not Routine, we have lost this functionality. Port it somehow?

Webpack driver does not support all events

I am trying to follow the recommended pattern for listening to events, however, it looks like the webpack driver does not trigger all the events. I am trying to listen to onCreateConfigFile but it does not get called and there are no errors reported either.

tool.onRunDriver.listen((context, driver) => {
  ...

  driver.onCreateConfigFile.listen(context => {
     ....
  });

  ...
}, 'webpack');

Convert Scripts to a Boost plugin

Right now they are pseudo-routines that exist in the configuration module. Converting them to a Boost plugin would enable the following:

Can be encapsulated into an NPM module: beemo-script-whatever. Reusability ++
Can be enabled and configured from within the config file ("scripts": ["whatever"]) or from the CLI (--script whatever)
Easier access to Boost tool and pipeline layer.

Switch to debug NPM module

Worthwhile to switch to debug?

Beemo v1.1.1 build failed

I built project superset-ui in https://github.com/apache-superset/superset-ui with master version and it failed. I found that it use @superset-ui/build-config and @superset-ui/build-config use beemo.

{
    "@beemo/cli": "^1.0.6",
    "@beemo/core": "^1.0.8",
    "@beemo/driver-babel": "^1.0.6",
    "@beemo/driver-eslint": "^1.0.7",
    "@beemo/driver-jest": "^1.0.4",
    "@beemo/driver-prettier": "^1.0.4",
    "@beemo/driver-typescript": "^1.1.5"
}

I have open an issue apache-superset/superset-ui#265.
superset-ui github link is https://github.com/apache-superset/superset-ui .
Maybe someone can check what causes the Beemo failure.

Add create-config * support

To create a config file for each enabled driver.

Jest driver doesnt export types

Allow module to be defined with env vars

Useful when using a custom binary.

// foobar
#!/usr/bin/env node

BEEMO_CONFIG_MODULE="@foo/bar"

require('@beemo/cli/lib/CLI');

And then can omit it during config.

{
  "foobar": {
    "drivers": ["babel"]
  }
}

Localization?

Is this something Beemo should do? Might be a simple enough change to support localized routine tiles. Possibly with a --locale option.

Custom bin name breaks imports

Missing driver. Attempted import in order: @foo/driver-babel, foo-driver-babel
at ModuleLoader.importModule (/Users/miles_johnson/Sites/airlab/repos/carbon/node_modules/@boost/core/lib/ModuleLoader.js:69:19)
    at ModuleLoader.loadModule (/Users/miles_johnson/Sites/airlab/repos/carbon/node_modules/@boost/core/lib/ModuleLoader.js:128:25)
    at modules.map.module (/Users/miles_johnson/Sites/airlab/repos/carbon/node_modules/@boost/core/lib/ModuleLoader.js:139:43)
    at Array.map (<anonymous>)
    at ModuleLoader.loadModules (/Users/miles_johnson/Sites/airlab/repos/carbon/node_modules/@boost/core/lib/ModuleLoader.js:139:24)
    at Tool.loadPlugins (/Users/miles_johnson/Sites/airlab/repos/carbon/node_modules/@boost/core/lib/Tool.js:189:31)
    at Tool.initialize (/Users/miles_johnson/Sites/airlab/repos/carbon/node_modules/@boost/core/lib/Tool.js:141:14)
    at new Beemo (/Users/miles_johnson/Sites/airlab/repos/carbon/node_modules/@beemo/core/lib/Beemo.js:47:19)
    at Object.<anonymous> (/Users/miles_johnson/Sites/airlab/repos/carbon/node_modules/@beemo/cli/lib/CLI.js:21:15)
    at Module._compile (module.js:653:30)

Compare error message and stderr

If they are the same, alter the error message.

Adding custom drivers

First of all, great package, and really solves a huge pain point. Something I've been struggling to cleanly abstract for a while. I created a "holder" package with various configs and a hacky solution to add them to the consuming package via postinstall hook, but this seems like a much cleaner solution!

Now on to the question.

Do you have any guidance for creating custom drivers? Shall they be part of the Beemo repo, or shall they be separate NPM packages?

I'm interested in the following drivers:

commitizen
commitlint
husky
lint-staged
markdownlint
semantic-release

Thanks!

Parallel executions

Most drivers don't need this, but some would benefit from it, like Babel. A possible use case/example: beemo babel --react --flow --parallel="--cjs" --parallel="--esm". This would run the babel command twice, with the args --react --flow --cjs and --react --flow --esm passed respectively.

Allow driver modules to be scoped outside of `@beemo/`

Hey Miles 👋

Folks at @auth0 have started using beemo for internal (and soon open source) projects. We would like to create our own drivers within our own @auth0/ scope to avoid name collisions, but it looks like only the following drivers will be resolved: @beemo/driver-X and beemo-driver-X. I think it would be beneficial if folks had the flexibility to use @X/beemo-driver-Y for their driver packages.

Looking into how driver resolution works, it looks like module resolution logic is handled by boost. I don't mind getting my hands dirty and making a PR, but it may take me some time to figure out how boost works. Some guidance would be appreciated.