mateodelnorte / meta Goto Github PK

This is likely happening because npm is wrapping bin files in .cmd files.

C:\Program Files (x86)\Nodist\v-x64\6.9.1\node_modules\meta\node_modules\.bin\meta-git:2
basedir=$(dirname "$(echo "$0" | sed -e 's,\\,/,g')")
          ^^^^^^^
SyntaxError: missing ) after argument list
    at Object.exports.runInThisContext (vm.js:76:16)
    at Module._compile (module.js:542:28)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)
    at Function.Module._load (module.js:438:3)
    at Module.runMain (module.js:604:10)
    at run (bootstrap_node.js:394:7)
    at startup (bootstrap_node.js:149:9)
    at bootstrap_node.js:509:3

Hello!

I've been using your project to manage about 150 repos and I've been loving it. I've had a few pain points, but one is that many of my meta exec statements only apply to a few repositories. For example, I'll do something like this:

script.sh

if ! [ -a .meta ]; then
  BRANCH=$(git remote show origin | grep "HEAD branch" | sed "s/.*: //")
  git fetch origin
  git checkout $BRANCH
  git reset --hard origin/$BRANCH
  git clean -fdx
  if [ -a package.json ] && ! [ -a .gitignore ]; then
    echo 'node_modules' > .gitignore
    git add .gitignore
    git commit -S -m 'Add .gitignore to ignore node_modules'
    git push origin $BRANCH
  fi
fi

meta exec "$(< script.sh)"

Here are my pain points:

• I'll often want to exclude the meta repo, and this seems better suited to a CLI flag than if ! [ -a .meta ]
• I only want to run commands on repos with package.json, but I get output from all repos -- it would be great if repos without any stdout just didn't show up at all
• I personally don't mind the shell command above, but meta exec -f script.sh might be more ergonomic

Thanks for your work on this project, I'm loving it!

because npm5 flattens dependencies so much, meta plugins may not have their own node_modules folder and therefore may break when running one of their executables

with travis-deploy-once - maybe something changed and env vars don't get passed through..

Error: The GitHub user of the "GH_TOKEN" has not authenticated Travis CI yet. Go to https://travis-ci.com/, login with the GitHub user of this token and then restart this job.
    at module.exports (/home/travis/.nvm/versions/node/v11.4.0/lib/node_modules/travis-deploy-once/lib/get-jobs.js:35:13)
    at process.internalTickCallback (internal/process/next_tick.js:77:7)

using just semantic-release works but doesn't wait for all builds to pass, and triggers the release process multiple times (they fail after the first one).

Also the version calculated is incorrect, so gotta add some tags to this repo to fix..

Here's the log:
2018-02-05T18_26_42_802Z-debug.log

I'm able to git clone that repo without problems so not sure what the error msg about permission is about.

I made this edit to the package.json and it was able to install (Note: I am a novice to npm, so not sure this is the correct fix but it helped me move forward).

diff --git a/package.json b/package.json
index d6e8f1b..26edcee 100644
--- a/package.json
+++ b/package.json
@@ -33,7 +33,7 @@
   "homepage": "https://github.com/mateodelnorte/meta#readme",
   "dependencies": {
     "chalk": "^2.1.0",
-    "commander": "mateodelnorte/commander.js",
+    "commander": "https://github.com/mateodelnorte/commander.js/archive/v2.9.0.tar.gz",
     "cross-env": "^3.1.4",
     "debug": "^2.3.3",
     "findup-sync": "^1.0.0",

Not working with pnpm package manager. https://github.com/pnpm/pnpm

There must be something about the way it looks for plugins that is non-standard. Maybe it expects them all to be top-level deps, which is not really reliable.

I'm currently working on a meta plugin I'd like to publish as @thing-it/meta-devtools. The current plugin discovery does not find this as it does not look for ^meta-.* directories inside organization scope directories.

In the long term, this may generally help to avoid NPM name clashes in the global scope.

I'm glad to work on a PR but would like to quickly gather some feedback if this is something you'd like to add.

Don't know if this is the place to ask.
But I would like to know how to do code sharing.

My project consist out of three microservices (NodeJS). I set up CI/CD on GitLab that builds the a microservice on change and pushes it into a cluster. These microservices have shared code, like database-models, policy-rules etc.

What's the best way to set this up? I was thinking about building the meta-project, but this makes CI/CD pretty complicated I think.

Maybe checkout the repo's in the build-fase? or use npm-modules? Or maybe a meta-project in a meta-project?

Any help is welcome, thanks!

Running any subcommand throws a SyntaxError on Windows:

$ meta git clone [email protected]:mateodelnorte/meta.git
C:\Users\Andreeib\AppData\Roaming\npm\node_modules\meta\node_modules\.bin\meta-git.cmd:1
(function (exports, require, module, __filename, __dirname) { @IF EXIST "%~dp0\node.exe" (
                                                              ^

SyntaxError: Invalid or unexpected token
    at createScript (vm.js:80:10)
    at Object.runInThisContext (vm.js:139:10)
    at Module._compile (module.js:607:28)
    at Object.Module._extensions..js (module.js:654:10)
    at Module.load (module.js:556:32)
    at tryModuleLoad (module.js:499:12)
    at Function.Module._load (module.js:491:3)
    at Function.Module.runMain (module.js:684:10)
    at startup (bootstrap_node.js:187:16)
    at bootstrap_node.js:608:3

$ node -v
v8.11.1

$ npm -v
5.8.0

If I replace this line:
https://github.com/mateodelnorte/commander.js/blob/245bc4c84ba72d447ab30ec3e2a4456e34052b59/index.js#L576
with:
proc = spawn(bin, args, { stdio: 'inherit', customFds: [0, 1, 2] });
it works.

In windows the npm bin is a batch file and not JavaScript.

meta-git-clone.cmd
@IF EXIST "%~dp0\node.exe" (
  "%~dp0\node.exe"  "%~dp0\..\meta-git\bin\meta-git-clone" %*
) ELSE (
  @SETLOCAL
  @SET PATHEXT=%PATHEXT:;.JS;=;%
  node  "%~dp0\..\meta-git\bin\meta-git-clone" %*
)

I may be misunderstanding things but when I ran meta project add <folder> <repo> it created an empty folder with the folder name I specified. I assumed it would contain the files/folders from the repo but it didn't so I tried meta yarn install and nothing improved.

I then tried meta yarn clean which removed the meta project node_modules and re-tried meta yarn install but it did nothing, in fact it didn't even install the meta project node_modules.

Where am I going wrong here?

I have:

node 8.9.4
yarn 1.3.2
meta 1.0.48
meta-yarn ^0.0.4

Running into problems when trying to add project in meta project that is with dot notation.

valdestrijus@DESKTOP-9LG90J8 MINGW64 /c/www/platform.meta (master)
$ meta project add platform.app [email protected]:1337homework/platform.app.git
meta project adding '[email protected]:1337homework/platform.app.git' at platform.app
created C:\www\platform.meta\platform.app
Initialized empty Git repository in C:/www/platform.meta/platform.app/.git/
set remote origin to [email protected]:1337homework/platform.app.git
adding "platform.app": "[email protected]:1337homework/platform.app.git" to .meta file at C:\www\platform.meta\.meta
adding platform.app to .gitignore at C:\www\platform.gitignore\.meta
C:\www\platform.gitignore\.meta does not exists. Creating it now.
fs.js:646
  return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);
                 ^

Error: ENOENT: no such file or directory, open 'C:\www\platform.gitignore\.meta'

getting the following on first run, after a new install

meta
TypeError: Cannot read property 'split' of undefined
    at module.exports (/Users/matt/n/lib/node_modules/meta/lib/findPlugins.js:33:25)
    at Object.exports.run (/Users/matt/n/lib/node_modules/meta/index.js:24:23)
    at Object.<anonymous> (/Users/matt/n/lib/node_modules/meta/bin/meta:8:15)
    at Module._compile (internal/modules/cjs/loader.js:688:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:699:10)
    at Module.load (internal/modules/cjs/loader.js:598:32)
    at tryModuleLoad (internal/modules/cjs/loader.js:537:12)
    at Function.Module._load (internal/modules/cjs/loader.js:529:3)
    at Function.Module.runMain (internal/modules/cjs/loader.js:741:12)
    at startup (internal/bootstrap/node.js:285:19)

Using an unsupported command yields no output and a successful exit code.
I think it should write an error message, and certainly not return 0.

Some example cases (the 0 in my prompt is the exit code of the last command):

# calling meta with a random word
   0[quezak@...]...$ meta asdf
# calling meta git with a random word
   0[quezak@...]...$ meta git asdf
# calling meta git with an alias to git status (in my ~/.gitconfig) -- shouldn't this produce same output as meta git status?
   0[quezak@...]...$ meta git s
   0[quezak@...]...$

we need to be able to see output from child processes, esp in the event that one prints text asking for input. subsequently, we need to be able to type responses and have the child process respond.

perhaps doing so can be circumvented with a --quiet option.

if i use this library behind proxy, where npm registry is some sort of artifactory I can not install meta via
npm install meta -g because there is some script that is loaded from github not from registry.

npm install meta -g
npm ERR! Error while executing:
npm ERR! C:\Program Files\Git\mingw64\bin\git.EXE ls-remote -h -t ssh://[email protected]/mateodelnorte/commander.js.git
npm ERR!
npm ERR! ssh: connect to host github.com port 22: Connection refused
npm ERR! fatal: Could not read from remote repository.
npm ERR!
npm ERR! Please make sure you have the correct access rights
npm ERR! and the repository exists.
npm ERR!
npm ERR! exited with error code: 128

npm ERR! A complete log of this run can be found in:
npm ERR!     C:\Users...\\2018-10-01T07_15_23_818Z-debug.log

Consider putting commander.js as npm package instead so that everyone behind proxy can also use your meta tool.

🚨 You need to enable Continuous Integration on Greenkeeper branches of this repository. 🚨

To enable Greenkeeper, you need to make sure that a commit status is reported on all branches. This is required by Greenkeeper because it uses your CI build statuses to figure out when to notify you about breaking changes.

Since we didn’t receive a CI status on the greenkeeper/initial branch, it’s possible that you don’t have CI set up yet. We recommend using Travis CI, but Greenkeeper will work with every other CI service as well.

If you have already set up a CI for this repository, you might need to check how it’s configured. Make sure it is set to run on all new branches. If you don’t want it to run on absolutely every branch, you can whitelist branches starting with greenkeeper/.

Once you have installed and configured CI on this repository correctly, you’ll need to re-trigger Greenkeeper’s initial pull request. To do this, please click the 'fix repo' button on account.greenkeeper.io.

Right now, I've got this working wonderfully if I add projects using https. But, I use bitbucket and the url is something of the form: https://[email protected]/organization/repo.git

I'd like to force my development staff to use this if possible, but they'd all have to change the username in the .meta;

I did get it to work with ssh when I added the project with the format [email protected]:organization/repo.git but it was challenging to say the least and I had ssh whining at me for quite a bit.

Perhaps I'm just ignorant to an easier way. Perhaps I should just write a script that's inside my metarepo; But I'm very fond of just using meta git clone metarepourl to clone everything. If I had to script the updation of the .meta file to hot-swap the end-users username, it wouldn't work etc..

Perhaps you can point me to some best practice around using this in teams?

For what it's worth, I'd love to figure this out and help you document the add project module a bit more; It was hard to tell if I just entered the repo wrong, or it was an ssh problem etc...

When trying to run meta from local npm modules you get this:

git:(meta)$ ./node_modules/.bin/meta
{ Error: ENOENT: no such file or directory, scandir '/projectName/node_modules/meta/node_modules'
    at Error (native)
    at Object.fs.readdirSync (fs.js:951:18)
    at module.exports.error (/projectName/node_modules/meta/lib/listDirectories.js:32:28)
    at Object.<anonymous> (/projectName/node_modules/meta/bin/meta:13:14)
    at Module._compile (module.js:556:32)
    at Object.Module._extensions..js (module.js:565:10)
    at Module.load (module.js:473:32)
    at tryModuleLoad (module.js:432:12)
    at Function.Module._load (module.js:424:3)
    at Module.runMain (module.js:590:10)
  errno: -2,
  code: 'ENOENT',
  syscall: 'scandir',
  path: '/projectName/node_modules/meta/node_modules' }

Is there anything that could be done about that or it has to be run globally?

Same thing happens if you run meta out of an npm package script.

It'd be great if there were a meta init command that could be used to bootstrap your meta setup.

At a minimum it should create a .meta file, with the basic JSON structure.

maybe something like this would do the trick? https://www.npmjs.com/package/nexe

Hello @mateodelnorte ,

Amazing repo! Love it so far. Quick question, doubtful this is code that should live in your repo, but do you know of a way to make my zsh aliases work?

for instance, git s for me equals git status - just shorter; I have like 20 other shortcuts that I love to use and I want to use them prefixed with meta so I can perform said operations on all my sub repos at once.

Perhaps you can just point me in the right direction?

It would be cool to have meta-init, meta-project, meta-git, etc in this repository as Meta child repos.

edit: Oh and loop too, since it's so vital to Meta.

Dear @mateodelnorte ,

i am currently evaluating this package for future use in my projects as i want to provide an easy to maintain approach for splitting up my packages into different modules and then building it all together.

I have a question, which i as not able to answer so far:
Is it possible to tag all sub-packages from the main repository? e.g., if i release a new version of my main application, can i tag all sub-packages at the same time?

Consider the following example:
I will release a 2.0.0 version of my main application and, therefore, adapt all my other packages. However, i would like to automatically tag them as well with 2.0.0 to have a consistent state across my overall application.

Is this possible? If yes, how?
All the best

Getting the following error when running meta init:

[SyntaxError: Block-scoped declarations (let, const, function, class) not yet supported outside strict mode]

Is this related to the node vs. nodejs version issues noted here:

https://stackoverflow.com/questions/21168141/cannot-install-packages-using-node-package-manager-in-ubuntu/38325376#38325376

Is there a way to get the current folder name, when running a command via meta exec?

Here's a concrete example of what I'm hoping to accomplish (although, I could foresee a handful of other scenarios where this might be useful):

We have a meta-repo, containing a .meta file with a bunch of other projects that all depend on each other. These sub-repos contain .NET Standard projects, and I would like run a command to compile and publish (to a local feed) each project one-by-one.

For example...

rmdir %USERPROFILE%/.nuget/packages/<name-of-project> /s /q
dotnet pack src/<name-of-project>/<name-of-project>.csproj --configuration Release --output package
nuget push -Source %USERPROFILE%/.nuget/packages/ src/<name-of-project>/package/<name-of-project>.<version>.nupkg

Due to the folder structure we have in place, the command that I'm running changes depending on which project I'm building, if that makes sense.

Can we babelify meta (or buble or something) to make the es6 code into es5 for library -- otherwise the lib doesnt work for older versions of node.

Hi!

Often I get detached HEADs when working with submodules in a meta project (which is immense pain).

How can I move from submodules to meta? I want to work in a single project root and split commits into their own module-repo's.

Thank you.

I ran touch .meta, then needed package.

/Users/thomas/.nvm/versions/node/v7.2.1/lib/node_modules/meta/node_modules/meta-git/bin/meta-git-clone:51
  const projects = meta.projects;
                       ^

TypeError: Cannot read property 'projects' of null
    at exec (/Users/thomas/.nvm/versions/node/v7.2.1/lib/node_modules/meta/node_modules/meta-git/bin/meta-git-clone:51:24)
    at module.exports (/Users/thomas/.nvm/versions/node/v7.2.1/lib/node_modules/meta/node_modules/meta-exec/index.js:54:18)
    at Object.<anonymous> (/Users/thomas/.nvm/versions/node/v7.2.1/lib/node_modules/meta/node_modules/meta-git/bin/meta-git-clone:21:1)
    at Module._compile (module.js:571:32)
    at Object.Module._extensions..js (module.js:580:10)
    at Module.load (module.js:488:32)
    at tryModuleLoad (module.js:447:12)
    at Function.Module._load (module.js:439:3)
    at Module.runMain (module.js:605:10)
    at run (bootstrap_node.js:420:7)

As a user, I would like to use meta commands that wrap the node-gh app.

The main use case is interacting with pull requests, mainly list, merge, and rebase.

Example:

npm i -g meta-npm
 .
 .
 .
+ [email protected]
added 282 packages in 17.005s

meta

  Usage: meta [options] [command]

  Options:

    -V, --version  output the version number
    -h, --help     output usage information

  Commands:

    git            manage your meta repo and child git repositories
    init           initialize a new meta repo
    exec|loop      execute a command against meta repo and child repo dirs
    project        add & remove child repositories
    help [cmd]     display help for [cmd]

as you can see... no npm command available.

According to trufflesuite/truffle#893 (comment), I had to several times insert my SSH password to be able to meta git update Truffle.

This is really annoying.

Hey Matt!

Im having issues adopting this behind a proxy because of the github connection for commander module.

https://github.com/mateodelnorte/meta/blob/master/package.json#L36

Is there any way we can make this a NPM dependency?

is there a meta git checkout feature?
e.g. meta git checkout <sha#>

Doing that will checkout that <sha#> in one of the repos. It will change all the other repos to be checkedout to the latest commit at that time of that <sha#>.

The reason for this feature is that some people prefer mono repos over multi repos because they get a snapshot of the whole stack when they checkout a <sha#>. So this will cover that scenario.

Thanks!

When running the installation procedure outlined in the README.md the meta-docs.git was not found.

Which is called out in https://github.com/mateodelnorte/meta/blob/master/.meta

Hi, I ran a command meta git pull, it pulled the newly created child repo, fixed conflicts and staged changes, but when i run command meta git status, the following error is thrown
`TypeError: Cannot read property 'projects' of null
at module.exports (/Users/embrace/.nvm/versions/node/v8.11.3/lib/node_modules/meta/node_modules/meta-git/node_modules/meta-loop/index.js:13:25)
at Object. (/Users/embrace/.nvm/versions/node/v8.11.3/lib/node_modules/meta/node_modules/meta-git/bin/meta-git-branch:25:1)
at Module._compile (module.js:652:30)
at Object.Module._extensions..js (module.js:663:10)
at Module.load (module.js:565:32)
at tryModuleLoad (module.js:505:12)
at Function.Module._load (module.js:497:3)
at Function.Module.runMain (module.js:693:10)
at startup (bootstrap_node.js:191:16)
at bootstrap_node.js:612:3
here is my meta file defintion
{
"projects": {
"auth": "path to repo",
"chat": "path to repo",
"groupchat": "path to repo"
}
}

what's going on here?

keep from picking up plugins that-have-meta-later in the name.

Setting up meta is currently a manual process.

A meta attach command would automate adding projects to the .meta file, as well as .gitignore

I expected "meta" to only perform operations in each sub-directory in the .meta file. However, it also tries to perform operations in the current directory.

Is there a way to disable this, or is it a bug?

some companies enforce https-only usage of their source control systems. this causes the repository urls to essentially have a username variable in their signature.

the following will work perfectly well, in a .meta file, but for now meta project add will not know how to add projects to it.

module.exports = {
  "projects": {
    "project1": `https://${process.env.USERNAME}@stash.mycompany.com/gedtwo/project1.git`,
    "project2": `https://${process.env.USERNAME}@stash.mycompany.com/gedtwo/project2.git`
  }
}

this issue is a placeholder to documentation for this capability.

Fixed the question:

Feature request:
Hello,
As a user, I would like to please have a way to automate nightly builds for the source depending on meta tool so that

1. I don't have to create separate scripts that clone my code using meta
2. The build system knows when to clone based on the changes done in code
3. The tags are applied to the specific repositories on successful completion of a build and test.

Description:

1. Our repositories are hosted on our private bitbucket server.
2. We have docker containers in local registry as build slaves.
3. We have a scenario, when the code from one repository is needed inside another repository to build a single binary. (the code is directly compiled from the sub-repository along with the code from the top-repository). I know this is not an ideal solution, but this is the requirement at present.

I wanted to use meta to be able to maintain the sub-repository without the developers bothering about cloning each repo separately etc. Meta is very good at that.
But to tell jenkins what to build (without cloning/pulling the repositories through scripts) and to track/list code changes in a build, I guess I need a plugin.

If I create new branches via meta git checkout -b new_branch, the new branches cannot be pushed as I would expect via meta git push -u origin new_branch, with each execution in the child directories failing with error: src refspec origin does not match any.

If I manually cd into the directories and execute the command or execute it via loop "git push -u origin new_branch" it works as expected. After the new branches are present in the remote repository, meta git push works as expected.

... fails, even on public project. What i did i initialized meta then i added some repos using meta project add [name][repo] and after meta git pull i see "

[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
[name]: command 'git pull ' exited with error: Error: Command failed: git pull

And its not repo specific issue, i tried my own repos, open source plugins its always like that.

TL;DR Add property to .meta configuration to specify a local path to each project.

I just found this project and I'm really intrigued with integrating it into my development workflow. The problem I find is that I have a ton of existing repos with various stashes and remotes added to each. I could just make a meta project in the root of all my sub-projects but if I wanted to have multiple meta configs for different project groupings I would not be able to. And while I think there's merit to doing a little fall cleaning on all these, I'd love to be able to use meta without a huge change to environment vars and scripts I have for various command line tools.

Additionally, there are certain languages *cough go-lang cough* that have a specific directory that they need to live in. Sure there are ways around it but there can be weird side-effects to this.

As a user of meta I'd like to be able to edit the .meta JSON to include a local path to each git repo so I may store them where I please.

feature request - ability to specify --recurse option, to recognize child meta repos and repeat command for them and their childre

Hi,

I was unable to find if meta supports nested meta repository.
By nested meta repository, I mean a meta repo which contains others meta repos.

While running yarn in a local clone:

warning package-lock.json found. Your project contains lock files generated by tools other than Yarn. It is advised not to mix package managers in order to avoid resolution inconsistencies caused by unsynchronized lock files. To clear this warning, remove package-lock.json.

$ yarn --version
1.12.3