Code Monkey home page Code Monkey logo

phabricator-tools's Introduction

phabricator-tools

Tools and daemons for administering lots of Phabricator instances and integrating them with other tools.

Build Status

Overview

Phabricator is an awesome, open-source application for communicating and collaborating with other software developers within an enterprise.

Many enterprises may be big enough such that it makes sense to have an instance of Phabricator per area of responsiblity or product being worked on; for example you may have one Phabricator for your website and one for each of your major products.

This project aims to complement Phabricator by making installation, administration and interoperation easier. At some point it will hopefully be merged with Phabricator-actual.

Extend Phabricator easily with Arcyon

There are often house-keeping tasks which you like to be able to automate; the Phabricator 'Conduit' API is designed to let you interact with your instance over HTTP.

Arcyon wraps up the API with a command-line interface for easy scripting.

e.g. say 'poke' on all open reviews not updated for 2 weeks

$ arcyon query --min-update-age "2 weeks" --status-type open | arcyon comment --ids-file - -m 'poke'

See the intro documentation here: glossy page.

See the raw documentation here: MAN PAGES.

Install Phabricator in one step

To get up and running quickly, a Vagrant configuration is included for creating a new Linux VM and making a fully working Phabricator installation, including all its dependencies, i.e. Apache, MySQL.

If you want to provision an existing VM or machine with Phabricator you can also use the included Puppet configuration directly.

To create a new local VM serving up Phabricator

  1. $ cd vagrant && vagrant up
  2. Point a web-browser at 'http://127.0.0.1:8080' to login to your new Phabricator instance

To install within an existing VM or machine

  • Requires Puppet
  • This method works only on Ubuntu or its derivatives
  • Tested on Ubuntu 14.04 and Lubuntu 14.04 on VirtualBox
  1. $ sudo puppet apply vagrant/puppet/phabricator/manifests/default.pp --modulepath vagrant/puppet
  2. Point a web-browser at 'http://127.0.0.1' to login to your new Phabricator instance

Note: Please note that if you have disabled anonymous user access or limited privilages then you would have to specify a user who has required privilages in the default puppet manifest file as follows. Change line reading

exec { "mysql < ${phab_dir}/initial.db && ${dev_dir}/phabricator/bin/storage upgrade --force":

to

exec { "mysql -u phab < ${phab_dir}/initial.db && ${dev_dir}/phabricator/bin/storage upgrade --user phab --force":

Pre-installed Users

alice, bob, phab (administrator) All pre-installed users have the password set to password

Monitor your Phabricator instance with phab-ping

A simple wrapper around the 'conduit.ping' API which Phabricator provides, phab-ping behaves much like the regular ping as a simple health-check tool.

It requests some information from the instance running at the specified URL and reports how long Phabricator takes to respond.

$ phab-ping https://secure.phabricator.com
conduit.ping https://secure.phabricator.com/api/
request 1 : ip-10-170-222-96 : 1336 ms
request 2 : ip-10-170-222-96 : 1352 ms
request 3 : ip-10-170-222-96 : 1355 ms
request 4 : ip-10-170-222-96 : 1353 ms
request 5 : ip-10-170-222-96 : 1456 ms
^C
--- https://secure.phabricator.com/api/ conduit.ping statistics ---
5 requests processed
min / mean / max = 1336.74 / 1371.00 / 1456.22 ms

Contacts

Angelos Evripiotis ([email protected])

Credits and Acknowledgements

Thanks to the awesome guys working on the Phabricator project!

License

Apache License, Version 2.0. See license text in LICENSE.

phabricator-tools's People

Contributors

aevri avatar amblp avatar cs-shadow avatar hashpling avatar kjedruczyk avatar kpfleming avatar psjay avatar ruoso avatar sstriker avatar tchetwin avatar thosrtanner avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

phabricator-tools's Issues

Arcyd: var/log/debug rotates pretty fast, please consider zipping

For a large arcyd installation, the var/log/debug log file can rotate completely multiple times per day. This means that it's not possible to look into events from previous days.

Please consider zipping the rotated logs so that we can afford to store more logs.

Arcyd-tester: push_new_review_branch does not handles slashes in filenames

push_new_review_branch() writes a file corresponding to each branch and the name of the file is obtained simply by appending the name of branch to the filepath. This logic fails when the name of branch itself has slashes which is a valid case because the function treats it as a directory and hence fails while trying to create the file.

This is a relatively minor issue since this code is just used by the tester and not actually by Arcyd but thought it will still be nice to at least have it documented here.

arcyon task-create/update fails to identify project

Running the following command:

arcyon task-create --uri URI --user USERNAME --cert CERT --projects '#PROJECT' --format-id TASK-ID
arcyon task-update --uri URI --user USERNAME --cert CERT --projects '#PROJECT' --format-id TASK-ID

Result in the following error:

Traceback (most recent call last):
File "./arcyon", line 31, in
sys.exit(aoncmd_arcyon.main())
File "/disk1/git/phabricator-tools/py/aon/aoncmd_arcyon.py", line 97, in main
return args.func(args)
File "/disk1/git/phabricator-tools/py/aon/aoncmd_taskupdate.py", line 160, in process
project_to_phid = phlcon_project.make_project_to_phid_dict(conduit)
File "/disk1/git/phabricator-tools/py/phl/phlcon_project.py", line 72, in make_project_to_phid_dict
return {i.name: i.phid for i in query_all(conduit)}
File "/disk1/git/phabricator-tools/py/phl/phlcon_project.py", line 57, in query_all
items = query_some(conduit, window_size, 0)
File "/disk1/git/phabricator-tools/py/phl/phlcon_project.py", line 45, in query_some
results = [QueryResponse(**r) for phid, r in response.iteritems()]
File "/disk1/git/phabricator-tools/py/phl/phlsys_namedtuple.py", line 111, in make_instance
raise Error("missing attributes: " + str(missing_attr))
phlsys_namedtuple.Error: missing attributes: set(['phid', 'name'])

I haven't dug into the code yet, but my brief glance at the docs and conduit suggested that the calls above ought to work. Am I missing anything obvious here?

Arcyd: Make arcyd respect no_proxy settings

Currently arcyd takes http_proxy and https_proxy as argument and the values are passed by the environment variables. So if your proxy is set and you're using http://127.0.0.1 to access phabricator, the command errors out. This happens because it tries to access a local address via a proxy and ends up generating a '403 Access Denied'.

We should add a no_proxy parameter to the command and pass its value from environment variable, if set.

arcyon doesn't push linebreaks correctly into conduit

Thanks for the super-quick resolution of #9 . As a reward, here is another issue that I noticed.

If I run an arcyon command:

arcyon task-update --uri $URI --user $USER --cert $CERT --comment "This is a comment with a\nline break" --format-id $TASK_ID

Then I get a comment on the task that says:

This is a comment with a\nline break

Whereas, if I run this through Conduit directly (via the web interface), I get the intended result:

This is a comment with a
line break

Seems like arcyon does something along the way or uses some methods that breaks the text encoding somehow.

Arcyd: Fix smoke tests

smoke-tests fail are flaky and often fail on local machines. Following error message is thrown by phabricator:

You can not access the user inside the implementation of a  method which does not require authentication (as per shouldRequireAuthentication()).'

The error is probably by virtue of arcyd running in a multiprocess environment because arcyon works fine when ran as a single process but fails when several instances are ran together in parallel.

Arcyd: add reason for stopping to info logs

When Arcyd stops gracefully, a log entry is left in var/log/info. This is useful for knowing when Arcyd was stopped, it would also be useful to have a reason in the log too.

This could be supplied as an argument to arcyd stop and arcyd restart.

mysql user account issue when anonymous users are disabled

Usually I disable all anonymous users on a mysql installation for security reasons. I had done so with the installation where I tested this and ran in to issues with puppet default manifest which assumes you can connect to the mysql instance as an anonymous user.

We should probably list this as part of the README.md that users who have disabled anonymous access should add -u flag and --user grab as follows to the default.pp file

-    exec { "mysql < ${phab_dir}/initial.db && ${dev_dir}/phabricator/bin/storage upgrade --force":
+    exec { "mysql -u phab < ${phab_dir}/initial.db && ${dev_dir}/phabricator/bin/storage upgrade --user phab --force":

Arcyd: Increase the size of individual debug logs to 50 MB

This a follow-up to #24. When that PR is merged, the size of the logs would be reduced but the amount of logs available would still be the same. Increase the value of maxBytes for the file handler to be 50 MB which is currently 10 MB.

The (maximum) total size of the logs would be ~100 MB (50 + (50 * 10) / 10) assuming a gzip compression factor 10. On the other hand, current (maximum) total size of the logs is 110 MB (10 + (10 * 10)). So we'll be able to store 5 times more logs without actually increasing the total size of the logs stored.

arcyon task-query seems to have a bug.

Even more stuff:

arcyon task-query --cert $CERT --uri $URI --user $USER

Fails with the following traceback (most recent call last):
File "./arcyon", line 31, in
sys.exit(aoncmd_arcyon.main())
File "/disk1/git/phabricator-tools/py/aon/aoncmd_arcyon.py", line 97, in main
return args.func(args)
File "/disk1/git/phabricator-tools/py/aon/aoncmd_taskquery.py", line 247, in process
r['statusName'] = phlcon_maniphest.STATUSES[int(r['status'])]
ValueError: invalid literal for int() with base 10: 'open'

This is running against the latest Phabricator.

REQUEST: Add support for project hashtags in 'arcyon task-create --projects ...'

Just a humble suggestion :)

It would be very, very nice if you could use a project's hashtag(s) when creating a task via arcyon task-create.

I've also noticed that the workflow requires that you specify a project name with precise casing. I have a project called Gator, and the command fails if I pass --projects gator.

Thanks for making this FANTASTIC, FANTASTIC tool!

Arcyd: 'restart' can be interrupted, leaving arcyd offline

In the case where a user becomes impatient and uses Ctrl-C to abort, Arcyd will most likely stop and not be restarted.

Potentially we could not use the var/command/killfile scheme in this case and perhaps use a var/command/reloadfile scheme, so that the running daemon can have the responsibility of both stopping and starting.

REQUEST: Integration of vcs-user and Repository support.

I just used the puppet resources of this repo to setup a phabricator VM and debployment and usage worked really well.

But I somehow miss a comprehensive configuration of diffusion and the repo tool.
Creating a new repo and directly pushing to it is not possible directly.
I had to invest some time of extra configuration and doc reading to make repo hosting and integration work.
Is this something that will change in future ?

Make arcyon Windows compatible

When trying to run arcyon on a Windows environment, you get the error:

$ arcyon
Traceback (most recent call last):
  File "c:/Program Files/arcanist/phabricator-tools/bin/arcyon", line 31, in <module>
    import aoncmd_arcyon  # noqa
  File "c:\Program Files\arcanist\phabricator-tools\py\aon\aoncmd_arcyon.py", line 33, in <module>
    import aoncmd_gitdiffhelper
  File "c:\Program Files\arcanist\phabricator-tools\py\aon\aoncmd_gitdiffhelper.py", line 31, in <module>
    import abdt_branch
  File "c:\Program Files\arcanist\phabricator-tools\py\abd\abdt_branch.py", line 61, in <module>
    import abdt_differ
  File "c:\Program Files\arcanist\phabricator-tools\py\abd\abdt_differ.py", line 32, in <module>
    import phlgit_diff
  File "c:\Program Files\arcanist\phabricator-tools\py\phl\phlgit_diff.py", line 26, in <module>
    import phlsys_fs
  File "c:\Program Files\arcanist\phabricator-tools\py\phl\phlsys_fs.py", line 33, in <module>
    import fcntl
ImportError: No module named fcntl

It seems that fcntl is used to get file lock permissions, but no fcntl module exists for Windows. It's worth noting that arcanist uses the EDITOR config variable to get blocking sessions (PhutilInteractiveEditor.php) so it is possible, just not with the fcntl module.

Worth noting, I have no idea if this is the only code that's not Windows compatible, so this request may be more than just what I've listed here.

Arcyd: Create a separate error log stream for errors

Currently, we only have info INFO and DEBUG logs for arcyd so any errors that are encountered also end in INFO logs. Ideally, we'd want to have a separate log for ERROR and the errors should not be in INFO logs at all.

'vagrant up' failing --- Cannot understand environment setting "http_proxy="

I'm a complete vagrant n00b, so I'll do my best to provide the relevant info. For all I know, I just have a messed-up setup.

When following the tutorial for creating a new vm (cd vagrant && vagrant up), this happens mid-way through the setup:

warning: Exec[git clone http://github.com/facebook/arcanist /phabricator/instances/dev/arcanist](provider=posix): Cannot understand environment setting "http_proxy="
debug: Executing 'git clone http://github.com/facebook/arcanist /phabricator/instances/dev/arcanist'
err: /Stage[main]/Phabricator/Phabricator::Phabgitclone[arcanist]/Exec[git clone http://github.com/facebook/arcanist /phabricator/instances/dev/arcanist]/returns: change from notrun to 0 failed: Command exceeded timeout at /tmp/vagrant-puppet-1/manifests/phabricator/manifests/default.pp:83
debug: Exec[git clone http://github.com/facebook/phabricator /phabricator/instances/dev/phabricator](provider=posix): Executing 'git clone http://github.com/facebook/phabricator /phabricator/instances/dev/phabricator'
warning: Exec[git clone http://github.com/facebook/phabricator /phabricator/instances/dev/phabricator](provider=posix): Cannot understand environment setting "http_proxy="

If it's relevant, this is the output of vagrant box list. (the host system is OSX, btw):

$ vagrant box list
lucid32   (virtualbox)
precise32 (virtualbox)

Please let me know if you need any other information -- or if I'm just doing something wrong.

Thanks!

static_tests.sh fails to run correctly

Seems that sfood did not like the multiline string it was passed, when I run it I get thrown on to pager. Haven't found where yet. After I quit it I am back at the prompt.

$ lsb_release -a | grep Release
    Release:    12.04
$ uname -a
    Linux os-compaq 3.2.0-48-generic #74-Ubuntu SMP Thu Jun 6 19:45:16 UTC 2013 i686 i686 i386 GNU/Linux
$ bash --version
    GNU bash, version 4.2.25(1)-release (i686-pc-linux-gnu)

I have a patch that I have tested and verified on my system. Have pushed it to branch fix_static_tests branch of my fork

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.