glavin001 / gitlab-pages Goto Github PK

View Code? Open in Web Editor NEW

203.0 19.0 30.0 33 KB

:eyes: GitHub Pages, for GitLab.

License: MIT License

JavaScript 84.42% CSS 0.53% Shell 1.87% HTML 13.17%

gitlab-pages's Introduction

GitLab Pages

GitHub Pages, for GitLab.

Author

Glavin Wiechert (Glavin001) - Twitter @GlavinW - LinkedIn

Installation

Requirements

It requires Jekyll to be installed. GitLab Pages will build Jekyll sites or simply copy content from non-jekyll sites to publicPagesDir.

Step 1) Create your deploy key pair

Create a SSH key pair for GitLab to use as deploy keys.

ssh-keygen -t rsa -C "[email protected]" -f "$(pwd)/keys/id_rsa"

They should be in path keys/id_rsa and keys/id_rsa.pub.

Step 2) Configure GitLab Pages server

Clone this repo and install it's necessary dependencies:

# Already cloned repo
npm install
bower install

Copy default config to custom config file.

cp default_config.js _config.js

Edit your new _config.js file for your setup.

Step 3) Start your GitLab Pages server

And start the server!

npm start

Deploy with Docker

Docker support is also available and the following env vars are used to configure the installation:

GITLAB_URL - Url to your gitlab server
DEPLOY_BRANCH - Specific branch that will be deployed when pushed
DEPLOY_PAGEDIR - Public directory of GitLab Pages
SERVER_URL - The public gitlab-pages server url

The /app/keys directory SHOULD be mount using docker volumes, this will allow you to use leave the ssh key outside of the docker. If the rsa key doesn't exist the docker_start.sh script will generate it.

Profit!

How It Works

User logs into Web UI, which authenticates against GitLab with node-gitlab and retrieves a private key for subsequent requests.
Select the Project to enable GitLab Pages support for it. Available Projects list is pulled from GitLab with node-gitlab.
The selected project will setup a Deploy Key and Webhook for GitLab Pages. (Note: when adding a Deploy Key, type "GitLab Pages" as a Title)
When the user pushes to GitLab, a webhook event is sent to GitLab Pages server.
The Webhook is handled by gitlab-webhook and the project is pulled (or cloned) with nodegit to the GitLab Pages configured clones install directory.
The gl-pages branch is checked out with nodegit. (This can optionally be configured to be gh-pages branch to work on both GitHub and GitLab).
The public content, a GitLab Page, is served using Expressjs. Each project will be served from a different subdirectory in the form of server.publicUrl/pages/:username/:projectName/. It is recommended that the publicPagesDir be served using Nginx or Apache.

Libraries & Tools

gitlab-pages's People

Contributors

Stargazers

Watchers

gitlab-pages's Issues

Start the server?

Hello,

So I started the server.
The publicUrl is set to http://IP:8081

[email protected] start /var/www/vhosts/pages
node ./bin/www

But when I go to IP, with port 8081 - I can't see anything, page not found.
Other problem is, that netstat -na does not show, that port 8081 is listening.

Any ideas? Thank you!

Clone Git repo from GitLab

Use SSH Key (Deploy key)
Clone to public directory
checkout commit from push

How to install properly

First of all, thank you for creating this. I have been looking for something like this for a long time!

Should I make a gitlab_pages user to run it?
Where should I install it to (i.e. /opt/gitlab_pages)?
Would you recommend installing a system-ctl service to automatically start it?
I know that it's on the same server in which the repositories aren't actually encrypted in the first place, but can I use an id_ecdsa key?

Switch to using nconf for configuration

See https://github.com/indexzero/nconf

nconf supports:

Command-line arguments
Environment variables
A file located at 'path/to/config.json'

Environment variables is important for automated deployment such as Docker ( see #19 , #18 ) and testing on Travis CI ( #10 ), etc. Currently only the _config.json file is supported. I would like to standardize the options and allow command-line, environment variables, along with the optional file support.

Error: authentication required but no callback set

I just grab a new copy in a new setup and get the following error when pushing to my pages branch:

routes:webhooks url +0ms git@gitlab:morph027/gl-pages-test.git
  morgan log request +52ms
POST /webhooks/pages.json 200 21.610 ms - -
[Error: authentication required but no callback set]
  routes:webhooks fetch all +51ms undefined { ignoreCertErrors: 1,
  remoteCallbacks: { credentials: [Function] } }

shows 404 error on Jekyll error

It would be nice if the old version of the page could still be served and the user who pushed would be notified about the error via email or similar. That way jekyll errors wouldn't result in accidental downtime.

Publish global CLI app to npm

This should also make it easier to install ( #23 ).

TODO:

not working - 404

I have long tried to make it work, but still have problem with it.
what work:
deploy key and webhook sucsesfully added, i can enable and disable projects in list.
what doesn't work:
after i commit into gl-pages branch (or master, does not matter) i've got only created new folder (as i configure as tmp) and its all. in DEBUG=* npm start:

express:router dispatching POST /webhooks/pages.json +33s
  express:router query  : /webhooks/pages.json +3ms
  express:router expressInit  : /webhooks/pages.json +1ms
  express:router session  : /webhooks/pages.json +1ms
  express-session no SID sent, generating session +2ms
  express:router logger  : /webhooks/pages.json +5ms
  express:router jsonParser  : /webhooks/pages.json +4ms
  body-parser:json content-type "application/json" +0ms
  body-parser:json content-encoding "identity" +5ms
  body-parser:json read body +0ms
  body-parser:json parse body +32ms
  body-parser:json parse json +1ms
  express:router urlencodedParser  : /webhooks/pages.json +0ms
  body-parser:urlencoded body already parsed +0ms
  express:router cookieParser  : /webhooks/pages.json +0ms
  express:router serveStatic  : /webhooks/pages.json +1ms
  express:router router  : /webhooks/pages.json +0ms
  express:router dispatching POST /webhooks/pages.json +1ms
  express:router trim prefix (/webhooks) from url /webhooks/pages.json +1ms
  express:router router /webhooks : /webhooks/pages.json +0ms
  express:router dispatching POST /pages.json +0ms
  routes:webhooks /pages.json +2ms
  routes:webhooks deploy ref: refs/heads/gl-pages, ref: refs/heads/gl-pages +0ms
  routes:webhooks config +0ms { gitlab: { url: 'http://my.gitlab.domain/' },
  deploy:
   { sshPublicKey: '/var/opt/pages/keys/id_rsa.pub',
     sshPrivateKey: '/var/opt/pages/keys/id_rsa',
     deployBranch: 'gl-pages',
     publicPagesDir: '/var/opt/pages/pages',
     tmpPagesDir: '/var/opt/pages/.tmp' },
  server: { publicUrl: 'http://my.gitlab.domain' } }
  routes:webhooks repoPath +4ms /var/opt/pages/.tmp/yan/test
  routes:webhooks url +0ms [email protected]:yan/test.git
  morgan log request +12ms
POST /webhooks/pages.json 200 51.923 ms - -

and i got nothing in my publicPagesDir, so when i try to load my page - i have 404..

express:router dispatching GET /pages/yan/test +35m
  express:router query  : /pages/yan/test +2ms
  express:router expressInit  : /pages/yan/test +0ms
  express:router session  : /pages/yan/test +1ms
  express-session fetching fN8fN58T5V1bApRU7MwgdG5InN6UZuF7 +5ms
  express-session no session found +2ms
  express:router logger  : /pages/yan/test +0ms
  express:router jsonParser  : /pages/yan/test +2ms
  body-parser:json skip empty body +0ms
  express:router urlencodedParser  : /pages/yan/test +0ms
  body-parser:urlencoded skip empty body +1ms
  express:router cookieParser  : /pages/yan/test +0ms
  express:router serveStatic  : /pages/yan/test +1ms
  send stat "/var/opt/pages/public/pages/yan/test" +3ms
  express:router router  : /pages/yan/test +1ms
  express:router dispatching GET /pages/yan/test +1ms
  express:router trim prefix (/pages) from url /pages/yan/test +0ms
  express:router serveStatic /pages : /pages/yan/test +0ms
  send stat "/var/opt/pages/pages/yan/test" +1ms
  express:router <anonymous>  : /pages/yan/test +1ms
  express:router <anonymous>  : /pages/yan/test +0ms
  express:view lookup "error.handlebars" +2ms
  express:view stat "/var/opt/pages/views/error.handlebars" +0ms
  express:view render "/var/opt/pages/views/error.handlebars" +1ms
  morgan log request +70ms
GET /pages/yan/test 404 79.668 ms - 1566
  express:router dispatching GET /stylesheets/style.css +9ms
  express:router query  : /stylesheets/style.css +1ms
  express:router expressInit  : /stylesheets/style.css +0ms
  express:router session  : /stylesheets/style.css +0ms
  express-session fetching fN8fN58T5V1bApRU7MwgdG5InN6UZuF7 +1ms
  express-session no session found +0ms
  express:router logger  : /stylesheets/style.css +1ms
  express:router jsonParser  : /stylesheets/style.css +1ms
  body-parser:json skip empty body +0ms
  express:router urlencodedParser  : /stylesheets/style.css +5ms
  body-parser:urlencoded skip empty body +0ms
  express:router cookieParser  : /stylesheets/style.css +0ms
  express:router serveStatic  : /stylesheets/style.css +1ms
  send stat "/var/opt/pages/public/stylesheets/style.css" +2ms
  send pipe "/var/opt/pages/public/stylesheets/style.css" +2ms
  send modified Sat, 06 Feb 2016 14:19:12 GMT +1ms
  send etag W/"6e-3110077135" +1ms
  send content-type text/css +1ms
  send not modified +2ms
  morgan log request +2ms
GET /stylesheets/style.css 304 16.051 ms - -

i try to instal it several times in different OS, (for different packages version) but the result steel the same.
Please help me to find the reason and and make it work.

Show User's Projects to enable Pages

Login to GitLab with User's Token
List Projects
Button to toggle (enable/disable) Pages

Cannot find module nodegit.node

This is on CentOS 6. Saw previous issues and tried those recommendations (deleting node_modules and npm). Permissions are all properly set. Scratching my head:

[autodeploy@GIT GitLab-Pages]$ DEBUG=* npm start

[email protected] start /home/autodeploy/GitLab-Pages
node ./bin/www

express:router:route new / +0ms
express:router:layer new / +1ms
express:router:route get / +2ms
express:router:layer new / +0ms
express:router:route new /login +0ms
express:router:layer new /login +0ms
express:router:route post /login +0ms
express:router:layer new / +0ms
express:router:route new /logout +0ms
express:router:layer new /logout +0ms
express:router:route get /logout +0ms
express:router:layer new / +0ms
express:router:route new / +1ms
express:router:layer new / +0ms
express:router:route get / +0ms
express:router:layer new / +0ms
express:router:route new /:project_id/enable +0ms
express:router:layer new /:project_id/enable +0ms
express:router:route get /:project_id/enable +1ms
express:router:layer new / +0ms
express:router:route new /:project_id/disable +0ms
express:router:layer new /:project_id/disable +0ms
express:router:route get /:project_id/disable +0ms
express:router:layer new / +0ms

module.js:340
throw err;
^
Error: Cannot find module '../build/Debug/nodegit.node'
at Function.Module._resolveFilename (module.js:338:15)
at Function.Module._load (module.js:280:25)
at Module.require (module.js:364:17)
at require (module.js:380:17)
at Object. (/home/autodeploy/GitLab-Pages/node_modules/nodegit/lib/nodegit.js:17:12)
at Module._compile (module.js:456:26)
at Object.Module._extensions..js (module.js:474:10)
at Module.load (module.js:356:32)
at Function.Module._load (module.js:312:12)
at Module.require (module.js:364:17)
npm ERR! weird error 8
npm ERR! not ok code 0
[autodeploy@GIT GitLab-Pages]$ node -v
v0.10.42
[autodeploy@GIT GitLab-Pages]$ npm -v
1.3.6

Use deploy key

See https://github.com/nodegit/nodegit/blob/ccca4d3204ef96a4e693d996876ff93cb8ce1a07/test/tests/clone.js#L70-L88

Highlighting code syntax, markdown tables, mistaken text, etc

Hi,

When I try to insert something like code snippets, tables, mistaken text, etc, it looks like that it is not fully generated to html. Maybe some plugins like pygments are missing? Maybe current GitLab-Pages doesn't have these features yet? Maybe I'm missing something very obvious?

Any suggestions for an easy way to add standard markdown syntax features like ones mentioned in the beginning of post?

Thanks

Error: Cannot find module '../build/Debug/nodegit.node'

Hello, I installed docker-gitlab in the virtual machine, but when I'm in the project npm start when the following error:

`root@a-virtual-machine:/data/project/GitLab-Pages# npm start

[email protected] start /data/project/GitLab-Pages
node ./bin/www

module.js:327
throw err;
^

Error: Cannot find module '../build/Debug/nodegit.node'
at Function.Module._resolveFilename (module.js:325:15)
at Function.Module._load (module.js:276:25)
at Module.require (module.js:353:17)
at require (internal/module.js:12:17)
at Object. (/data/project/GitLab-Pages/node_modules/nodegit/lib/nodegit.js:16:12)
at Module._compile (module.js:409:26)
at Object.Module._extensions..js (module.js:416:10)
at Module.load (module.js:343:32)
at Function.Module._load (module.js:300:12)
at Module.require (module.js:353:17)
at require (internal/module.js:12:17)

npm ERR! Linux 3.19.0-25-generic
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "start"
npm ERR! node v4.3.2
npm ERR! npm v3.7.5
npm ERR! code ELIFECYCLE
npm ERR! [email protected] start: node ./bin/www
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] start script 'node ./bin/www'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the GitLab-Pages package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! node ./bin/www
npm ERR! You can get information on how to open an issue for this project with:
npm ERR! npm bugs GitLab-Pages
npm ERR! Or if that isn't available, you can get their info via:
npm ERR! npm owner ls GitLab-Pages
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR! /data/project/GitLab-Pages/npm-debug.log`

sorry, my English is very general, through the google translation tool, please forgive

Docker support

See https://github.com/csakoda/GitLab-Pages#docker and #18

Automatically create pages after enabling project

It is probably would be best to automatically trigger freshly 'Enabled' project.
So it would be automaticly available after enabling.

Git Deploy Manager

See https://gist.github.com/oodavid/1809044 for how-to Git Deploy in PHP.

Install for Dummies

Following your guide I was able to get GitLab-Pages up, however everything except the index throws a 404.
What works
Login
Enable Project

What doesn't work
/webhooks dir throws a 404
/pages dir throws a 404

I am sure I am just doing something wrong.

Clone to working directory
Run jekyll build --destination <destination>

Can not enable project

I followed your guid in the Readme.md file and after starting the server, I'm able to see my projects. But when I click on the enable button nothing happens. Output from npm start on the commanlind:

 GET /stylesheets/style.css 304 1.647 ms - -
 GET /projects/37/enable 302 264.973 ms - 58

Any ideas how to start debugging? Also I can not see any Deploy Keys or Wehooks for the project I enabled, do they have to appear "normal" in the project settings view?
I'm using a selfhosted gitlab server if that matters.

Thanks for helping,
Flo

Cannot build the Docker image

When I just check out the source and do a simple docker build, after a lengthy compilation I get:

npm ERR! Linux 3.19.0-39-generic
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "install"
npm ERR! node v5.2.0
npm ERR! npm  v3.3.12
npm ERR! path /app/node_modules/nodegit/node_modules/amdefine
npm ERR! code EXDEV
npm ERR! errno -18
npm ERR! syscall rename

npm ERR! EXDEV: cross-device link not permitted, rename '/app/node_modules/nodegit/node_modules/amdefine' -> '/app/node_modules/amdefine'
npm ERR!
npm ERR! If you need help, you may report this error at:
npm ERR!     <https://github.com/npm/npm/issues>

npm ERR! Please include the following file with any support request:
npm ERR!     /app/npm-debug.log

I am using the official node image because the one from Google is deprecated.

It fails at step 9: RUN npm install -g bower && npm install && bower --allow-root install

Allow custom user domains

Allow users to specify a domain for example like GitHub does with a CNAME file.

bind localhost only

Hello,

I have two ideas for now:

I think there somehow should be possibility, to bind to localhost only.
For example in my scenario, I have set up nginx proxy, like html.website.com; so I don't have port or something in the url.

I am not a nodejs guy, to solve these issues:)

Thank in advance!

Problem with SSH key.

Hi!
I'm trying to run Gitlab Pages but Gitlab Pages always crash. I followed the steps on Readme. I can open Gitlab Pages web page with my private token and select the project. When I clicked on Enable, the deploy key and a web hook is created on Gitlab. I'm running Gitlab on address 192.168.1.1:10081 and Gitlab Pages is on 192.168.1.1:3000. The variable publicUrl is set to 192.168.1.1:3000. When I push gl-pages branch to gitlab the server npm stop and I get the error:

/home/docker/GitLab-Pages/node_modules/nodegit/node_modules/nodegit-promise/node_modules/asap/asap.js:45
                throw e;
                      ^
Error: Failed to authenticate SSH session: Username/PublicKey combination invalid
npm ERR! weird error 8
npm WARN This failure might be due to the use of legacy binary "node"
npm WARN For further explanations, please read
/usr/share/doc/nodejs/README.Debian

SSH key was created like described on README.
If I click on project that I Enabled on Gitlab Pages web page I get this error:

Error: Not Found
    at app.use.res.render.message (/home/docker/GitLab-Pages/app.js:56:15)
    at Layer.handle [as handle_request] (/home/docker/GitLab-Pages/node_modules/express/lib/router/layer.js:82:5)
    at trim_prefix (/home/docker/GitLab-Pages/node_modules/express/lib/router/index.js:302:13)
    at /home/docker/GitLab-Pages/node_modules/express/lib/router/index.js:270:7
    at Function.proto.process_params (/home/docker/GitLab-Pages/node_modules/express/lib/router/index.js:321:12)
    at next (/home/docker/GitLab-Pages/node_modules/express/lib/router/index.js:261:10)
    at SendStream.error (/home/docker/GitLab-Pages/node_modules/express/node_modules/serve-static/index.js:107:7)
    at SendStream.EventEmitter.emit (events.js:95:17)
    at SendStream.error (/home/docker/GitLab-Pages/node_modules/express/node_modules/send/index.js:244:17)
    at SendStream.onStatError (/home/docker/GitLab-Pages/node_modules/express/node_modules/send/index.js:340:48)

I'm little confused about setting, Is missing something? Gitlab version is 7.8.4.

Bur I see another problem. There are simple users at gitlab, when they try to login I simply get 'no data received' error from GitlabPages in Chrome.

In the logs I see:

POST /login 302 25.387 ms - 58

> [email protected] start /var/www/vhosts/pages
> node ./bin/www

Any idea what can be done?

Thank you!

Docker

Hello,
is anyone able to build the docker image?

I am getting different errors...

I was trying the different node image, but similar results.

Can someone post a working Dockerfile
or guide me on how to run it via docker?

Thank you

Add Unit Tests

See https://www.npmjs.com/package/super-request

Tests
Travis CI