Code Monkey home page Code Monkey logo

groot-api-gateway's Introduction

GROOT

Build Status

This repo contains the API Gateway written in Fall of 2015 It provides the following capabilities:

  • Easy registration of services
  • Universal Authentication for the entire application - via an external authentication provided (Atlassian crowd)
  • Proxying API calls
  • Managing inter-service communication

When registering as service specify the data encoding and when requesting a resource though groot make the request using json.

Join the chat at https://acm-uiuc.slack.com/messages/C6XGZD212/

RUNNING GROOT

Add the API spec in a new file (ex. todo.go) in the services package

There is a set of proxy api calls defined by the Arbor framework which the API Gateway is written in.

AS OF 4/24/17

/**
 *  Pass the http Request from the client and the ResponseWriter it expects
 *  Pass the target url of the backend service (not the url the client called)
 *  Pass the format of the service
 *  Pass a authorization token (optional)
 *  Will call the service and return the result to the client.
 **/
 arbor.GET(w http.ResponseWriter, url string, format string, token string, r *http.Request)
 /**
  *  Pass the http Request from the client and the ResponseWriter it expects
  *  Pass the target url of the backend service (not the url the client called)
  *  Passes the encoded json(only format currently supported) to the service.
  *  Pass a authorization token (optional)
  *  Will call the service and return the result to the client.
  **/
  arbor.POST(w http.ResponseWriter, url string, format string, token string, r *http.Request)
 /**
  *  Pass the http Request from the client and the ResponseWriter it expects
  *  Pass the target url of the backend service (not the url the client called)
  *  Passes the encoded json(only format currently supported) to the service.
  *  Pass a authorization token (optional)
  *  Will call the service and return the result to the client.
  **/
  arbor.PUT(w http.ResponseWriter, url string, format string, token string, r *http.Request)
/**
 *  Pass the http Request from the client and the ResponseWriter it expects
 *  Pass the target url of the backend service (not the url the client called)
 *  Pass a authorization token (optional)
 *  Will call the service and return the result to the client.
 **/
 arbor.DELETE(w http.ResponseWriter, url string, format string, token string, r *http.Request)

All secret data should be kept in a file called config.go in the config directory

Make sure you have go dependencey management installed

go get github.com/golang/dep/...

Install Dependencies [First time setup]

dep ensure 

Compile the service

./build.sh

CLI

groot-api-gateway [-r | --register-client client_name] [-c | --check-registration token] [-u | --unsecured]

-r | --register-client client_name

registers a client, generates a token

-c | --check-registration token

checks if a token is valid and returns name of client

-u | --unsecured

runs groot without the security layer

without args

runs groot with the security layer

Docker

Creating a client key:

docker-compose run groot-api-gateway ./build/groot-api-gateway -r <client_name>

Checking a client key:

docker-compose run groot-api-gateway ./build/groot-api-gateway -c <client_key>

Config Values

To allow all services to work with the docker network, make sure to set these values in config.go:

//Service hosts
const CreditsURL = "http://groot-credits-service:8765"
const EventsURL = "http://groot-events-service:8002"
const GigsURL = "http://groot-gigs-service:8964"
const GroupsURL = "http://groot-groups-service:9001"
const HardwareURL = "http://groot:4523/api/v1.0"
const MemesURL = "http://groot-meme-service:42069"
const MerchURL = "http://groot-merch-service:6969"
const QuotesURL = "http://groot-quotes-service:9494"
const RecruiterURL = "http://groot-recruiters-service:4567"
const UsersURL = "http://groot-users-service:8001"

//Arbor configurations
func LoadArborConfig() {
    security.AccessLogLocation = "/var/groot-api-gateway/access.log"
    security.ClientRegistryLocation = "/var/groot-api-gateway/clients.db"
    proxy.AccessControlPolicy = "*"
}

License

This project is licensed under the University of Illinois/NCSA Open Source License. For a full copy of this license take a look at the LICENSE file.

When contributing new files to this project, preappend the following header to the file as a comment:

Copyright © 2017, ACM@UIUC

This file is part of the Groot Project.  
 
The Groot Project is open source software, released under the University of Illinois/NCSA Open Source License. 
You should have received a copy of this license in a file with the distribution.

groot-api-gateway's People

Contributors

aashishkapur avatar bcongdon avatar gitter-badger avatar modi95 avatar narendasan avatar perilynn avatar sameetandpotatoes avatar theishshah avatar therishidesai avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

groot-api-gateway's Issues

Funding Request Interface

A quick standardized interface/tool to create funding requests, easier to make one and for top4 to read them off and track their completion during and after exec meetings.

API Gateway

We need an API gateway that is scalable enough to handle future services(we don't really want to rewrite this over and over). From my research I have found good implementations in lua (kong - http://getkong.org, what I am leaning towards), go (gin - https://gin-gonic.github.io/gin/) and node (loopback).
Ideally this contains the auth for the site, so we don't need to rewrite that logic over, some of the load management and support for inter service communication (some cache will be necessary for this)

Stack for Groot

Current ideas Node, Express, Mongoose, Backbone, sass, skeleton

Remove Sensitive Transactions from Logs

Just to formalize this as an issue:

Currently, when users log in to groot, their login credentials are displayed in plaintext (not sure if to file or just to console).

This should probably be addressed before deployment.

Minimal Deployable Product

In order to deploy groot, the specified features need to be:

  1. We need to be able to add new members
    • This implies we need some sort of members service built
  2. All static content needs to be up
    • As I see this we can deploy this hard coded into one of our frontend services or even one that we are going to scrap after groot comes online as static content until a full static site generator comes online.
    • This includes linking to mailman and the SIG sites which from what I gather is not too difficult especially since it seems the sites are hosted independently.
    • If we go with hardcoding this NEEDS to be temporary. Eventually we would want SIGs coming under the groot umbrella, We also want content to be easy to change which is not the case when you need to dig through code to change site content.
  3. We need LDAP support?
    * It would be nice to use University Auth for a lot of the site. But we also want a copy. So I am not sure this is necessary for deployment but we need some sort of auth. This is more of a discussion point

From the above requests I think what we need to build this summer is:

  1. A frontend service for desktop at least
    • Stack is the implementers choice. All it needs to do is be able to query the API gateway and look better than the current ACM website. It should also have all the static and SIG content from the current site in static pages at least for V1 and support for signing people up.
  2. A API gateway
    • We need an API gateway that is scalable enough to handle future services(we don't really want to rewrite this over and over). From my research I have found good implementations in lua (kong - http://getkong.org, what I am leaning towards), go (gin - https://gin-gonic.github.io/gin/) and node (loopback).
    • Ideally this contains the auth for the site, so we don't need to rewrite that logic over, some of the load management and support for inter service communication (some cache will be necessary for this)
  3. A User Service
    • From what I see I think this is the only service we need to implement before school. The schema/DB can be completely different from liquid since it looks like now we are migrating the data. Required attributes are those used for signup. Whoever takes this up, feel free to choose the framework but then be prepared to serve as support for any future updates depending on how archaic the framework is. All that needs to be exposed is the API hooks for the API gateway.

It doesn't seem like there is a lot that we need to do so we should be able to get this going before school. But we need at least one or two people on each of these if they are going to get done. Individual issues will go up for each of these so for any non architecture specific convos look there until a long term structure is built since it looks like these might be separate repos.

SQLite locking prevents creating access tokens while api-gateway is running

To reproduce:

  1. Start the groot-api-gateway docker container
  2. Run a second version of the api-gateway container with the -r command to try to create a new access token.
  3. The operation fails, giving timeout as the error message.
    (Note that the both containers must share the same data volume to persist the clients.db file)

It seems like SQLite locks the database file, so it's not possible to create a new token while the api-gateway is running.

I feel like this is a bug with bolt, or our use of it. It might be due to how docker handles sharing files in volumes between containers, I haven't tried to do this locally.

Resume Service

Service that dumps resumes that are uploaded to the ACM website into S3. Named by a hash function that sorts via netID. End point needs to be defined by hashfunction. Endpoint to pass in PDF.

API Documentation

We need a standard way of documenting the apis we create. There are few tools out there to manage this. RAML and API Blueprint are the best two options for our uses in my opinion. API Blueprint in particular allows both markup and json descriptions of our apis which is nice. RAML is straight json. API Blueprint also has cool plugins to test and display documentation. The ultimate goal is to have a tools that helps use define the API so a script in the gateway can generate the integration (if we create our own gateway).

Users Service

From what I see I think this is the only service we need to implement before school. The schema/DB can be completely different from liquid since it looks like now we are migrating the data. Required attributes are those used for signup. Whoever takes this up, feel free to choose the framework but then be prepared to serve as support for any future updates depending on how archaic the framework is. All that needs to be exposed is the API hooks for the API gateway.

Quote Database

Microservice with details on Quote, ID, Person who posted, what time they posted.

Make route handlers optional

It would be nice if you could just write the RouteCollection for defining routes, like:

Route{
    "GigInfo",
    "GET",
    "/gigs/{gig_id}",
    GigInfo,
}

... without having to create the additional boilerplate function for handling the request:

func GigInfo(w http.ResponseWriter, r *http.Request) {
    proxy.GET(w, GigURL+r.URL.String(), GigFormat, "", r)
}

It's nice to have the possibility of extensibility with these request handlers, but it often just makes the process of registering a service with {Groot, Arbor} more tedious.

End Goal Spec for Groot

This is the list of services that we would like to have before we declare groot stable

  1. Desktop UI
  2. Feed Service
    • We should have a way for top4/admin/SIG chairs to post quick updates on the ACM site.
  3. Events Service
    • SIG meetings, Techtalks, and other events should be logged, easy to create and be displayed on the site.
  4. Users Service
    • Member log, should be exposed through the ability to have user profiles, and sig membership
  5. SIGs Service
    • Log members, host site, change description, post events and host data like presentations and meeting notes
  6. Quotes Service
    • Migrate quotes, expose on UI
  7. Sponsor Service
    • Give sponsors access to resume database and log contact info
  8. TV UI
  9. Chroma Integration
  10. Caffiene Integration
  11. Mailers service
  12. Beats Service
  13. iCard Service
    • Handle and expose an api to parse i Cards
  14. Funds Service
    • Tools to manage ACM funds

Store MySQL credentials in environment

It's pretty cumbersome to necessitate a separate MySQL config in each service. We should allow services to pull MySQL credentials (user, pass, host, port) from env variables.

This will make things a lot easier for setting up a dev / prod environment, as well as for setting up Docker.

This change will need to be made in all the services that use MySQL.

Hardware Service

ACM has a good deal of hardware that both SIGs and members use. It would be nice if we had a master list of serial numbers and a quick way to check out and see what is available. Would allow us to track the usage of ACM's assets better.

Desktop Frontend Service

We need at least for the time being we need a front end that replicates the current site

  • It should look better
  • Most of the content can be hardcoded but this will be removed
  • The intranet does not need to be ported for fall
  • Integrates with groot api to support member registration

Non-200 HTTP Response Bodies Dropped by proxy

Error responses are often not getting conveyed through the proxy. Consider: https://github.com/acm-uiuc/groot/blob/master/proxy/GET.go#L49

Suppose you reject a request service-side with a 403. err is nil, but res.StatusCode != http.StatusOK, so we call InvalidGET(w, err). Thus, the client sees a 404, get's nil as the server-error. (And doesn't get the error message attached to the original 403)

Obviously, we want to do some sort of panic on a non-200 response, but it would be worth forwarding the response body anyways, IMO.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.