systems-talk's People
systems-talk's Issues
inter-host communication and service discovery
The following is probably something I've ranted to you at one point or another, but here it is in word form.
I was thinking about my CoreOS cluster, and how I want to do service discovery, high-availability, circuit-breakers, etc. My plan is to run a local proxy on all hosts in the cluster, which I am going to give the incredibly overloaded name of the application router.
The app router will do a lot. I would love to separate concerns more, but I want to minimize the number of network hops involved. The routers job is to route network connections between apps in your cluster.
- your app registers what dependencies it requires somewhere (tbd)
- for each dependency, the app router uses etcd to resolve where that service resides
- a unix domain socket is created named after the service at
/services/$NAME.sock
Optionally the app router may setup a persistent TCP connection to the target host ahead of time. When a request comes in, the application should issue a single request to the local unix socket. The router then forwards the request to the remote hosts router, which forwards the request to another unix domain socket.
Only the router deals with network sockets. It might be better to have the router talk directly to an app listening on a public port, but for now I want the flexibility of intercepting requests on both sides.
There are a number of things you can introduce now
- Complete network monitoring: the app router can report stats about all incoming/outgoing connections
- Latency-monkey: the app routers can slow down your requests to test robustness of the system. You can literally dial in the chaos.
- Circuit breakers: the app router can immediately error to applications when downstream latency starts to build up.
The most interesting piece however is that the data center network is now entirely a private interface. The only public interface that applications talk to are unix domain sockets.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. ๐๐๐
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google โค๏ธ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.