Comments (5)
metson: Could it run in the same webserver (I assume you're using cherrypy/WMCore.Webtools) as the rest of the DAS pages? Why would you want it as a separate instance?
from das.
valya: It can, but rather I want this separation. I don't want to expose DAS analytics to end-users. I want to have separation for maintenance and scheduling. I want to put DAS analytics to different node eventually.
from das.
metson: Hmmm, ok. How does that work in the clustered environment that you're targeting? Surely this is something you should protect with authorisation, rather than running on another port (which will still require the auth.)?
from das.
valya: We need to pick up a node where we will run analytics. Since it has scheduler, it means we can assign as many tasks as we want to the analytics server. Therefore the node will be pretty busy. The clustered environment still works, since I made DAS core transparent to connection pooling. Analytics invokes DAS core, which by itself can connect to Mongo URIs (routers in case of shards or master in case of replica pairs) which redirects requests to shards/replicas pairs.
Regarding ports. Right now we have DAS web server on port 8212, the DAS cache server on port 8211 and we will have DAS analytics server on port 8213. Such separation allows to assign different nodes for every server it is required, I consider this flexibility as benefit rather complication.
The DAS cache server is not visible to end-users at all, since there is no redirect apache rules for it.
The DAS web server is visible to all users and it talks to DAS cache server. Right now I want to hide analytics too from end-users, but make it visible to CERN operators. Here we have two options. Either CERN operator connect to dedicated DAS analytics node via host:port, or we set group authorization to allow certain people to access it via cmsweb. As you well know the group authorization will require OpenID stuff, which as far as I know is not yet in place. So we left with first option so far.
This certainly needs to be reviewed/adjusted as time goes by, but during commissioning phase I opted with non-visible DAS analytics running on specified host/port.
from das.
valya: This is done and accepted by HTTP group.
from das.
Related Issues (20)
- pagination links do not remember DBS instance in DAS2GO HOT 3
- Handling of large run range queries HOT 4
- Duplicate entries in JSON output HOT 2
- Perform final migration of DAS python server to DAS go server HOT 3
- Display of Fraction of Datasets with invalid Files HOT 20
- Given a block name and a run number, query for file names HOT 2
- status filter not working HOT 3
- 'plain' option sometimes misbehaves HOT 3
- DAS not found but by "crab status" it exists HOT 2
- Config files not available from the Configs link in web queries HOT 7
- Error="invalid character 'p' after array element" HOT 14
- "DAS query guide" link points to nowhere HOT 1
- finding the file that contains a run,lumi is not working HOT 6
- error=invalid character '<' looking for beginning of value" HOT 2
- for results format = list, total number of entries is not correct HOT 1
- panic: runtime error from dasgoclient HOT 2
- Turn on Rucio requests from DAS by default HOT 1
- Explore data consistency checking between Rucio and PhEDEx HOT 3
- Remove Rucio-Account header from command line version of DAS HOT 2
- Remove PhEDEx from default list of services to query HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from das.