Better Visualizations Catalyzed by Better Data
timrdf / prizms Goto Github PK
View Code? Open in Web Editor NEWCatalyzing better visualizations with better data -- being your own third party.
Catalyzing better visualizations with better data -- being your own third party.
e.g. Jim with http://visualizing.org/datasets/economic-return-education-2012
Prizms can use existing upstream LODSPeaKrs by referencing them within settings.inc.php.
find: `/home/semantgeo/opt/prizms/lodspeakrs': No such file or directory
/home/bashby/opt
Apache Jena requires the shell environent variable JENAROOT to be set.
For details, see https://github.com/timrdf/csv2rdf4lod-automation/wiki/Apache-Jena
The following command should appear in your data/source/csv2rdf4lod-source-me-as-bashby.sh.
export JENAROOT=
Add this command to your data/source/csv2rdf4lod-source-me-as-bashby.sh? [y/n] n
We didn't your data/source/csv2rdf4lod-source-me-as-bashby.sh, so you'll need to make sure you set the paths correctly each time.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Try to install virtuoso at /opt? (note: sudo *required*) (y/N) y
curl -L -o virtuoso.tar.gz --progress-bar http://sourceforge.net/projects/virtuoso/files/latest/download from /opt
######################################################################## 100.0%
sudo tar xzf virtuoso.tar.gz
sudo aptitude build-dep virtuoso-opensource
Unable to find an archive "stable" for the package "virtuoso-opensource"
Unable to find the source package for "virtuoso-opensource".
Unable to find an archive "stable" for the package "virtuoso-opensource"
Unable to find the source package for "virtuoso-opensource".
No packages will be installed, upgraded, or removed.
0 packages upgraded, 0 newly installed, 0 to remove and 26 not upgraded.
Need to get 0 B of archives. After unpacking 0 B will be used.
sudo dpkg-buildpackage -rfakeroot
dpkg-buildpackage: warning: using a gain-root-command while being root
dpkg-buildpackage: export CFLAGS from dpkg-buildflags (origin: vendor): -g -O2
dpkg-buildpackage: export CPPFLAGS from dpkg-buildflags (origin: vendor):
dpkg-buildpackage: export CXXFLAGS from dpkg-buildflags (origin: vendor): -g -O2
dpkg-buildpackage: export FFLAGS from dpkg-buildflags (origin: vendor): -g -O2
dpkg-buildpackage: export LDFLAGS from dpkg-buildflags (origin: vendor):
dpkg-buildpackage: source package virtuoso-opensource
dpkg-buildpackage: source version 6.1.6
dpkg-buildpackage: source changed by OpenLink Virtuoso Maintainer <[email protected]>
dpkg-buildpackage: host architecture amd64
dpkg-source --before-build virtuoso-opensource-6.1.6
dpkg-checkbuilddeps: Unmet build dependencies: gperf (>= 2.7.2) libreadline5-dev
dpkg-buildpackage: warning: Build dependencies/conflicts unsatisfied; aborting.
dpkg-buildpackage: warning: (Use -d flag to override.)
dpkg -i ./virtuoso-opensource_6.1.6_amd64.deb
dpkg: error processing ./virtuoso-opensource_6.1.6_amd64.deb (--install):
cannot access archive: No such file or directory
Errors were encountered while processing:
./virtuoso-opensource_6.1.6_amd64.deb
bash-3.2$ head foaf.rdf.graffle
<plist xmlns:grddl="http://www.w3.org/2003/g/data-view#" version="1.0"
grddl:transformation="https://raw.github.com/timrdf/vsr/master/src/xsl/grddl/graffle.xsl">
<dict>
bash-3.2$ open foaf.rdf.graffle
bash-3.2$ head foaf.rdf.graffle
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>ActiveLayerIndex</key>
Prizms deploys DataFAQs services by linking to them from within the htdocs directory, which is currently /var/www
sudo ln -s /home/semantgeo/opt/prizms/repos/DataFAQs/services /var/www/services
Q: May we link the DataFAQs services from your htdocs directory using the command above? [y/n] y
sudo ln -s /home/semantgeo/opt/prizms/repos/DataFAQs/services /var/www/services
ln: creating symbolic link `/var/www/services': File exists
e.g. http://ieeevis.tw.rpi.edu Prizms node should pull the datasets listed in http://datahub.io/group/ieee-vis. This requires an addition to --upstream-ckan
install argument. Just check to see if group/X
is in that URL and "know" to only pull from that group?
<http://purl.org/twc/health/source/healthdata-tw-rpi-edu/dataset/cr-full-dump/version/latest>
dcterms:modified "2013-01-08T19:21:00+00:00"^^xsd:dateTime .
[3/14/13 5:05:01 PM] Tim L: <http://purl.org/twc/health/void> a void:Dataset;
void:rootResource <http://purl.org/twc/health/void>;
void:subset <http://purl.org/twc/health/source/healthdata-tw-rpi-edu/dataset/cr-full-dump>;
https://github.com/timrdf/csv2rdf4lod-automation/wiki/One-click-data-dump
on grindstone with difrad
git pull
In settings.inc.php, add
$conf['components']['services'][] = '/var/www/visualbox/visualbox/components/services/kubrick_filmography';
$conf['components']['types'][] = '/var/www/visualbox/visualbox/components/types/foaf:Agent';
--------------------------------------------------------------------------------------------------------
You can now visit http://purl.org/twc/semantgeo/ to navigate through your data.
--------------------------------------------------------------------------------------------------------
sudo a2enmod rewrite | grep 'already enabled'
The Apache2 module rewrite needs to be enabled to
run LODSPeaKr.
The rewrite module needs to be enabled, which can be done with the following command:
sudo a2enmod rewrite
Q: May we enable the module above using the command above? [y/n] y
Q: Apache needs to restart for rewrite to take effect. Restart apache? [y/n] y
Okay, we didn't restart Apache, but you'll need to restart it for rewrite to take effect.
sudo a2enmod php5 | grep 'already enabled'
(module php5 is already enabled: Module php5 already enabled)
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
LODSPeaKr needs .htaccess
.htaccess files only work if the 'AllowOverride All' directive is set in /etc/apache2/sites-available/default, similar to:
https://github.com/tetherless-world/iogds/blob/master/.gitignore
.refresh-prizms-installation
.refresh-prizms-installation
.refresh-prizms-installation
.refresh-prizms-installation
bashby@publicserver:~/prizms/semantgeo$ ./.refresh-prizms-installation pull
./.refresh-prizms-installation: line 3: pushd: /home/lebot/opt/prizms: No such file or directory
Already up-to-date.
./.refresh-prizms-installation: line 3: popd: directory stack empty
mcat-users.xml does not establish you as a manager role.
Q: Please specify a password for administering tomcat (default will be bashby528): deploy FAqT (SADI) Services implemented in Java.
The tomcat6-admin package needs to be installed, which can be done with the following command:
sudo apt-get install tomcat6-admin
Q: May we install the package above using the command above? [y/n] y
sudo apt-get install tomcat6-admin
iogds VM.
Hitting enter one hangs up to raptor-utils
Q: May we try to install the dependencies listed above? (We'll need root for most of them) [y/n] y
[okay] git already available at /usr/bin/git
sudo apt-get install openjdk-6-jre
[okay] awk already available at /usr/bin/awk
[okay] curl already available at /usr/bin/curl
sudo apt-get install raptor-utils
sudo apt-get install unzip
sudo apt-get install screen
sudo apt-get install tidy
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
[okay] perl -MCPAN install YAML
[okay] perl -MCPAN install URI::Escape
[okay] perl -MCPAN install Data::Dumper
[okay] perl -MCPAN install HTTP:Config
[okay] perl -MCPAN install LWP:UserAgent
[okay] perl -MCPAN install IO::Socket::SSL
[okay] perl -MCPAN install Text:CSV
[okay] perl -MCPAN install Text::CSV_XS
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
sudo apt-get install python-setuptools
[okay] python egg "surf" is already available at (surf 1)
[okay] python egg "surf.sesame2" is already available at (surf.sesame2 1)
[okay] python egg "surf.sparql_protocol" is already available at (surf.sparql.protocol 1)
[okay] python egg "surf.rdflib" is already available at (surf.rdflib 1)
[okay] python egg "python-dateutil" is already available at (python.dateutil 1)
[okay] python egg "ckanclient" is already available at (ckanclient 1)
[okay] git already available at /usr/bin/git
[okay] curl already available at /usr/bin/curl
sudo apt-get install raptor-utils
Could not find rapper on path. Try to install with command shown above? (y/n):
LODSPeaKr needs to know three (3) URIs to minimally configure itself:
(1/3) At what URL will /var/www/ be available? (e.g. http://localhost/www/)
**Note** DO NOT include 'lodspeakr/' in the URL
(default 'http://localhost/my/data'):
same as from --our-base-uri http://aquarius.tw.rpi.edu/projects/ieeevis
iogds
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=- lebot
Prizms uses the shell environment variable DATAFAQS_HOME
to find the scripts that DataFAQs needs to run.
see https://github.com/timrdf/DataFAQs/wiki/DATAFAQS-environment-variables
DATAFAQS_HOME is currently set to '/home/nidhirastogi/opt/prizms/repos/DataFAQs' in data/faqs/datafaqs-source-me.sh
Q: May we change DATAFAQS_HOME to '/home/lebot/opt/prizms/repos/DataFAQs' in data/faqs/datafaqs-source-me.sh? [y/n]
Expose the developer clone, e.g., /home/lebot/prizms/lofd/lodspeakr/components/
at http://lofd.tw.rpi.edu/~lebot.
Commit/pushing from there into https://github.com/timrdf/lofd/tree/master/lodspeakr allows the lofd
user to git pull
into /home/lofd/prizms/lofd/lodspeakr
to deploy into production http://lofd.tw.rpi.edu.
as iogds user:
[okay] serdi available at
currently uses rapper -q -i ntriples -o rdfxml -I
pwd/$artifact
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
We're all done installing Prizms production environment for the user asthma.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
We're all done installing Prizms development environment for the user difrad.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Now what?
* Check out the data site http://lod.theasthmafiles.org/
* Check out the SPARQL endpoint http://lod.theasthmafiles.org/sparql
* Start committing DCAT and eparams into github repository [email protected]:difrad/semantic-asthma-files.git
fatal: destination path 'semantic-asthma-files' already exists and is not an empty directory.
It seems that you didn't have permissions to clone [email protected]:difrad/semantic-asthma-files.git
GitHub requires an ssh key to check out a writeable working clone
See https://help.github.com/articles/generating-ssh-keys
[email protected]:difrad/semantic-asthma-files.git is already clone'd into semantic-asthma-files; pull'ing it...
Enter passphrase for key '/home/difrad/.ssh/id_dsa':
Already up-to-date.
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Prizms uses the shell environment variable CSV2RDF4LOD_BASE_URI
to indicate the Linked Data base URI to use for all datasets that it creates.
see https://github.com/timrdf/csv2rdf4lod-automation/wiki/Conversion-process-phase:-name
(CSV2RDF4LOD_BASE_URI is already correctly set to 'http://lod.theasthmafiles.org' in data/source/csv2rdf4lod-source-me-for-asthma.sh)
then
However, the following do not seem to be installed:
[TODO] ln /home/difrad/opt/prizms/repos/DataFAQs/services/sadi/sadi-services.war /var/lib/tomcat6/webapps/
Q: May we try to install the dependencies listed above? (We'll need root for most of them) [y/n]
Q: May we try to install the dependencies listed above? (We'll need root for most of them) [y/n] y
[okay] git already available at /usr/bin/git
[okay] java already available at /usr/bin/java
[okay] awk already available at /usr/bin/awk
[okay] curl already available at /usr/bin/curl
[okay] rapper already available at /usr/bin/rapper
[okay] unzip already available at /usr/bin/unzip
sudo apt-get install screen
sudo a2enmod proxy_http | grep 'already enabled'
The Apache2 module proxy_http needs to be enabled to
expose your (port 8080) Tomcat application server of SADI services at the URL http://purl.org/twc/semantgeo/sadi-services.
The proxy_http module needs to be enabled, which can be done with the following command:
sudo a2enmod proxy_http
Q: May we enable the module above using the command above? [y/n] y
Q: Apache needs to restart for proxy_http to take effect. Restart apache? [y/n] y
Okay, we didn't restart Apache, but you'll need to restart it for proxy_http to take effect.
e.g. http://datahub.io/dataset/twc-ieeevis
add to group http://datahub.io/group/prizms
The installer currently is hard-coded to use git://github.com/timrdf/DataFAQs.git but if one wanted to use their own fork with their own services, they'd need to be able to set a parameter in the installer to use that instead. This affects bin/install/prizms-dependency-repos.sh
and change_source_me $target DATAFAQS_PROVENANCE_CODE_RAW_BASE
in bin/install.sh
per Brendan
iogds VM
sudo easy_install -U ckanclient
Try to install python module ckanclient using the command above? (y/n) y
Prizms encapsulates all of the environment variables and PATH setup that is needed within
a single source-me.sh script dedicated to the user that needs it. The script is version-controlled
so we can manage the environment variables that everybody uses. The single source-me.sh should be the *only*
source-me.sh that is called from your ~/.bashrc. The following command is the only
source-me.sh that you need to run, and should be placed within your ~/.bashrc.
source /home/semantgeo/prizms/semantgeo/data/source/csv2rdf4lod-source-me-as-semantgeo.sh
grep: /home/semantgeo/.bashrc: No such file or directory
Add this command to your ~/.bashrc? [y/n]
Q: Changes to tomcat require it to restart. Restart it (requires sudo) ? [y/n] y
sudo /etc/init.d/tomcat6 stop
* Stopping Tomcat servlet engine tomcat6 [ OK ]
sudo /etc/init.d/tomcat6 start
* no JDK found - please set JAVA_HOME
[okay] easy_install already available at /usr/local/bin/easy_install
LODSPeaKr lives within the htdocs directory (/var/www),
while your semantgeo Prizms will maintain the model/views within
the version-controlled repository ([email protected]:ashbyb/semantgeo.git).
/var/www/lodspeakr is not set up yet. It can be installed with the command:
sudo bash -s base-url=http://purl.org/twc/semantgeo -s base-namespace=http://purl.org/twc/semantgeo -s sparql-endpoint=http://purl.org/twc/semantgeo/sparql chown=www-data:semantgeo < <(curl -sL http://lodspeakr.org/install)
Q: Would you like to install LODSPeaKr? [y/n] y
Cloning into lodspeakr...
remote: Counting objects: 5350, done.
remote: Compressing objects: 100% (2804/2804), done.
remote: Total 5350 (delta 2319), reused 5312 (delta 2286)
Receiving objects: 100% (5350/5350), 2.85 MiB | 1.11 MiB/s, done.
Resolving deltas: 100% (2319/2319), done.
Created new configuration file: settings.inc.php
WARNING: Copying root.htaccess as .htaccess in parent directory
WARNING: Automatically changing ownership of cache, meta, components and settings.inc.php to www-data:semantgeo
--------------------------------------------------------------------------------------------------------
You can now visit http://purl.org/twc/semantgeo/ to navigate through your data.
--------------------------------------------------------------------------------------------------------
We've finished setting up your development environment.
The next step is to set up the semantgeo's production environment,
which we can do by running this script again as user semantgeo
Q: Set up the production environment as the semantgeo user? [y/n] y
/home/semantgeo/opt/prizms
sudo su - semantgeo -c cd; mkdir -p opt; cd opt; git clone git://github.com/timrdf/prizms.git
No directory, logging in with HOME=/
fatal: could not create work tree dir 'prizms'.: Permission denied
No directory, logging in with HOME=/
-su: opt/prizms/bin/install.sh: not found
However, the following do not seem to be installed:
[TODO] curl -O --progress-bar http://archive.apache.org/dist/jena/binaries/apache-jena-2.10.0.zip from /home/iogds/opt
Q: May we try to install the dependencies listed above? (We'll need root for most of them) [y/n] y
[WARNING] Skipping apt-get git git-core b/c no sudo.
[WARNING] Skipping apt-get java openjdk-6-jre b/c no sudo.
[WARNING] Skipping apt-get awk gawk b/c no sudo.
[WARNING] Skipping apt-get curl curl b/c no sudo.
[WARNING] Skipping apt-get rapper raptor-utils b/c no sudo.
[WARNING] Skipping apt-get unzip unzip b/c no sudo.
[WARNING] Skipping apt-get screen screen b/c no sudo.
[WARNING] Skipping apt-get tidy tidy b/c no sudo.
[okay] serdi available at
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
However, the following do not seem to be installed:
[TODO] curl -O --progress-bar http://archive.apache.org/dist/jena/binaries/apache-jena-2.10.0.zip from /home/semantgeo/opt
Q: May we try to install the dependencies listed above? (We'll need root for most of them) [y/n] y
[okay] serdi available at /usr/local/bin/serdi
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
Could not find tdbloader on path. Try to install jena at /home/semantgeo/opt? (y/n): y
"May we add the environment variables"
to submit this one to the SADI registry as part of the prizms installer - so not to spam the registry.
follow https://github.com/jimmccusker/twc-healthdata/wiki/VM-Installation-Notes#wiki-ckan
include Jim's Datapub extension https://github.com/jimmccusker/ckanext-datapub
http://docs.ckan.org/en/ckan-1.8/install-from-package.html
https://github.com/okfn/ckan/wiki/How-to-Install-CKAN-2.0-from-source-on-Ubuntu-12.04-on-EC2
http://stackoverflow.com/questions/5485790/mod-wsgi-pylons-ckan-installation-not-working
The deployment docs are currently being updated 1, but they do cover
mounting CKAN at non-root URLs:
http://docs.ckan.org/en/latest/deployment.html#mounting-ckan-at-a-non-root-url
Although Nigel commented that some people have had problems with some
CKAN features after doing this.
Those docs are out official documentation for setting up CKAN with
Apache.
The CKAN package install only supports Ubuntu 12.04, 64-bit.
To install CKAN on Ubuntu 13.04, do a source install. See:
http://docs.ckan.org/en/latest/installing.html
https://lists.okfn.org/pipermail/ckan-dev/2013-December/006474.html
currently using 777
https://github.com/timrdf/prizms/blob/master/bin/install.sh#L2095
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.