richelbilderbeek / peregrine Goto Github PK

R package to work on the RuG Peregrine cluster

Shell 35.16% R 63.99% Makefile 0.80% Rez 0.06%

groningen peregrine cluster computer-cluster university

peregrine's Introduction

`peregrine`

Branch
`master`
`develop`

peregrine is:

A collection of notes on the Groningen Peregrine computer cluster
A collection of scripts for the Groningen Peregrine computer cluster
An R package for some Peregrine functions
Some presentations on the Groningen Peregrine computer cluster (see below)

Videos

How to run an R script on Peregrine download (.ogv) YouTube
How to install the pirouette R package on Peregrine download (.ogv) YouTube
How to run a pirouette example on Peregrine download (.ogv) YouTube

Presentations

Running (independent) jobs on the Peregrine computer cluster PDF ODP
Running jobs with dependencies on the Peregrine computer cluster PDF ODP

Example

The most important function is is_pff: 'Is Peregrine-friendly Filename/Folder'. If this function returns FALSE, one better not uses that path on the Peregrine cluster!

library(peregrine)
library(testthat)

expect_true(is_pff(get_pff_tempdir()))
expect_true(is_pff(get_pff_tempfile()))
expect_false(is_pff("/local/tmp/peregrine_unfriendly"))
expect_false(is_pff("/tmp/peregrine_unfriendly"))

One can use to_pff to convert a filename or folder to its Peregrine-friendly equivalent.

Measurements

Using scripts/test_is_pff.R.

Can create file?

filename	`can_create_file`
`/local/tmp.txt`	TRUE
`/local/tmp/tmp.txt`	TRUE
`/tmp.txt`	FALSE
`/tmp/tmp.txt`	TRUE
`/data/tmp.txt`	FALSE
`/data/p230198/tmp.txt`	TRUE
`/home/p230198/tmp.txt`	TRUE
`/home/tmp.txt`	FALSE

Can start BEAST2?

prefix	`can_start_beast`
`/local`	TRUE
`/local/tmp`	TRUE
`/tmp`	FALSE
`/`	FALSE
`/data`	FALSE
`/data/p230198`	FALSE
`/home`	FALSE
`/home/p230198`	FALSE

Can use BEAST2?

folder_name	`can_use_beast`
`/tmp	FALSE
`/local/tmp	TRUE
`/data/p230198	FALSE
`/home/p230198	TRUE

Can use working dir?

folder_name	`can_use_working_dir`
`/tmp`	FALSE
`/local/tmp`	FALSE
`/data/p230198`	FALSE
`/home/p230198`	TRUE

peregrine's People

Contributors

Stargazers

Watchers

Forkers

jorgeamaya theopannetier ramonssouza

peregrine's Issues

Add video on run_r_cmd

On the terminal:

sbatch run_r_cmd.sh 'print("hello")'

The log file:

p230198@peregrine:scripts cat run_r_cmd_9494322.log 
Rscript -e print("hello")
[1] "hello"


###############################################################################
Peregrine Cluster
Job 9494322 for user 'p230198'
Finished at: Tue Feb 11 00:49:25 CET 2020

Job details:
============

Name                : run_r_cmd
User                : p230198
Partition           : gelifes
Nodes               : pg-node215
Cores               : 1
State               : COMPLETED
Submit              : 2020-02-10T15:23:58
Start               : 2020-02-11T00:49:21
End                 : 2020-02-11T00:49:25
Reserved walltime   : 10-00:00:00
Used walltime       :    00:00:04
Used CPU time       :    00:00:01 (efficiency: 34.70%)
% User (Computation): 63.04%
% System (I/O)      : 36.89%
Mem reserved        : 10G/node
Max Mem used        : 0.00  (pg-node215)
Max Disk Write      : 0.00  (pg-node215)
Max Disk Read       : 0.00  (pg-node215)


Acknowledgements:
=================

Please see this page for information about acknowledging Peregrine in your publications:

https://wiki.hpc.rug.nl/peregrine/additional_information/scientific_output

################################################################################

Can create files?

p230198@peregrine:scripts Rscript test_is_pff.R 
[1] "Run on login node"
[1] "Testing prefix '/tmp/'"
[1] "File '/tmp//tmp.txt' exists: TRUE"
[1] "Testing prefix '/'"
[1] "File '//tmp.txt' exists: FALSE"
[1] "Testing prefix '/data/'"
[1] "File '/data//tmp.txt' exists: FALSE"
[1] "Testing prefix '/data/p230198'"
[1] "File '/data/p230198/tmp.txt' exists: TRUE"
[1] "Testing prefix '/tmp/'"
[1] "Testing prefix '/'"
[1] "Testing prefix '/data/'"
[1] "Testing prefix '/data/p230198'"
^C
Warning messages:
1: In file.rename(from = actual_log_filename, to = output_log_filename) :
  cannot rename file '/local/tmp/RtmptyTmOx/beast2_tmp_folder1bc1d488c084e/test_output_0.log' to '/tmp//out.log', reason 'Invalid cross-device link'
2: In file.rename(from = actual_trees_filename, to = to) :
  cannot rename file '/local/tmp/RtmptyTmOx/beast2_tmp_folder1bc1d488c084e/test_output_0.trees' to '/tmp//out.trees', reason 'Invalid cross-device link'
3: In file.rename(from = actual_log_filename, to = output_log_filename) :
  cannot rename file '/local/tmp/RtmptyTmOx/beast2_tmp_folder1bc1d4d3786c1/test_output_0.log' to '//out.log', reason 'Invalid cross-device link'
4: In file.rename(from = actual_trees_filename, to = to) :
  cannot rename file '/local/tmp/RtmptyTmOx/beast2_tmp_folder1bc1d4d3786c1/test_output_0.trees' to '//out.trees', reason 'Invalid cross-device link'
5: In file.rename(from = actual_log_filename, to = output_log_filename) :
  cannot rename file '/local/tmp/RtmptyTmOx/beast2_tmp_folder1bc1d7e39ff65/test_output_0.log' to '/data//out.log', reason 'Invalid cross-device link'
6: In file.rename(from = actual_trees_filename, to = to) :
  cannot rename file '/local/tmp/RtmptyTmOx/beast2_tmp_folder1bc1d7e39ff65/test_output_0.trees' to '/data//out.trees', reason 'Invalid cross-device link'
Execution halted

Cannot install R package rstan on Peregrine

trying URL 'http://cran.xl-mirror.nl/src/contrib/rstan_2.9.0-3.tar.gz'
Content type 'application/x-gzip' length 1099610 bytes (1.0 Mb)
opened URL
==================================================
downloaded 1.0 Mb

* installing *source* package ‘rstan’ ...
** package ‘rstan’ successfully unpacked and MD5 sums checked
** libs
g++ -I/software/software/R/3.1.2-goolfc-2.7.11-default/lib64/R/include -DNDEBUG -I"../inst/include" -I"../inst/include/boost_not_in_BH" -I"`"/software/software/R/3.1.2-goolfc-2.7.11-default/lib64/R/bin/Rscript" --vanilla -e "cat(system.file('include', 'src', package = 'StanHeaders'))"`" -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -I/software/software/OpenBLAS/0.2.13-gompic-2.7.11-LAPACK-3.5.0/include -I/software/software/ScaLAPACK/2.0.2-gompic-2.7.11-OpenBLAS-0.2.13-LAPACK-3.5.0/include -I/software/software/FFTW/3.3.4-gompic-2.7.11/include -I"/home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/Rcpp/include" -I"/home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/RcppEigen/include" -I"/home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/BH/include" -I"/home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/StanHeaders/include"   -fpic  -O2 -march=native  -c chains.cpp -o chains.o
g++ -I/software/software/R/3.1.2-goolfc-2.7.11-default/lib64/R/include -DNDEBUG -I"../inst/include" -I"../inst/include/boost_not_in_BH" -I"`"/software/software/R/3.1.2-goolfc-2.7.11-default/lib64/R/bin/Rscript" --vanilla -e "cat(system.file('include', 'src', package = 'StanHeaders'))"`" -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -I/software/software/OpenBLAS/0.2.13-gompic-2.7.11-LAPACK-3.5.0/include -I/software/software/ScaLAPACK/2.0.2-gompic-2.7.11-OpenBLAS-0.2.13-LAPACK-3.5.0/include -I/software/software/FFTW/3.3.4-gompic-2.7.11/include -I"/home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/Rcpp/include" -I"/home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/RcppEigen/include" -I"/home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/BH/include" -I"/home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/StanHeaders/include"   -fpic  -O2 -march=native  -c init.cpp -o init.o
g++ -I/software/software/R/3.1.2-goolfc-2.7.11-default/lib64/R/include -DNDEBUG -I"../inst/include" -I"../inst/include/boost_not_in_BH" -I"`"/software/software/R/3.1.2-goolfc-2.7.11-default/lib64/R/bin/Rscript" --vanilla -e "cat(system.file('include', 'src', package = 'StanHeaders'))"`" -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -I/software/software/OpenBLAS/0.2.13-gompic-2.7.11-LAPACK-3.5.0/include -I/software/software/ScaLAPACK/2.0.2-gompic-2.7.11-OpenBLAS-0.2.13-LAPACK-3.5.0/include -I/software/software/FFTW/3.3.4-gompic-2.7.11/include -I"/home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/Rcpp/include" -I"/home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/RcppEigen/include" -I"/home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/BH/include" -I"/home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/StanHeaders/include"   -fpic  -O2 -march=native  -c lang__ast_def.cpp -o lang__ast_def.o
g++ -I/software/software/R/3.1.2-goolfc-2.7.11-default/lib64/R/include -DNDEBUG -I"../inst/include" -I"../inst/include/boost_not_in_BH" -I"`"/software/software/R/3.1.2-goolfc-2.7.11-default/lib64/R/bin/Rscript" --vanilla -e "cat(system.file('include', 'src', package = 'StanHeaders'))"`" -DBOOST_RESULT_OF_USE_TR1 -DBOOST_NO_DECLTYPE -DBOOST_DISABLE_ASSERTS -I/software/software/OpenBLAS/0.2.13-gompic-2.7.11-LAPACK-3.5.0/include -I/software/software/ScaLAPACK/2.0.2-gompic-2.7.11-OpenBLAS-0.2.13-LAPACK-3.5.0/include -I/software/software/FFTW/3.3.4-gompic-2.7.11/include -I"/home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/Rcpp/include" -I"/home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/RcppEigen/include" -I"/home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/BH/include" -I"/home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/StanHeaders/include"   -fpic  -O2 -march=native  -c lang__grammars__bare_type_grammar_inst.cpp -o lang__grammars__bare_type_grammar_inst.o
In file included from /home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/BH/include/boost/iostreams/detail/is_dereferenceable.hpp:12:0,
                 from /home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/BH/include/boost/iostreams/detail/resolve.hpp:26,
                 from /home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/BH/include/boost/iostreams/detail/push.hpp:24,
                 from /home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/BH/include/boost/iostreams/detail/streambuf/indirect_streambuf.hpp:31,
                 from /home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/BH/include/boost/iostreams/stream_buffer.hpp:22,
                 from /home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/BH/include/boost/iostreams/stream.hpp:21,
                 from /home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/BH/include/boost/spirit/home/qi/stream/detail/iterator_source.hpp:14,
                 from /home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/BH/include/boost/spirit/home/qi/stream/stream.hpp:16,
                 from /home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/BH/include/boost/spirit/home/qi/stream.hpp:15,
                 from /home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/BH/include/boost/spirit/home/qi.hpp:30,
                 from /home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/BH/include/boost/spirit/include/qi.hpp:16,
                 from /home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/StanHeaders/include/src/stan/lang/grammars/bare_type_grammar_def.hpp:4,
                 from /home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/StanHeaders/include/src/stan/lang/grammars/bare_type_grammar_inst.cpp:1,
                 from lang__grammars__bare_type_grammar_inst.cpp:18:
/home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/BH/include/boost/type_traits/detail/bool_trait_def.hpp:18:79: note: #pragma message: NOTE: Use of this header (bool_trait_def.hpp) is deprecated
 # pragma message("NOTE: Use of this header (bool_trait_def.hpp) is deprecated")

[...]                                                                               ^
                                                                                    ^
g++: internal compiler error: Killed (program cc1plus)
0x409a70 execute
    ../../gcc/gcc.c:2823
0x409da4 do_spec_1
    ../../gcc/gcc.c:4615
0x40c725 process_brace_body
    ../../gcc/gcc.c:5872
0x40c725 handle_braces
    ../../gcc/gcc.c:5786
0x40acc7 do_spec_1
    ../../gcc/gcc.c:5269
0x40c725 process_brace_body
    ../../gcc/gcc.c:5872
0x40c725 handle_braces
    ../../gcc/gcc.c:5786
0x40acc7 do_spec_1
    ../../gcc/gcc.c:5269
0x40a9bf do_spec_1
    ../../gcc/gcc.c:5374
0x40c725 process_brace_body
    ../../gcc/gcc.c:5872
0x40c725 handle_braces
    ../../gcc/gcc.c:5786
0x40acc7 do_spec_1
    ../../gcc/gcc.c:5269
0x40c725 process_brace_body
    ../../gcc/gcc.c:5872
0x40c725 handle_braces
    ../../gcc/gcc.c:5786
0x40acc7 do_spec_1
    ../../gcc/gcc.c:5269
0x40c725 process_brace_body
    ../../gcc/gcc.c:5872
0x40c725 handle_braces
    ../../gcc/gcc.c:5786
0x40acc7 do_spec_1
    ../../gcc/gcc.c:5269
0x40c725 process_brace_body
    ../../gcc/gcc.c:5872
0x40c725 handle_braces
    ../../gcc/gcc.c:5786
Please submit a full bug report,
with preprocessed source if appropriate.
Please include the complete backtrace with any bug report.
See <http://gcc.gnu.org/bugs.html> for instructions.
make: *** [lang__grammars__statement_grammar_inst.o] Error 4
ERROR: compilation failed for package ‘rstan’
* removing ‘/home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/rstan’
* restoring previous ‘/home/p230198/R/x86_64-unknown-linux-gnu-library/3.1/rstan’

The downloaded source packages are in
    ‘/tmp/RtmpxeZByE/downloaded_packages’
Warning message:
In install.packages("rstan") :
  installation of package ‘rstan’ had non-zero exit status

Error in if (capabilities("libcurl")) { : argument is of length zero

> devtools::install_github("richelbilderbeek/Cer2016")
Downloading GitHub repo richelbilderbeek/Cer2016@master
from URL https://api.github.com/repos/richelbilderbeek/Cer2016/zipball/master
Installing Cer2016
Error in if (capabilities("libcurl")) { : argument is of length zero

Is '/data/...' a Peregrine unfriendly filename?

Bug report @thijsjanzen

Ik heb pirouette geupdate op het cluster (en richelbilderbeek/peregrine), opnieuw BEAST2 geinstalleerd.... geen soelaas nog helaas!

Error in beastier::run_beast2(input_filename = beast2_input_filename, :

BEAST2 .log file not created at '/data/p251362/nodeSub/no_extinct/shared/0/temp.log'

Maybe no permission to write at that location?

Can use /scratch folder?

From @thijsjanzen:

Weet jij of BEAST op peregrine ook op /scratch kan runnen? Op /home heb ik maar 20GB ruimte, en ik kan daardoor maar een gelimiteerd aantal BEAST simulaties tegelijk runnen. Had er gisteren 90 tegelijk gestart, en dat wil niet ;)

Can pirouette still run on Peregrine?

There are too many Issues that hint a no.

Use BEAST2 smart

From @richelbilderbeek on May 17, 2016 12:26

That is: use more cores

$ module load beagle-lib Beast; beast -beagle_info

Checking out /home/p230198/.beast/2.4/BEAST/lib
Loaded URL file:/home/p230198/.beast/2.4/BEAST/lib/beast.jar
jardir = /software/software/Beast/2.4.0-foss-2016a/lib/launcher.jar
Loading package BEAST v2.4.0
Loading package BEAST v2.4.0

                        BEAST v2.4.0, 2002-2016
             Bayesian Evolutionary Analysis Sampling Trees
                       Designed and developed by
 Remco Bouckaert, Alexei J. Drummond, Andrew Rambaut & Marc A. Suchard

                     Department of Computer Science
                         University of Auckland
                        [email protected]
                        [email protected]

                   Institute of Evolutionary Biology
                        University of Edinburgh
                           [email protected]

                    David Geffen School of Medicine
                 University of California, Los Angeles
                           [email protected]

                      Downloads, Help & Resources:
                           http://beast2.org/

  Source code distributed under the GNU Lesser General Public License:
                   http://github.com/CompEvol/beast2

                           BEAST developers:
   Alex Alekseyenko, Trevor Bedford, Erik Bloomquist, Joseph Heled, 
 Sebastian Hoehna, Denise Kuehnert, Philippe Lemey, Wai Lok Sibon Li, 
Gerton Lunter, Sidney Markowitz, Vladimir Minin, Michael Defoin Platel, 
                 Oliver Pybus, Chieh-Hsi Wu, Walter Xie

                               Thanks to:
          Roald Forsberg, Beth Shapiro and Korbinian Strimmer

BEAGLE resources available:
0 : CPU
    Flags: PRECISION_SINGLE PRECISION_DOUBLE COMPUTATION_SYNCH EIGEN_REAL EIGEN_COMPLEX SCALING_MANUAL SCALING_AUTO SCALING_ALWAYS SCALERS_RAW SCALERS_LOG VECTOR_SSE VECTOR_NONE THREADING_NONE PROCESSOR_CPU FRAMEWORK_CPU

Copied from original issue: richelbilderbeek/beast2_on_peregrine#1

Test gelifes nodes

Dear Michael and colleagues,

The Peregrine nodes that have been bought by Gelifes have been configured, and are available. I did some first tests myself, and everything seems to be working fine.

This means that I would like to invite the Gelifes members to test the nodes for themselves. Joke suggested that you, Michael, and your group members, are available and probably would be willing to try the new nodes. So please go ahead.

Some details about the setup. The nodes have been put in a separate partition, called 'gelifes'. They share the same storage and software as all the other nodes.
Since these nodes have been paid by Gelifes, users from Gelifes are currently the only ones who can access them. To separate the usage between the Gelifes nodes and the rest of the cluster a separate 'gelifes' account has been made in the scheduler, the other account being 'users'.

Only users which have an account in 'gelifes' are able to submit to the gelifes partition. These people also have an account in 'users'. Job submitted in the gelifes partition will be accounted for in the gelifes account, and jobs in the other partitions will be accounted for in the users account.

This sounds a bit complicated, but the main thing to take home is:

submit to the partition 'gelifes' to get to the new nodes.
usage (and priorities) of the gelifes nodes is accounted separately from the rest of the cluster.

The node themselves are 64 core AMD nodes with 512GB of memory. These should be suitable for most of the workloads you have. There is also quite a lot (16 TB) of temporary local scratch space per node, available for jobs through the use $TMPDIR.

I added all people from Gelifes in the cc: to the gelifes account. If anyone else needs to be added, please let me know. Unfortunately adding people is a manual process.

I would suggest to use [email protected] for any questions or problems you may have when using the systems. This makes sure that my colleagues will take over when I am on holiday, and that not a whole group of other people gets these e-mails.
For any questions related to the contents of this email a reply, is of course fine.

Good luck with the extra capacity! I expect that some of you can really benefit from it.

Kind regards,

Fokke Dijkstra

Peregrine terminal freezes after cp

Dear CIT,

After copying some bigger files using cp, my terminal to Peregrine does not allow typing anymore.

Perhaps it has to do that I use ssh [email protected] (thus lacking the -X for the X server)?

I have attached two screenshots. The terminal does respond, except that I cannot type anymore, as if I have lost the connection.

Thanks for your help, Richel Bilderbeek, p230198