Comments (2)
I inspect makeClusterFunctionsTorque.R and I cannot understand how you handle the template, it seems you read the template file once template = cfReadBrewTemplate(template, "##"), and then drop it as it is not returned by the function.
You are right, I read the template file once. But the variable will not be garbage collected, because it lives in the environment of the returned functions. Here is a small example to demonstrate this mechanism (called closure):
pow = function(p) {
force(p)
function(x) x^p
}
square = pow(2)
square(1:10)
root = pow(0.5)
root(1:10)
ls.str(environment(square))
After parsing the template once during construction with makeClusterFunctionsTorque
, the template is stored as string in the environment of the functions:
ee = environment(reg$cluster.functions$submitJob)
ls.str(ee)
ee$template
If passed to cfBrewTemplate
the variable template
will get automatically resolved to variable previously defined in respective scope/environment.
Regarding the hooks:
Sorry, I was pointing you in the wrong direction in the other issue. To set the hooks, put something like this in your config file:
# vanilla cluster functions implementations without hooks
cluster.functions = makeClusterFunctionsTorque(template = "~/torque.tmpl")
# hooks is initialized as an empty list, thus we can just set one here
cluster.functions$hooks$pre.submit = function(...) Sys.sleep(2)
from batchtools.
I salute you !
I have overlooked the closure part until now.
thx
from batchtools.
Related Issues (20)
- scoping errors depending on objects in global environment
- Job requeue on Slurm - 'no such file or directory' & workaround HOT 1
- ability to suppress warning on failure to source `source` when calling makeRegistry
- All jobs expired with SGE cluster, job script permission denied HOT 2
- makeClusterFunctions store.job.collection flag documentation
- Yet another `BatchtoolsExpiration: Future ('<none>') expired` error
- `$conditions` blows up rds `results/*.rds` files HOT 2
- Publish 0.9.16 on CRAN HOT 3
- Edit the job template without clearing the entire registry
- Make 'options(error = function(e) traceback(2))` optional in testJob
- Error in (function () : unused argument (x = 3) HOT 3
- Question: How are template files interpretted? HOT 1
- Effectively use batchtools for Targets-based workflow in SLURM HOT 4
- squeue cmd not found when running R targets on singularity docker image HOT 1
- calling `batchMap` + `submitJob` in a loop triggers "Registry must be empty" error HOT 1
- submitJob - can't find qsub function HOT 1
- Slurm resource named `clusters` in docs, and `cluster` in `getClusters`
- Creating jobs without package dependencies installed on head node
- Old version of Slurm, "invalid job state specified: SI" HOT 1
- Error : Log file '/../../../...log' for job with id 1 not available
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from batchtools.