Variable solve times about circuitscape.jl HOT 27 CLOSED

Could you upload the files somewhere for us to download and reproduce? Are you running it on your laoptop? How much memory?

from circuitscape.jl.

Yes, could you upload the files to a drop box or Google Drive location, enable sharung and paste the link here please? It would help me debug.

from circuitscape.jl.

Here is a link to some example files:
https://drive.google.com/open?id=1OtiSZsubtN47ydfOKZf0mAcuM1eEtuEf

I've been running these on a Desktop (i7 3.6 GHz; 24 GB RAM).

There is also a .txt file with the WARNING message that is printed when I use CS.jl.

from circuitscape.jl.

Curious to know if you all had any insights regarding this issue (or if you were able to recreate the issue). Happy to provide more examples or more detail concerning my implementation, if it will be of help.

from circuitscape.jl.

@wpeterman I just ran the julia_quickrun example multiple times straight from Julia and didn't see any variability in solve time. But I ran this on my Mac. Let me run this on a Windows laptop and get back to you.

from circuitscape.jl.

julia_longrun should be the problem surface.

from circuitscape.jl.

Oh my bad, I think I can see it now. I'll get back to you ASAP

from circuitscape.jl.

Just as a note, the warning has been fixed in #130

from circuitscape.jl.

@ranjanan
Has your rounding solution been implemented? If so, what needs to be done to update the package and/or dependencies?
Thanks for your time and effort on this!

from circuitscape.jl.

Hi @wpeterman, I implemented it on a branch called round over here. You can access it by doing:

Pkg.checkout("Circuitscape", "round")

and then restarting Julia.

Could you confirm that this works?

Thank you.

from circuitscape.jl.

Thanks, @ranjanan
However, I get the following error:

INFO: Checking out Circuitscape round...
Error: Error happens in Julia.
GitError(Code:ERROR, Class:Merge, There is no tracking information for the current branch.)
Stacktrace:
[1] (::Base.LibGit2.##117#125{Base.LibGit2.GitRepo})(::Base.LibGit2.GitReference) at .\libgit2\libgit2.jl:709
[2] with(::Base.LibGit2.##117#125{Base.LibGit2.GitRepo}, ::Base.LibGit2.GitReference) at .\libgit2\types.jl:608
[3] #merge!#109(::String, ::String, ::Bool, ::Base.LibGit2.MergeOptions, ::Base.LibGit2.CheckoutOptions, ::Function, ::Base.LibGit2.GitRepo) at .\libgit2\libgit2.jl:706
[4] (::Base.#kw##merge!)(::Array{Any,1}, ::Base.#merge!, ::Base.LibGit2.GitRepo) at .<missing>:0
[5] (::Base.Pkg.Entry.##16#18{String,String,Bool,Bool})(::Base.LibGit2.GitRepo) at .\pkg\entry.jl:230
[6] transact(::Base.Pkg.Entry.##16#18{String,String,Bool,Bool}, ::Base.LibGit2.GitRepo) at .\libgit2\libgit2.jl:882
[7] with(::Base.Pkg.Entry.##15#17{String,String,Bool,Bool}, ::Base.LibGit2.GitRepo) at .\libgit2\types.jl:608
[8] checkout(::String, ::String, ::Bool

from circuitscape.jl.

Okay, can you try:

Pkg.checkout("Circuitscape") 
Pkg.checkout("Circuitscape", "round") 

in that order?

from circuitscape.jl.

Almost! The 'round' branch efficiently solved several thousand different scenarios with the standard solver as well as cholmod. I did encounter one problematic surface (files here)

from circuitscape.jl.

Hi @ranjanan
Just checking in to see if you had any insights about the problematic surface I encountered with the 'round' branch. Thanks!

from circuitscape.jl.

@wpeterman Sorry about the late response time. Been moving cities and it's been hard getting some time to work. #134 should have fixed all these issues. Could you please check how this runs on all your problems? Thanks.

from circuitscape.jl.

You should first do a Pkg.checkout("Circuitscape", "master") first if you're already on the round branch

from circuitscape.jl.

@ranjanan
No worries about the delay. I generated several scenarios that were slow to solve. Some examples are here.

Pkg.status("Circuitscape") – Circuitscape 5.2.1+ round

from circuitscape.jl.

@wpeterman It looks like you're on the round branch. Could you get on the master branch and check if they solve faster?

from circuitscape.jl.

Also, thank you so much for the nice collection of problems! Do you mind if I include any of them as tests on the repo?

from circuitscape.jl.

Also I believe there should be another column in your points file on the link.

from circuitscape.jl.

I just added a leading column 1, 2, 3, 4, 5 in your points file. I hope that's okay. On the master branch these seem to solve fairly quickly. Could you please verify, @wpeterman? Thank you!

from circuitscape.jl.

Sorry for my confusion. I didn't realize that the fix had been merged with the master. Several hundred scenarios seem to all solve fine now, thanks! Feel free to use any of my 'problems' as tests.

from circuitscape.jl.

@wpeterman Thank you! Could I ask you for one more favour please? Could you share the results of your benchmarking, especially if you've done it on a large problem? I would request your permission to quote these results on a presentation I'm giving about the package.

from circuitscape.jl.

Happy to share any benchmarking results. Most of what I've done has been pretty quick and minimally replicated (although there is typically little variation in runtimes). Are there specific comparisons or combinations you're looking for? Otherwise, feel free to use anything already summarized here. The most recent test (final entry in the linked file) seems to suggest that CS.jl loses its speed advantage as the number of sample points on the landscape increases.

from circuitscape.jl.

It would be good to have the example where the speed advantage of CS.jl is the smallest, so that we can try figure out what is happening under the hood and if there are other opportunities.

from circuitscape.jl.

Here are files for testing 10, 50, & 100 points on a 500x500 grid, including a single run Benchmark times comparing CIRCUITSCAPE, CS.jl, and commuteDistance from the R package gdistance.

from circuitscape.jl.

@wpeterman I just examined the benchmarks in your link, and I found that in all your problems you don't write current or voltage maps. In Circuitscape, this triggers a shortcut calculation which significantly reduces the number of pairs to solve and hence the amount of work done. And since the problem size is small, each solve itself is very small (sub 1 second). Since there's relatively little work, the Julia version seem to be performing just a little bit better than the Python version.

Now having said that, the CHOLMOD solver mode was introduced in the Julia version to solve these kinds of problems very quickly because it batches and solves pairs together. And upon inspection, I found that there was a bug in the code that made the CHOLMOD solver path do a lot more work than it should in the shortcut mode. So I wrote #138 to fix it. Just to give a sense of how much extra work it is:

This is the timing without patch #138

julia> compute("100s_500g.ini")                                                                                                                               
[2018-07-22T06:10:34 | info | root]: Precision used: Double                                                                                                   
[2018-07-22T06:10:34 | info | root]: Reading maps                                                                                                             
[2018-07-22T06:10:34 | info | root]: Resistance/Conductance map has 250000 nodes                                                                              
[2018-07-22T06:10:34 | info | root]: Solver used: CHOLMOD                                                                                                     
[2018-07-22T06:10:34 | info | root]: Graph has 250000 nodes, 100 focal points and 1 connected components                                                      
[2018-07-22T06:10:34 | info | root]: Total number of pair solves = 4950                                                                                       
[2018-07-22T06:10:34 | info | root]: Triggering resistance calculation shortcut                                                                               
[2018-07-22T06:10:34 | info | root]: Total number of pair solves has been reduced to 99                                                                       
[2018-07-22T06:10:36 | info | root]: Time taken to construct cholesky factor = 1.635645162                                                                    
[2018-07-22T06:10:36 | info | root]: Time taken to construct local nodemap = 0.031588787 seconds                                                              
[2018-07-22T06:10:36 | info | root]: Solving points 1 to 1000                                                                                                 
[2018-07-22T06:11:28 | info | root]: Solving points 1001 to 2000                                                                                              
[2018-07-22T06:12:20 | info | root]: Solving points 2001 to 3000                                                                                              
[2018-07-22T06:13:12 | info | root]: Solving points 3001 to 4000                                                                                              
[2018-07-22T06:14:04 | info | root]: Solving points 4001 to 4950                                                                                              
[2018-07-22T06:14:53 | info | root]: Time taken to complete job = 259.682150018         

And this is the timing with patch #138:

julia> compute("100s_500g.ini")                                                                                                                               
[2018-07-22T06:18:17 | info | root]: Precision used: Double                                                                                                   
[2018-07-22T06:18:17 | info | root]: Reading maps                                                                                                             
[2018-07-22T06:18:18 | info | root]: Resistance/Conductance map has 250000 nodes                                                                              
[2018-07-22T06:18:18 | info | root]: Solver used: CHOLMOD                                                                                                     
[2018-07-22T06:18:18 | info | root]: Graph has 250000 nodes, 100 focal points and 1 connected components                                                      
[2018-07-22T06:18:18 | info | root]: Total number of pair solves = 4950                                                                                       
[2018-07-22T06:18:18 | info | root]: Triggering resistance calculation shortcut                                                                               
[2018-07-22T06:18:18 | info | root]: Total number of pair solves has been reduced to 99                                                                       
[2018-07-22T06:18:19 | info | root]: Time taken to construct cholesky factor = 1.533338919                                                                    
[2018-07-22T06:18:19 | info | root]: Time taken to construct local nodemap = 0.036663354 seconds                                                              
[2018-07-22T06:18:19 | info | root]: Solving points 1 to 99                                                                                                   
[2018-07-22T06:18:25 | info | root]: Time taken to complete job = 7.281965952     

I hope this explanation helps. I shall merge #138 soon, so you should see an improvement when you benchmark the CHOLMOD solver mode. Just do a Pkg.checkout("Circuitscape", "master") and you should get the patch.

from circuitscape.jl.