ctf consensus about xmipp HOT 5 CLOSED

I agree on that. When this protocol was created by merging CTFselection and CTFdiscrepancy, it was coded taking into account the streaming processing and, then, following the usual streaming procedure, but this is very inefficient for the non streaming workflows in this case...

We should to redo it, as you point out. Perhaps detecting two modes (streaming and static) and working accordingly is the solution.

@delarosatrevin also pointed this out some time ago, but we had no time to fix it. I will put it to the task list with a high priority.

Thanks!

from xmipp.

Hi @dmaluenda,

I do not think two different implementations of this protocol are needed. Even if we are in streaming there is no need to create thousand of jobs when a single one can do the work.

In summary we only need to create a single job, process all available CTFs and then relaunch the job, no need to create a job per CTF.

from xmipp.

Maybe, but following what you suggest, we will repeat some work again and again in streaming processing (if I correctly understand). I mean, when the 100th CTF pair comes in, we will "restart" the whole work instead of just comparing the 100th CTFs remaining the first 99 as they are.

However, if repeating the work in this way is faster than how it's currently done, it's good for me! If not, just detecting at the beginning in which mode (streaming/static) we are is the solution.

from xmipp.

Hi @dmaluenda ,

In C -but unfortunately not in python-, there is something called multiquery that allows you to query the database with several constrains at the same time (for ejemple id greater than certain number and defocus between some values). So one possibility is to add these multiqueries in he binding.

Another possibility is two make two queries, the first one will give you all entries with id greater than some value while the next one focus in the parameter you want to filter.

hope this helps

from xmipp.

This PR-442 close the issue. Optimized Ctf consensus

from xmipp.