drat 'go' command stalls at reduce prompt about drat HOT 11 CLOSED

Hmm. The go function should wait for all mappers to finish before firing off the reducer. But, the reducer still checks for running mappers. In this case, it seems the go didn't find a mapper, but the reduce function did. My guess is that one unluckily fired off in the short time between the two.

Regardless, we should give the command to rerun later in the confirmation message. Will send a PR tomorrow.

from drat.

We could also only wait for 10 or so seconds for user input if the reduce function finds running mappers, then simply trying to run reduce again. That way, asleep go users don't get an unpleasant surprise in the morning and reduce users can just say no then try again later.

from drat.

I'm wondering why it stalls - I thought the point of go was to be automated - I think it dynamically checks for the mappers to be done and then fires the reducer, right? Where does the prompt come in? I thought it comes in if you don't use "go" and if you ran each cycle (crawl, index, map) before running reduce?

from drat.

Correct, go waits for mappers to finish, then calls the reduce function. The reduce function checks for running mappers, then prompts the user for confirmation if there are still mappers running.

This double-check is useful for when a user doesn't want to use go, and instead call reduce manually. For automation, go attempts to prevent this check from finding any mappers.

In this issue, go found no mappers, but reduce did. I don't know why, except for maybe being really unlucky. This should never happen, since they both use the same method to check for currently running mappers.

Unless, does this happen every time, Lewis?

from drat.

every time Tyler. It seems tobe built in.

On Thu, Jul 24, 2014 at 8:57 PM, Tyler Palsulich [email protected]
wrote:

Correct, go waits for mappers to finish, then calls the reduce function.
The reduce function checks for running mappers, then prompts the user for
confirmation if there are still mappers running.

This double-check is useful for when a user doesn't want to use go, and
instead call reduce manually. For automation, go attempts to prevent this
check from finding any mappers.

In this issue, go found no mappers, but reduce did. I don't know why,
except for maybe being really unlucky. This should never happen, since they
both use the same method to check for currently running mappers.

Unless, does this happen every time, Lewis?

—
Reply to this email directly or view it on GitHub
chrismattmann/drat#17 (comment).

Lewis

from drat.

BTW another observation of running on Usergrid codebase was that some tasks too around 18-20 mins whereas others took literally a second or two. Please see screenshot of OPSUI. I am investigating further folks.

from drat.

Lewis that's totally expected (some tasks taking 20 mins, versus seconds). It has to do with the type of code it's looking at - usually Javascript or other MIME types, etc., take WAY longer than e.g., Java, for instance to check. I think this is more a functionality of RAT than anything else, but it could be that there is some difficulty RAT has in finding licenses in particular programming languages. But note, this is a KNOWN issue, (that Tika and DRAT helped to uncover, check out the DRAT docs for more info via the XNET presentation).

from drat.

Great.
This is something new I am learing and I am extremely happy that you've now
put DRAT into ASLv2.0...
I am now a contributor :)

On Thu, Jul 24, 2014 at 10:42 PM, Chris Mattmann [email protected]
wrote:

Lewis that's totally expected (some tasks taking 20 mins, versus seconds).
It has to do with the type of code it's looking at - usually Javascript or
other MIME types, etc., take WAY longer than e.g., Java, for instance to
check. I think this is more a functionality of RAT than anything else, but
it could be that there is some difficulty RAT has in finding licenses in
particular programming languages. But note, this is a KNOWN issue, (that
Tika and DRAT helped to uncover, check out the DRAT docs for more info via
the XNET presentation).

—
Reply to this email directly or view it on GitHub
chrismattmann/drat#17 (comment).

Lewis

from drat.

you are awesome!

Chris Mattmann
[email protected]

-----Original Message-----
From: Lewis John McGibbney [email protected]
Reply-To: chrismattmann/drat
<reply+i-38691321-173039f6446ef9f81838cea84193de75f255e45a-395887@reply.git
hub.com>
Date: Friday, July 25, 2014 12:08 AM
To: chrismattmann/drat [email protected]
Cc: Chris Mattmann [email protected]
Subject: Re: [drat] drat 'go' command stalls at reduce prompt (#17)

Great.

This is something new I am learing and I am extremely happy that you've
now

put DRAT into ASLv2.0...

I am now a contributor :)

On Thu, Jul 24, 2014 at 10:42 PM, Chris Mattmann
[email protected]

wrote:

Lewis that's totally expected (some tasks taking 20 mins, versus
seconds).

It has to do with the type of code it's looking at - usually Javascript
or

other MIME types, etc., take WAY longer than e.g., Java, for instance to

check. I think this is more a functionality of RAT than anything else,
but

it could be that there is some difficulty RAT has in finding licenses in

particular programming languages. But note, this is a KNOWN issue, (that

Tika and DRAT helped to uncover, check out the DRAT docs for more info
via

the XNET presentation).

�

Reply to this email directly or view it on GitHub

chrismattmann/drat#17 (comment).

Lewis

�
Reply to this email directly or view it on GitHub
chrismattmann/drat#17 (comment).

from drat.

WOW I totally missed all of this conversation and also @tpalsulich patch.
The ...................... counting update for map is not only familar but also very professional.
Thanks for this functionality folks. MUCH more precise.

from drat.

BOOM XNET PRESSIE
https://www.youtube.com/watch?v=9w3fpnNWdIE

from drat.