I have a Subversion repository with multiple projects in different locations on the same repository. I can't change this. To get opengrok to index all the projects, I've created a directory in the repository which just has externals which point to the locations of each of the projects:

http://server/repos/code_scratchpad/build
[svn:externals]
scripty http://server/repos/tools/script/trunk
main-project http://server/repos/dev/trunk
other-project http://server/repos/tools/other/trunk

Where "server" is the same server in all instances.

Hence, when I checkout "http://server/repos/code_scratchpad/build" into the "src" directory of an opengrok server, I get subfolders for every one of our projects with their trunk code in it.

Indexing works fine for this. However the svn history does not. You get errors like this:

2013-04-19 13:28:04.397+0000 INFO t14 SubversionHistoryParser$Handler.endElement: Skipping file outside repository: /dev/trunk/src/.../Place.java

The reason for this appears to be that the SubversionRepository gets the repository path (reposPath) by doing a svn info on the base directory and removing the root from it. Hence, above we'd get:

url: http://server/repos/code_scratchpad/build
root: http://server/repos
reposPath: /code_scatchpad/build

However, in my case, this isn't the reposPath - that should be "/".

My suggestion is to add a configuration option which enables the user to specify the repository root rather than guessing it.

Any alternative suggestions? Any recommendations for how this should be presented given that we have to support other VCSs which don't do this?

I'm happy to fork and code, just wanted to get some design guidance before I hack something together.

It would be useful to ignore paths with -i, not just filenames.

For example, on a site like BXR.SU, one may find such set of options useful: -i "*/OpenBSD*/gnu/*" -i "*/OpenBSD*/regress/gnu/*" -i "*/DragonFly/contrib/*" -i "*/DragonFly/gnu/*" etc.

I.e. you might want to ignore certain paths only within certain projects, and not otherwise.

When Lucene was upgraded to 3.5, the -m option was made a no-op by commenting out the following line in IndexDatabase:

//writer.setMaxFieldLength(RuntimeEnvironment.getInstance().getIndexWordLimit());

It was presumably commented out because the setMaxFieldLength() method was deprecated in Lucene 3.5, and subsequently removed in Lucene 4.

The new way to limit the number of tokens is LimitTokenCountAnalyzer.

When I cp -p /grok/grok/dist/source.war /var/tomcat/webapps/ROOT.war, and request any documents within the next couple of seconds after deploying in such a way, I get empty pages, with a 404 Not Found status code. OpenBSD 5.2, tomcat-7.0.29, {OpenGrok 0.11-rc2 from an old trunk.

From my nginx cache:

KEY: http://localhost:8080/s?n=25&start=50&sort=relevancy&q=test&project=DragonFly%2CFreeBSD%2CNetBSD%2COpenBSD
HTTP/1.1 404 Not Found
Server: Apache-Coyote/1.1
Content-Length: 0
Date: Tue, 02 Apr 2013 19:00:14 GMT
Connection: close

Why is this happening, and how do I avoid it?

status CLOSED severity minor in component webapp for ---
Reported in version unspecified on platform ANY/Generic
Assigned to: Trond Norbye

On 2008-04-07 04:54:58 +0000, Lubos Kosco wrote:

Sending and sharing links to code between developers is important.
At the moment (0.6 version) the search will be broken and asking for project or searching in totally different project when sharing link e.g.
http://src.opensolaris.org/source/xref/opengrok/trunk/src/org/opensolaris/opengrok/index/Indexer.java
and then doing a search from the form.
(either cookies must point to other project, or it must be a new browser session to see the error / odd behavior).

Thanks for fixing this.

On 2008-04-07 05:03:14 +0000, Trond Norbye wrote:

The project cookie was added as a part of the project prototype, and to avoid modifying all of the different xref generators we just used a cookie. We should update the xref generation to add the project id to the link, and just remove the project cookie.

On 2008-04-07 05:07:40 +0000, Trond Norbye wrote:

Fixed in 271:fd2ea8c62397. Please report back if you still have problems.

OpenGrok has the following documentation in regards to symlinks:

-N /path/to/symlink
        Allow this symlink to be followed. Option may be repeated.

However, it seems like it has no effect in master anymore -- links in source are already followed by default.

The gzip and bzip2 analyzers have code to produce xref if the uncompressed file is xreffable. That code doesn't work, and the xref is never produced.

To reproduce the problem:

1. Create a text file called plain.txt containing one line: "This is a test"
2. gzip the file
3. Run the indexer on the directory that contains the file
4. Search for "test" in the webapp

You'll see that the webapp lists plain.txt.gz in the search result. However, when you click on the result, you'll just get an empty xref page with a download link.

When starting Catalina run, I get the following two problems:

WARNING: ClearCaseRepository not working (missing binaries?): SNAPSHOT-VIEW-DIR

May 02, 2013 3:11:36 PM org.opensolaris.opengrok.web.WebappListener contextInitialized
SEVERE: OpenGrok: Failed to start configuration listener thread

Any idea, Anyone?

Thanks,
Avi

The string and/or, when appearing in plain text files, and in comments in C and other source code (e.g. in the licence text), is always assumed to be a path, and search links are given. This is wrong. The string is very unlikely to ever be a path.

The History/Annotate view of a git repository displays full SHA1 digests. The readability suffers as a result of that. Shorter albeit unique version can ba used as mentioned on http://git-scm.com/book/ch6-1.html :"Git can figure out a short, unique abbreviation for your SHA-1 values. If you pass --abbrev-commit to the git log command, the output will use shorter values but keep them unique; it defaults to using seven characters but makes them longer if necessary to keep the SHA-1 unambiguous"

In the source trees of *BSD projects, some binary files are sometimes present (e.g. font, firmware and architecture-specific files), and are often encoded with uuencode.

As it stands, this results in a lot of rubbish being indexed, and those files, once converted to xref/, contain a lot of links for random paths, as / is one of the characters of the uuencode encoding.

http://bxr.su/search?q=begin&path=fnt+OR+uu+OR+uue

For bxr.su, I wrote a simple parser, that only indexes the top and the bottom portions of such files, skipping all the uuencode content.

as per pull request #25
@VikasRK wrote:

@ajayk : https://gist.github.com/schmmd/1277336 provides ctags support for Scala. Just need to copy the regexes into ~/.ctags file, it works with vim/ctags.

But with Opengrok it won't. Do you know how to fix it, now that ctags supports scala??

• we support local ctags file - @vladak do you have some pointer to a howto? did we document this?

I'm not sure whether this is something easy to do, but it would be very useful to provide a summary of results grouped by project.

For example, on BXR.SU, right now I was trying to see which projects have .fnt files for issue #34. http://bxr.su/s?path=fnt (IPv6) There are 6 pages worth of results, and I have no clue whether these files are only present in FreeBSD/DragonFly, or NetBSD/OpenBSD, too. Easy if you only have 4 projects (e.g. can deselect f/d to see if there are hits on n/o), but, say, on code.metager.de, there is a hundred of projects, and no way to know which ones return hits.

When doing a search, and especially with several separate projects, a summary should be provided on the number of hits in each project / index.

When you try to print from source code view from either firefox or safari (tested on mac and solaris). You either get only first page with the top part of the code in firefox or nothing at all in safari. I can easily reproduce this if needed.

status CLOSED severity minor in component doc for ---
Reported in version unspecified on platform Other
Assigned to: Trond Norbye

On 2008-04-07 05:12:13 +0000, Trond Norbye wrote:

Mail from [email protected] on the mailing list:

"Running with the -e Economical option Opengrok 6 built a spelling correction database.
This is not a bad thing because it is usefull and does not consume nearly as much space as the hyper text cross reference files. The documentation is now wrong."

On 2008-04-07 05:55:43 +0000, Trond Norbye wrote:

Updated the documentation for the -e option

The history cache of renamed files in Mercurial repository is generated in useless way during reindexing (contrary to when generated by clicking on the History link).

This means the commit c8af363 will not be of any help.

status CLOSED severity normal in component webapp for ---
Reported in version unspecified on platform i86pc/i386
Assigned to: Trond Norbye

Original attachment names and IDs:

• opengrok.JPG (ID 2315)

On 2009-07-26 22:17:16 +0000, Harshit wrote:

Hi,

I just downloaded the latest opengrok (0.7). Checked out the code from the CVS and placed it under the SRC_ROOT directory. Ran the indexer to populate the DATA_ROOT directory.

But when I deploy source.war (after modifying its web.xml to point to correct DATA_ROOT and SRC_ROOT) and move a particular file then the page shows the "History" and "Annotate" links as disabled.

The History link was supposed to provide the cool diffs (m a big fan of that !) http://src.opensolaris.org/source/history/opengrok/trunk/src/org/opensolaris/opengrok/management/oga.properties and the 'Annotate' link is supposed to provide the details that who last edited a particular line and in what revision.

Please let me know if I missed something in the steps. And also note that this was a fresh checkout from the repository, but each folder has 'cvs' folder inside it which is supposed to carry all the cvs meta-information.

Thanks in advance,
Guntas

[Please check the attached screenshot]

On 2009-07-26 22:18:51 +0000, Harshit wrote:

Created attachment 2315
Screenshot of the disabled History and Annotate link

On 2009-07-26 23:48:55 +0000, Lubos Kosco wrote:

Can you try this:

http://blogs.sun.com/trond/entry/using_cvs_with_opengrok

also note, that cvs has to be in the path of web container, otherwise you need to explicitly provide it to indexer so it will get to config.xml:
http://src.opensolaris.org/source/xref/opengrok/trunk/OpenGrok# 147

thanks
L

On 2009-07-27 02:09:23 +0000, Harshit wrote:

(In reply to comment # 2)

Can you try this:

http://blogs.sun.com/trond/entry/using_cvs_with_opengrok

also note, that cvs has to be in the path of web container, otherwise you need
to explicitly provide it to indexer so it will get to config.xml:
http://src.opensolaris.org/source/xref/opengrok/trunk/OpenGrok# 147

thanks
L

Thanks Lubos. The problem is not with the path but rather inability of the OpenGrok (0.7) to connect to remote CVS repositories. I read that this has been fixed in 0.7+ but the release is pending. For the time being I have installed opengrok so that repository is available locally.

And secondly, the diffs which OpenGrok is showing are reverse. Say in 'Revision2'f a file you added some method in a file which was not there in 'Revision1'. OpenGrok is showing me that what should be deleted in 'Revision2' of the file to get 'Revision1'!! And also, it marking the latest revision as 'Old' and the oldest as 'New' while showing the tabs.

Refer to this http://cvs.opensolaris.org/source/diff/opengrok/trunk/src/org/opensolaris/opengrok/management/AgentIndexRunner.java?r1=%252Fopengrok%252Ftrunk%252Fsrc%252Forg%252Fopensolaris%252Fopengrok%252Fmanagement%252FAgentIndexRunner.java%40654%3Ae5c593bd885e&r2=%252Fopengrok%252Ftrunk%252Fsrc%252Forg%252Fopensolaris%252Fopengrok%252Fmanagement%252FAgentIndexRunner.java%40639%3A637aeb8fec5e to get an idea.

• Is this expected?
• How can I get the diff in such a way that it shows me that what should be added in 'Revision1' to get the 'Revision2'?

Thanks,
Guntas

On 2009-07-27 02:14:40 +0000, Harshit wrote:

(In reply to comment # 3)

(In reply to comment # 2)

Can you try this:

http://blogs.sun.com/trond/entry/using_cvs_with_opengrok

also note, that cvs has to be in the path of web container, otherwise you need
to explicitly provide it to indexer so it will get to config.xml:
http://src.opensolaris.org/source/xref/opengrok/trunk/OpenGrok# 147

thanks
L

Thanks Lubos. The problem is not with the path but rather inability of the
OpenGrok (0.7) to connect to remote CVS repositories. I read that this has been
fixed in 0.7+ but the release is pending. For the time being I have installed
opengrok so that repository is available locally.

And secondly, the diffs which OpenGrok is showing are reverse. Say in
'Revision2'f a file you added some method in a file which was not there in
'Revision1'. OpenGrok is showing me that what should be deleted in
'Revision2' of the file to get 'Revision1'!! And also, it marking the latest
revision as 'Old' and the oldest as 'New' while showing the tabs.

Refer to this
http://cvs.opensolaris.org/source/diff/opengrok/trunk/src/org/opensolaris/opengrok/management/AgentIndexRunner.java?r1=%252Fopengrok%252Ftrunk%252Fsrc%252Forg%252Fopensolaris%252Fopengrok%252Fmanagement%252FAgentIndexRunner.java%40654%3Ae5c593bd885e&r2=%252Fopengrok%252Ftrunk%252Fsrc%252Forg%252Fopensolaris%252Fopengrok%252Fmanagement%252FAgentIndexRunner.java%40639%3A637aeb8fec5e
to get an idea.

• Is this expected?
• How can I get the diff in such a way that it shows me that what should be
  added in 'Revision1' to get the 'Revision2'?

Thanks,
Guntas

Ohhh.. my bad :-)
Found that I was selecting the files to be diff in such a way that newest was selected first and then the oldest one.

Sorry for the confusion.

Thanks,
Guntas

Fresh clone of opengrok repo and:

$ ant
Buildfile: build.xml

BUILD FAILED
/tmp/jflex-test/OpenGrok/build.xml:149: Please download JFlex (http://jflex.de/) and put JFlex.jar it into ./lib directory (or in ant classpath).

Total time: 0 seconds

My expectation is that jflex should be downloaded automatically just like Lucene given the download-jflex target in build.xml.

After downloading Jflex.jar by hand and placing it into the lib directory and re-running ant, Lucene is downloaded successfully so there is no problem with proxies etc.

I'm running {OpenGrok 0.11-rc2 with the -O off option (together with -v etc), and I am keeping a separate log file for each run.

I've noticed that my fresh daily log only grows with each day; that is, the log of today's run is bigger than the log of yesterday's run. Even doing an empty run (without any changes of the indexed source code) results in a lot of entries of files being removed (org.opensolaris.opengrok.index.DefaultIndexChangedListener fileRemove), when the source code hasn't changed at all from the prior run, and these files that are listed as removed, are not simultaneously listed as being added back, either, but they're still supposed to be part of the index.

When the index is not optimised, is it indeed expected to be bombarded with all the fileRemove messages for all the files that have changed since the original index, but which were already supposed to have been re-indexed on the follow-up non-optimised reindex runs?

more examples for regexp ( http://lucene.apache.org/core/4_2_1/core/org/apache/lucene/util/automaton/RegExp.html?is-external=true )
more explanation about fields

status CLOSED severity major in component indexer for ---
Reported in version unspecified on platform i86pc/i386
Assigned to: Trond Norbye

On 2009-07-28 03:14:07 +0000, Harshit wrote:

Hi,

In my "source_root" folder there are some 30 checked out CVS modules. When I try to run the indexer to populate the "data_root" folder i see that that the xref (inside ${data_root}/xref) got generated only for few of the projects (and not all projects in source_root). While the indexer was running I got the following WARNINGs in the log:

===========ERROR============
Jul 28, 2009 5:54:35 AM org.opensolaris.opengrok.index.IndexDatabase indexDown
WARNING: Failed to add file path/to/file
java.lang.IndexOutOfBoundsException
at java.nio.Buffer.checkIndex(Buffer.java:514)
at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:209)
at org.opensolaris.opengrok.analysis.executables.ELFAnalyzer.getName(ELFAnalyzer.java:166)
at org.opensolaris.opengrok.analysis.executables.ELFAnalyzer.parseELF(ELFAnalyzer.java:111)
at org.opensolaris.opengrok.analysis.executables.ELFAnalyzer.analyze(ELFAnalyzer.java:79)
at org.opensolaris.opengrok.analysis.AnalyzerGuru.getDocument(AnalyzerGuru.java:246)
at org.opensolaris.opengrok.index.IndexDatabase.addFile(IndexDatabase.java:458)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase.java:578)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase.java:561)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase.java:561)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase.java:561)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase.java:561)
at org.opensolaris.opengrok.index.IndexDatabase.update(IndexDatabase.java:251)
at org.opensolaris.opengrok.index.IndexDatabase$1.run(IndexDatabase.java:139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)

at java.lang.Thread.run(Thread.java:619)

Any idea why not all the projects are getting indexed? How does the indexer get to know that what all projects need to be indexed or does it assume that everything inside the ${source_root} needs to be indexed?

Thanks,
Guntas.

On 2009-07-28 03:26:58 +0000, Harshit wrote:

I want the indexer to continue indexing the files even after encountering the above exception.

Is there any switch by which i can convey to the indexer that continue indexing and just ignore these exceptions (and the files which could not be added) ?

Please help me know this. FYI, I am using latest 0.7 version.

Thanks,
Guntas.

On 2009-07-28 04:18:02 +0000, Harshit wrote:

(In reply to comment # 1)

I want the indexer to continue indexing the files even after encountering the
above exception.

Is there any switch by which i can convey to the indexer that continue indexing
and just ignore these exceptions (and the files which could not be added) ?

Please help me know this. FYI, I am using latest 0.7 version.

Thanks,
Guntas.

I just found that the indexer is not getting killed because of the exception "java.lang.IndexOutOfBoundsException".

Rather the indexer is not able to find that there are other projects in ${source_root}.

Could you please help me know that how does the indexer gets to know that what all projects need to be indexed?

Thanks.

On 2009-07-28 05:57:17 +0000, Harshit wrote:

(In reply to comment # 2)

(In reply to comment # 1)

I want the indexer to continue indexing the files even after encountering the
above exception.

Is there any switch by which i can convey to the indexer that continue indexing
and just ignore these exceptions (and the files which could not be added) ?

Please help me know this. FYI, I am using latest 0.7 version.

Thanks,
Guntas.

I just found that the indexer is not getting killed because of the exception
"java.lang.IndexOutOfBoundsException".

Rather the indexer is not able to find that there are other projects in
${source_root}.

Could you please help me know that how does the indexer gets to know that what
all projects need to be indexed?

Thanks.

I solved the above problem by passing -P option to the indexer. Went through the source code and saw that by using -P option OpenGrok will consider all the top-level folders inside "source-root" as separate project.

Thanks.

With the latest {OpenGrok from master, I ran re-index of minix several times, without optimisation (-O off), on a brand new source tree.

Now if I search for anything, I get each file repeated in the search results 3 separate times, with all the same lines and stuff found. (3 is probably the total number of times I ran the re-indexing.)

It doesn't seem like it's possible for the administrator to pre-select multiple default projects, for the situation where no project is provided by the user-agent through the cookies.

The -p option seems to require a path, e.g. -p "/OpenBSD", and it would seem that providing multiple comma-separated values does not result in any desired effects.

This is what the documentation says:

-p /path/to/default/project
This is the path to the project that should be selected by default in the web application(when no other project set either in cookie or in parameter). You should strip off the source root.

The latest code from master duplicates (sometimes multiplicates) the results when searching.

We should migrate the bugs from the old Bugzilla instance which had been running on defect.opensolaris.org. This should include both closed and open bugs.

I am in the process of figuring out the best translation/formatting etc.

This will be done with the help of https://github.com/vladak/BugzillaMigrate

If the search query has quotes and an OR operator, the results are not highlighted correctly — only the last term from within the quotes is highlighted.

For example, http://bxr.su/s?q=%22Robert%20Watson%22%20OR%20rwatson, where the query is, "Robert Watson" OR rwatson, produces highlighting like Robert <b>Watson</b>, e.g. only the second/last term from within the quotes is highlighted. Similarly, http://bxr.su/s?q=%22Constantine+A.+Murenin%22+OR+cnst highlights as Constantine A. <b>Murenin</b> &lt;<b>cnst</b>.

I can reproduce this with OpenGrok 0.10, some 0.11-rc2 from an old trunk, as well as the recent master, so, it probably never worked right.

status CLOSED severity minor in component analyzer for ---
Reported in version unspecified on platform ANY/Generic
Assigned to: Trond Norbye

On 2009-08-04 05:09:24 +0000, Lubos Kosco wrote:

Aug 3, 2009 6:55:33 AM org.opensolaris.opengrok.index.IndexDatabase indexDown
WARNING: Failed to add file /tank/opengrok-sources/n1aa-HEAD/tools/jdk142_04_sparc/usr/j2se/jre/lib/sparcv9/libmlib_image_v.s
o
java.lang.IndexOutOfBoundsException
at java.nio.Buffer.checkIndex(Buffer.java:444)
at java.nio.DirectByteBuffer.getInt(DirectByteBuffer.java:624)
at org.opensolaris.opengrok.analysis.executables.ELFAnalyzer.parseELF(ELFAnalyzer.java:103)
at org.opensolaris.opengrok.analysis.executables.ELFAnalyzer.analyze(ELFAnalyzer.java:79)
at org.opensolaris.opengrok.analysis.AnalyzerGuru.getDocument(AnalyzerGuru.java:246)
at org.opensolaris.opengrok.index.IndexDatabase.addFile(IndexDatabase.java:458)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase.java:578)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase.java:561)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase.java:561)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase.java:561)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase.java:561)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase.java:561)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase.java:561)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase.java:561)
at org.opensolaris.opengrok.index.IndexDatabase.update(IndexDatabase.java:251)
at org.opensolaris.opengrok.index.IndexDatabase$1.run(IndexDatabase.java:139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269)
at java.util.concurrent.FutureTask.run(FutureTask.java:123)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)

On 2009-10-24 21:26:35 +0000, Knut Anders Hatlen wrote:

*** This bug has been marked as a duplicate of bug 9426 ***

It is not easy to display diffs of all files changed by a single commit. It should be possible to get similar view as with Mercurial or git web interfaces.

appears for most of my builds:

junit.framework.AssertionFailedError
at org.opensolaris.opengrok.history.HistoryGuruTest.testBug16465(HistoryGuruTest.java:129)

happens to Lubos too.

{OpenGrok does not let you cross reference manual pages. Instead of cross-referencing, it supposedly attempts to perform some form of conversion from troff into html.

No cross referencing is done, no source code can be viewed in the browser (the Download link wants you to save the file first), the Annotate function doesn't work at all.

I'm not sure if this design decision was intentional or simply a proof-of-concept, but it has been proving especially troublesome for the popular open-source BSD flavours and the mdoc / mandoc troff format, where the HTML rendering is simply a complete mess:

http://nxr.netbsd.org/xref/src/share/man/man7/mdoc.7

For my http://BXR.SU project (IPv6-only for now), I have rewritten OpenGrok's troff grammars, with the cross-reference and BSD mdoc format in mind:

http://bxr.su/n/share/man/man7/mdoc.7 (might require IPv6)

I was originally thinking that perhaps it would make sense to contribute my grammars as a separate mdoc format to the main {OpenGrok upstream, but have since contemplated that perhaps it would make more sense to replace the existing non-xref xref grammar to actually be xref as advertised? I'm not sure that many people do come to an xref service to find groff instead; I'd think more users would expect xref when requesting xref.

Any thoughts of how I should go here?

(I'm in the process of converting my modifications from my {OpenGrok 0.11-rc2 fork from an old hg trunk to the recent github master, code will be avail soon, pull requests to follow.)

status CLOSED severity critical in component webapp for ---
Reported in version unspecified on platform i86pc/i386
Assigned to: Trond Norbye

On 2009-07-28 07:00:18 +0000, Harshit wrote:

Hi,

I generated the indices by running the indexer. But when I try to search for a file then it does not show me result from the just-now indexed projects.

Two things could be possible:

1. Either the file to be searched is not present in the newly indexed projects.
2. Or there is some synchronization problem between the webapp and the indexer. Indexer did the job but webapp is not able to read from the latest index.

Negating 1 as i am sure that the file resides in the newly indexed project.

Could you please help me know why the webapp is not showing me the records from the newly indexed projects?

But when i re-deploy the webapp by starting and stopping the application server, it shows me the results correctly.

Thanks.

On 2009-07-28 07:49:11 +0000, Lubos Kosco wrote:

restarting the webapp is one option

what the rest of us is doing is remote update using send of configuration xml
see http://opensolaris.org/os/project/opengrok/manual/installdescription/
"Suggested Step - Send the project XML config file to local web application"

On 2009-07-31 02:12:24 +0000, Harshit wrote:

Thanks Lucos this procedure worked for me.

Closing this ticket.

If you use eclipse, it would be nice to have a eclipse plugin for OpenGrok that supported a search interface. Today I use opengrok in a browser, find what i want, then go back to Eclipse to locate the same file. It would be nice if it was all inside eclipse.

Even better, if it used a google-suggest method for giving options as you typed.

Related discussion:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=162160

Some xref directory pages have multiple README files, and these pages are not valid XHTML.

https://github.com/OpenGrok/OpenGrok/blob/master/web/list.jsp#L106

        List<String> readMes = dl.listTo(resourceFile, out, path, files);
        File[] catfiles = cfg.findDataFiles(readMes);
        for (int i=0; i < catfiles.length; i++) {
            if (catfiles[i] == null) {
                continue;
            }
%><h3><%= readMes.get(i) %></h3>
<div id="src">
<pre><%
            Util.dump(out, catfiles[i], catfiles[i].getName().endsWith(".gz"));
%></pre>
</div><%
        }

Is it at all reliable to use {OpenGrok with only one staging area?

I'm fine if there is a certain point in time that there is an inconsistency between the updated source files and the actual data/index (besides, with the use of data/xref, isn't even this problem already fixed?), and in my limited and non-scientific testing, I found no problems whatsoever with having a single stage area.

So, are there any known problems or weird bugs when using a single staging area?

When the index database is temporarily gone, OpenGrok returns "Index database(s) not found." (https://github.com/OpenGrok/OpenGrok/blob/master/src/org/opensolaris/opengrok/web/SearchHelper.java) with a 200 OK HTTP status code.

This breaks down HTTP accelerators (and probably various web-crawlers, too), which cache error pages momentarily (seconds), but non-error pages for an extended period of time (minutes or hours), where such OpenGrok error pages will be cached extensively, instead of being expired promptly. Plus web-crawlers might update its index, instead of ignoring the result, and trying back later.

With "Index database(s) not found.", a non-200 http status code should be returned instead, probably 503 Service Unavailable.

Based on the feedback from the users of our OpenGrok instance, the history browser part deserves lots of love. The main gripe is that it is not possible to comprehensively display last couple of commits to a repository and from there display all files changed by a commit and respective diffs.

Basically, the idea is to have something like presented by web interfaces for git or Mercurial. For example http://git.openssl.org/gitweb/?p=openssl.git;a=summary displays 3 sections: shortlog, tags and heads. Each of these sections has couple of links for its entries (like: commit details, commit diff, source tree view, download snapshot of the repo).

The problem of current way of dealing with history view for the whole repo is that it is not paginated. For big repos this can make the view completely unusable. This can be treated like a separate bug but the ultimate solution is to present dashboard like view as described above.

status CLOSED severity minor in component analyzer for 0.8
Reported in version unspecified on platform ANY/Generic
Assigned to: Knut Anders Hatlen

On 2009-08-04 05:10:23 +0000, Lubos Kosco wrote:

Aug 3, 2009 6:55:02 AM org.opensolaris.opengrok.index.IndexDatabase indexDown
WARNING: Failed to add file /tank/opengrok-sources/coso-Main-HEAD/src/artifacts/ChassisMgmtTOI.odp
java.lang.IllegalArgumentException
at java.util.zip.ZipInputStream.getUTF8String(ZipInputStream.java:299)
at java.util.zip.ZipInputStream.readLOC(ZipInputStream.java:238)
at java.util.zip.ZipInputStream.getNextEntry(ZipInputStream.java:73)
at org.opensolaris.opengrok.analysis.archive.ZipAnalyzer.analyze(ZipAnalyzer.java:69)
at org.opensolaris.opengrok.analysis.AnalyzerGuru.getDocument(AnalyzerGuru.java:246)
at org.opensolaris.opengrok.index.IndexDatabase.addFile(IndexDatabase.java:458)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase.java:578)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase.java:561)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase.java:561)
at org.opensolaris.opengrok.index.IndexDatabase.update(IndexDatabase.java:251)
at org.opensolaris.opengrok.index.IndexDatabase$1.run(IndexDatabase.java:139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269)
at java.util.concurrent.FutureTask.run(FutureTask.java:123)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)

On 2009-08-30 19:00:02 +0000, Knut Anders Hatlen wrote:

I tried to index an odp myself and didn't see any exception, so it apparently doesn't affect all odps. Could you please attach a reproducible test case?

As far as I understand the stack trace, it indicates that the odp file is not a valid zip file. Or at least that ZipInputStream doesn't understand it. In either of those cases the bug is not in OpenGrok, but it's hard to tell without seeing an odp that shows the reported behaviour.

On 2009-08-31 06:40:34 +0000, Lubos Kosco wrote:

(In reply to comment # 1)

I tried to index an odp myself and didn't see any exception, so it apparently
doesn't affect all odps. Could you please attach a reproducible test case?

As far as I understand the stack trace, it indicates that the odp file is not a
valid zip file. Or at least that ZipInputStream doesn't understand it. In
either of those cases the bug is not in OpenGrok, but it's hard to tell without
seeing an odp that shows the reported behaviour.

the odp is broken indeed ... I couldn't open it using zip either (will send it to you internally, since it's Sun proprietary) ...
but on the other hand I think we should fix this as being reported only in FINE level and just a WARNING about the file not beind indexed because unreadable

So it's really like a bug against logging of stacks, where they really don't make sense to report to WARNING level

we had a talk with Trond about this, I agree with him on showing stacks where appropriate, but this is not appropriate for sure ...

thnx
L

On 2009-08-31 07:30:38 +0000, Knut Anders Hatlen wrote:

Would you be fine with logging a message about the file being skipped because the analyzer didn't understand it with INFO level, and the actual exception with FINE?

It would probably make sense to change FileAnalyzer.analyze(Document,InputStream) so that it's declared to throw exceptions. Then the logging doesn't have to be done in each analyzer, but can instead be done centralized on a higher level which makes it easier to get consistent logging across the analyzers.

On 2009-08-31 07:35:19 +0000, Knut Anders Hatlen wrote:

*** Bug 10443 has been marked as a duplicate of this bug. ***

On 2009-08-31 08:32:14 +0000, Lubos Kosco wrote:

ACK, go for it kah ! :)

(In reply to comment # 3)

Would you be fine with logging a message about the file being skipped because
the analyzer didn't understand it with INFO level, and the actual exception
with FINE?

It would probably make sense to change
FileAnalyzer.analyze(Document,InputStream) so that it's declared to throw
exceptions. Then the logging doesn't have to be done in each analyzer, but can
instead be done centralized on a higher level which makes it easier to get
consistent logging across the analyzers.

On 2009-10-25 19:23:24 +0000, Knut Anders Hatlen wrote:

Fixed in changeset 889:70983e950e18.

On 2009-10-25 19:30:29 +0000, Knut Anders Hatlen wrote:

*** Bug 9426 has been marked as a duplicate of this bug. ***

On 2010-02-17 12:47:24 +0000, Knut Anders Hatlen wrote:

*** Bug 3481 has been marked as a duplicate of this bug. ***

With {OpenGrok 0.10, a /s?defs=D&project=P link, when only one definition of D is available, returns a 302 Moved Temporarily.

With {OpenGrok 0.11-rc2, this is no longer the case.

If you index the OpenGrok code base and then search for "bject" in the full text search, you'll get a match in testdata/sources/java/Main.class. However, there is no "bject" token in that file. It's probably "Object" that has been garbled.

This is most likely a regression. Tested the same search on src.opensolaris.org, which runs 0.9, and it returned 0 matches.

status CLOSED severity normal in component analyzer for 0.9
Reported in version unspecified on platform Other
Assigned to: Knut Anders Hatlen

On 2009-07-16 09:55:06 +0000, Bruce Furber wrote:

INFO: Starting indexExecution
Remove stale file: /gal/devl/r/rr55uav4.asm
Jul 16, 2009 12:40:31 PM org.opensolaris.opengrok.analysis.plain.PlainAnalyzer a nalyze
WARNING: An error occured while analyzing stream.
sun.io.MalformedInputException
at sun.io.ByteToCharUTF8.convert(ByteToCharUTF8.java:262)
at sun.nio.cs.StreamDecoder$ConverterSD.convertInto(StreamDecoder.java:3 16)
at sun.nio.cs.StreamDecoder$ConverterSD.implRead(StreamDecoder.java:366)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:252)
at java.io.InputStreamReader.read(InputStreamReader.java:212)
at org.opensolaris.opengrok.analysis.plain.PlainAnalyzer.analyze(PlainAn alyzer.java:85)
at org.opensolaris.opengrok.analysis.AnalyzerGuru.getDocument(AnalyzerGu ru.java:246)
at org.opensolaris.opengrok.index.IndexDatabase.addFile(IndexDatabase.ja va:458)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase. java:578)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase. java:561)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase. java:561)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase. java:561)
at org.opensolaris.opengrok.index.IndexDatabase.update(IndexDatabase.jav a:251)
at org.opensolaris.opengrok.index.IndexDatabase$1.run(IndexDatabase.java :139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:43 2)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:284)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExec utor.java:665)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor .java:690)
at java.lang.Thread.run(Thread.java:810)
Adding: /gal/devl/r/rr55uav4.asm (PlainAnalyzer)
Jul 16, 2009 12:40:42 PM org.opensolaris.opengrok.index.IndexDatabase optimize

On 2009-07-16 12:29:49 +0000, Bruce Furber wrote:

The file in question had a single broken bar ? x'A6' in it

On 2010-02-03 13:21:37 +0000, Bruce Furber wrote:

It would be a big improvement if it simply treated characters outside the range or x'20' thru x'7F' as whitespace.

On 2010-04-16 11:43:51 +0000, Lubos Kosco wrote:

xref analyzers are now Unicode compatible in a way that they parse every character which is not an ISO control char:

http://src.opensolaris.org/source/xref/opengrok/trunk/src/org/opensolaris/opengrok/analysis/JFlexXref.java# 116

if it's an ISO control char, it's ignored

hence closing this bug

On 2010-04-16 11:44:47 +0000, Lubos Kosco wrote:

oh, forgot to mention that bug # 437 holds more on similar issues

Since opensolaris.org is going away in a month or so and the source code has a lot of references to it (not mentioning the directory structure) we should clean them up.

status CLOSED severity major in component indexer for ---
Reported in version unspecified on platform i86pc/i386
Assigned to: Trond Norbye

Original attachment names and IDs:

• OpenGrok-Cron.txt (ID 2362)
• run-modified.sh (ID 2363)

On 2009-08-02 23:21:45 +0000, Harshit wrote:

Hi,

I have daily cron job which does the following things:

1. cvs update the ${SRC_ROOT}
2. Run the indexer: java -jar ${PROGDIR}/opengrok.jar -S -W ${PROGDIR}/opengrok-configuration.xml -c ${EXUB_CTAGS} -s ${SRC_ROOT} -d ${DATA_ROOT} -P -v
3. Pushes the new configuration to the web app: java -jar ${PROGDIR}/opengrok.jar -U ${WEBAPP_URL} -R ${PROGDIR}/opengrok-configuration.xml -n

But I am seeing that the opengrok processes are never getting terminated. Please see the process details of the user ("my_user") using which the cron job is started.
=============PROCESS DETAILS====================
/user/my_user> ps -ef | grep my_user
my_user 16568 15336 0 01:17:30 ? 0:00 /usr/local/bin/exuberant-ctags --c-kinds=+l --java-kinds=+l --sql-kinds=+l --Fo
my_user 15336 4469 0 01:16:14 ? 5:28 java -jar opengrok.jar -S -W config.xml
my_user 16414 15336 0 01:16:40 ? 0:00 /usr/local/bin/exuberant-ctags --c-kinds=+l --java-kinds=+l --sql-kinds=+l --Fo
my_user 27338 21569 0 02:10:38 pts/40 0:00 ps -ef
my_user 16549 15336 0 01:17:18 ? 0:00 /usr/local/bin/exuberant-ctags --c-kinds=+l --java-kinds=+l --sql-kinds=+l --Fo
my_user 18096 15336 13 01:20:18 ? 50:15 /usr/local/bin/exuberant-ctags --c-kinds=+l --java-kinds=+l --sql-kinds=+l --Fo
my_user 16745 15336 0 01:17:40 ? 0:00 /usr/local/bin/exuberant-ctags --c-kinds=+l --java-kinds=+l --sql-kinds=+l --Fo
my_user 16770 15336 0 01:18:02 ? 0:00 /usr/local/bin/exuberant-ctags --c-kinds=+l --java-kinds=+l --sql-kinds=+l --Fo
my_user 21569 21567 0 01:28:04 pts/40 0:00 -tcsh
my_user 16746 15336 0 01:17:40 ? 0:00 /usr/local/bin/exuberant-ctags --c-kinds=+l --java-kinds=+l --sql-kinds=+l --Fo
my_user 16780 15336 0 01:18:31 ? 0:00 /usr/local/bin/exuberant-ctags --c-kinds=+l --java-kinds=+l --sql-kinds=+l --Fo
my_user 23994 1 0 Aug 02 ? 0:57 /usr/local/java/jdk/bin/java -Djava.util.logging.config.file=config.xml
my_user 16775 15336 0 01:18:15 ? 0:00 /usr/local/bin/exuberant-ctags --c-kinds=+l --java-kinds=+l --sql-kinds=+l --Fo
my_user 4442 15582 0 00:00:01 ? 0:00 sh -c opengrok-0.7/run.sh > /tmp/logs/logfile.log 2>&1
my_user 16747 15336 0 01:17:41 ? 0:00 /usr/local/bin/exuberant-ctags --c-kinds=+l --java-kinds=+l --sql-kinds=+l --Fo
my_user 27339 21569 0 02:10:38 pts/40 0:00 grep opengrok
my_user 4469 4442 0 00:00:01 ? 0:00 /usr/bin/sh opengrok-0.7/run.sh

my_user 17221 15336 0 01:19:15 ? 0:00 /usr/local/bin/exuberant-ctags --c-kinds=+l --java-kinds=+l --sql-kinds=+l --Fo

Could you please help me know why the above processes are always running? Please check the cmds above which I am running using the cronjob.

Thanks.

On 2009-08-03 03:02:22 +0000, Trond Norbye wrote:

This is most likely caused by an exception in the thread reading the configurations in the web server. Please run /usr/jdk/latest/bin/jstack and add the output here...

On 2009-08-03 03:36:08 +0000, Harshit wrote:

(In reply to comment # 1)

This is most likely caused by an exception in the thread reading the
configurations in the web server. Please run /usr/jdk/latest/bin/jstack and add the output here...

This is how the jstack output looks like:

=========JSTACK OUTPUT=======
2009-08-03 06:18:08
Full thread dump Java HotSpot(TM) Server VM (14.0-b16 mixed mode):

"Attach Listener" daemon prio=3 tid=0x084a9000 nid=0x20 waiting on condition [0x00000000]
java.lang.Thread.State: RUNNABLE

"http-8888-5" daemon prio=3 tid=0x08778800 nid=0x1f in Object.wait() [0xb60dd000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xf6b4e0e0> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
at java.lang.Object.wait(Object.java:485)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
- locked <0xf6b4e0e0> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449) at java.lang.Thread.run(Thread.java:619)

"http-8888-4" daemon prio=3 tid=0x08265000 nid=0x1e in Object.wait() [0xb612e000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xf6b50068> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
at java.lang.Object.wait(Object.java:485)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
- locked <0xf6b50068> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449) at java.lang.Thread.run(Thread.java:619)

"http-8888-3" daemon prio=3 tid=0x0876ac00 nid=0x1d runnable [0xb61b4000]
java.lang.Thread.State: RUNNABLE
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:129)
at org.apache.coyote.http11.InternalInputBuffer.fill(InternalInputBuffer.java:735)
at org.apache.coyote.http11.InternalInputBuffer.parseRequestLine(InternalInputBuffer.java:366)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:808)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:454) at java.lang.Thread.run(Thread.java:619)

"http-8888-2" daemon prio=3 tid=0x088bd400 nid=0x1c in Object.wait() [0xb6205000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xbae048d0> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
at java.lang.Object.wait(Object.java:485)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
- locked <0xbae048d0> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449) at java.lang.Thread.run(Thread.java:619)

"http-8888-1" daemon prio=3 tid=0x08749800 nid=0x1b in Object.wait() [0xb6256000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xbae05068> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
at java.lang.Object.wait(Object.java:485)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.await(JIoEndpoint.java:423)
- locked <0xbae05068> (a org.apache.tomcat.util.net.JIoEndpoint$Worker)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:449) at java.lang.Thread.run(Thread.java:619)

"TP-Monitor" daemon prio=3 tid=0x08886c00 nid=0x1a in Object.wait() [0xb62a7000]
java.lang.Thread.State: TIMED_WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xbae05080> (a org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable)
at org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable.run(ThreadPool.java:565)
- locked <0xbae05080> (a org.apache.tomcat.util.threads.ThreadPool$MonitorRunnable)
at java.lang.Thread.run(Thread.java:619)

"TP-Processor4" daemon prio=3 tid=0x0874c000 nid=0x19 runnable [0xb62f8000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
- locked <0xbae00120> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:453)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at org.apache.jk.common.ChannelSocket.accept(ChannelSocket.java:307)
at org.apache.jk.common.ChannelSocket.acceptConnections(ChannelSocket.java:661)
at org.apache.jk.common.ChannelSocket$SocketAcceptor.runIt(ChannelSocket.java:872)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:690)
at java.lang.Thread.run(Thread.java:619)

"TP-Processor3" daemon prio=3 tid=0x08347c00 nid=0x18 in Object.wait() [0xb6349000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xbae007f8> (a org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
at java.lang.Object.wait(Object.java:485)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:662)
- locked <0xbae007f8> (a org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
at java.lang.Thread.run(Thread.java:619)

"TP-Processor2" daemon prio=3 tid=0x08ac9000 nid=0x17 in Object.wait() [0xb639a000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xbae003c8> (a org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
at java.lang.Object.wait(Object.java:485)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:662)
- locked <0xbae003c8> (a org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
at java.lang.Thread.run(Thread.java:619)

"TP-Processor1" daemon prio=3 tid=0x088b1800 nid=0x16 in Object.wait() [0xb63eb000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xbae00468> (a org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
at java.lang.Object.wait(Object.java:485)
at org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:662)
- locked <0xbae00468> (a org.apache.tomcat.util.threads.ThreadPool$ControlRunnable)
at java.lang.Thread.run(Thread.java:619)

"http-8888-Acceptor-0" daemon prio=3 tid=0x08ac8800 nid=0x15 runnable [0xb643c000] java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
- locked <0xbae020f8> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:453)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at org.apache.tomcat.util.net.DefaultServerSocketFactory.acceptSocket(DefaultServerSocketFactory.java:61)
at org.apache.tomcat.util.net.JIoEndpoint$Acceptor.run(JIoEndpoint.java:317)
at java.lang.Thread.run(Thread.java:619)

"ContainerBackgroundProcessor[StandardEngine[Catalina]]" daemon prio=3 tid=0x08429c00 nid=0x14 waiting on condition [0xb648d000]
java.lang.Thread.State: TIMED_WAITING (sleeping)
at java.lang.Thread.sleep(Native Method)
at org.apache.catalina.core.ContainerBase$ContainerBackgroundProcessor.run(ContainerBase.java:1579)
at java.lang.Thread.run(Thread.java:619)

"Thread-1" prio=3 tid=0x08613400 nid=0x13 runnable [0xb64de000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
- locked <0xbae02298> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:453)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at org.opensolaris.opengrok.configuration.RuntimeEnvironment$2.run(RuntimeEnvironment.java:615)
at java.lang.Thread.run(Thread.java:619)

"Low Memory Detector" daemon prio=3 tid=0x081ab000 nid=0x11 runnable [0x00000000]
java.lang.Thread.State: RUNNABLE

"CompilerThread1" daemon prio=3 tid=0x081a8400 nid=0x10 waiting on condition [0x00000000]
java.lang.Thread.State: RUNNABLE

"CompilerThread0" daemon prio=3 tid=0x081a6c00 nid=0xf waiting on condition [0x00000000]
java.lang.Thread.State: RUNNABLE

"Signal Dispatcher" daemon prio=3 tid=0x081a5400 nid=0xe runnable [0x00000000]
java.lang.Thread.State: RUNNABLE

"Finalizer" daemon prio=3 tid=0x0818c800 nid=0xd in Object.wait() [0xb6794000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xbae025d0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:118)
- locked <0xbae025d0> (a java.lang.ref.ReferenceQueue$Lock)
at java.lang.ref.ReferenceQueue.remove(ReferenceQueue.java:134)
at java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:159)

"Reference Handler" daemon prio=3 tid=0x0818b400 nid=0xc in Object.wait() [0xb67e5000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0xbae02690> (a java.lang.ref.Reference$Lock)
at java.lang.Object.wait(Object.java:485)
at java.lang.ref.Reference$ReferenceHandler.run(Reference.java:116)
- locked <0xbae02690> (a java.lang.ref.Reference$Lock)

"main" prio=3 tid=0x08070400 nid=0x2 runnable [0xfe42e000]
java.lang.Thread.State: RUNNABLE
at java.net.PlainSocketImpl.socketAccept(Native Method)
at java.net.PlainSocketImpl.accept(PlainSocketImpl.java:390)
- locked <0xbae027c0> (a java.net.SocksSocketImpl)
at java.net.ServerSocket.implAccept(ServerSocket.java:453)
at java.net.ServerSocket.accept(ServerSocket.java:421)
at org.apache.catalina.core.StandardServer.await(StandardServer.java:389)
at org.apache.catalina.startup.Catalina.await(Catalina.java:647)
at org.apache.catalina.startup.Catalina.start(Catalina.java:607)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:288)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:413)

"VM Thread" prio=3 tid=0x08187800 nid=0xb runnable

"GC task thread# 0 (ParallelGC)" prio=3 tid=0x08076c00 nid=0x3 runnable

"GC task thread# 1 (ParallelGC)" prio=3 tid=0x08078000 nid=0x4 runnable

"GC task thread# 2 (ParallelGC)" prio=3 tid=0x08079400 nid=0x5 runnable

"GC task thread# 3 (ParallelGC)" prio=3 tid=0x0807a800 nid=0x6 runnable

"GC task thread# 4 (ParallelGC)" prio=3 tid=0x0807bc00 nid=0x7 runnable

"GC task thread# 5 (ParallelGC)" prio=3 tid=0x0807d000 nid=0x8 runnable

"GC task thread# 6 (ParallelGC)" prio=3 tid=0x0807e400 nid=0x9 runnable

"GC task thread# 7 (ParallelGC)" prio=3 tid=0x0807fc00 nid=0xa runnable

"VM Periodic Task Thread" prio=3 tid=0x081ad000 nid=0x12 waiting on condition

JNI global references: 1185

Trond, before reading your update I actually killed all the lingering processes. Will this output be sufficient or do you want me to reproduce that state and send you the output?

Please help me know how to go about this.

Thanks.

On 2009-08-03 03:43:14 +0000, Harshit wrote:

After killing the lingering processes this is what I got in the logfile:

=======Logs=======
Killed
Aug 3, 2009 6:16:17 AM org.opensolaris.opengrok.index.Indexer sendToConfigHost
INFO: Send configuration to: my_host:2424
Aug 3, 2009 6:16:17 AM org.opensolaris.opengrok.index.Indexer sendToConfigHost
SEVERE: Failed to send configuration to my_host:2424
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
at java.net.Socket.connect(Socket.java:519)
at java.net.Socket.connect(Socket.java:469)
at java.net.Socket.(Socket.java:366)
at java.net.Socket.(Socket.java:208)
at org.opensolaris.opengrok.configuration.RuntimeEnvironment.writeConfiguration(RuntimeEnvironment.java:565)
at org.opensolaris.opengrok.index.Indexer.sendToConfigHost(Indexer.java:558)
at org.opensolaris.opengrok.index.Indexer.main(Indexer.java:349)
Aug 3, 2009 6:16:17 AM org.opensolaris.opengrok.index.Indexer sendToConfigHost

INFO: Configuration successfully updated

Thanks.

On 2009-08-03 04:36:27 +0000, Trond Norbye wrote:

What is the OpenGrok version you are using on your web server? Please look at the source for a web page, and you should find something like:

On 2009-08-03 05:08:11 +0000, Harshit wrote:

(In reply to comment # 4)

What is the OpenGrok version you are using on your web server? Please look at
the source for a web page, and you should find something like:

Trond, I am using OpenGrok 0.7 released version. The one which is available for download from http://www.opensolaris.org/os/project/opengrok/ and I downloaded a week back from the above site.

I actually could not find the which you were referring. Could you please point me to the exact file to open up in source.war's exploded version and look for the version?

Thanks.

On 2009-08-03 05:47:47 +0000, Trond Norbye wrote:

Ok, I'll figure out the exact source revision you are using. For future references, just open any page from your opengrok server and select view source in the browser. You should find the meta tag in the page header :-)

On 2009-08-03 21:22:01 +0000, Harshit wrote:

(In reply to comment # 6)

Ok, I'll figure out the exact source revision you are using. For future
references, just open any page from your opengrok server and select view source
in the browser. You should find the meta tag in the page header :-)

Ohh.. I got the meta tag. Here are the details:

Thanks Trond for helping.

On 2009-08-04 00:56:53 +0000, Trond Norbye wrote:

I just verified that the thread that is supposed to read the configuration request from the client is present. Is the problem still present (after you killed all of the old java processes), it could be that killing one of the process generated an IOException (when the socket is closed) in the webapp thread causing it to proceed.

Anyway, we should modify this code to be more robust by adding a timer and running each connection in its own thread.

On 2009-08-04 02:33:16 +0000, Harshit wrote:

Created attachment 2362
OpenGrok cron job

On 2009-08-04 02:36:42 +0000, Harshit wrote:

Created attachment 2363
run-modified.sh which is called daily by cron

On 2009-08-04 02:38:23 +0000, Harshit wrote:

(In reply to comment # 8)

I just verified that the thread that is supposed to read the configuration
request from the client is present. Is the problem still present (after you
killed all of the old java processes), it could be that killing one of the
process generated an IOException (when the socket is closed) in the webapp
thread causing it to proceed.

Yes the problem is still present. Today again when the cron job ran it left the
processes (mostly, exuberant-tags related) running.

And secondly, the problem is not with the IOException which got generated on
killing. The problem is why on running the "indexer" and "pushing the config to
web app" opengrok leaves so many processes running. Are the exuberant-tags
related processes should always be running? Or I am missing something.

I am attaching the cron and the run-modified script which it calls.

Could you please check what is the problem here?

Thanks for all the help.

On 2009-08-04 06:32:53 +0000, Harshit wrote:

Trond,

Could you please check the attached run script? Is this the way that we should call the "indexer" and the "config push"?

And any idea on how to get away with the lingering processes once the script is executed?

Thanks.

On 2009-08-04 07:19:05 +0000, Lubos Kosco wrote:

mhmm

I don't want to interrupt your party guys , but I think we solved this issue in
changeset 750 bdab6a143036

Guntas, can you try out a trunk build ?
(follow http://opensolaris.org/os/project/opengrok/manual/builddescription/ then e.g. $ ant package to get a package of trunk build, or just use stuff from dist/ after the clean build)

I seriously think you get errors in exctags and 0.7 doesn't consume them
if you go and get stacks (pstack on opensolaris) from exctags stuck processes, they will all be in writing state and none in reading, but none of them move, since they wait.
Wait in kernel for someone to consume their stderr ... which was fixed after 0.7 ...

if your stacks of exctags show something different, then we shall go ahead with this bug, otherwise I seriously think you hit what Trond fixed in above changeset and this bug should be closed.

On 2009-08-05 23:09:28 +0000, Harshit wrote:

(In reply to comment # 13)

mhmm

I don't want to interrupt your party guys , but I think we solved this issue in
changeset 750 bdab6a143036

Guntas, can you try out a trunk build ?
(follow http://opensolaris.org/os/project/opengrok/manual/builddescription/
then e.g. $ ant package to get a package of trunk build, or just use stuff
from dist/ after the clean build)

Thanks Lubos and Trond.

I built opengrok from trunk and ran the indexer. But I am seeing that the exuberant-ctags process started by OpenGrok indexer gets stuck up on one of the files. (and it is been like this for the last 12 hrs. or so)

This is the current state of the process:

pstack 9440
9440: /usr/local/bin/exuberant-ctags --c-kinds=+l --java-kinds=+l --sql-kind
fef70eb7 _read (808d750, 808d640, fef13490) + 7
0806f5fd iFileGetc () + 3d

List of files opened by this process

pfiles 9440
9440: /usr/local/bin/exuberant-ctags --c-kinds=+l --java-kinds=+l --sql-kind
Current rlimit: 65536 file descriptors
0: S_IFIFO mode:0000 dev:278,0 ino:6219858 uid:23169 gid:10 size:0
O_RDWR
1: S_IFIFO mode:0000 dev:278,0 ino:6219859 uid:23169 gid:10 size:0
O_RDWR
2: S_IFIFO mode:0000 dev:278,0 ino:6219860 uid:23169 gid:10 size:0
O_RDWR
3: S_IFREG mode:0600 dev:274,2 ino:39192577 uid:23169 gid:10 size:0
O_RDWR|O_CREAT|O_EXCL
/tmp/tags.vybaCs
4: S_IFREG mode:0664 dev:284,17925 ino:17423762 uid:23169 gid:23169 size:383920
O_RDONLY
/the/path/to/file/my_sql_file.sql

Any idea why exuberant-ctags is giving me this problem. The version of exuberant-ctags that we are using is 5.5.4

Thanks.

On 2009-08-05 23:21:57 +0000, Lubos Kosco wrote:

umh, giving that exctags are currently on 5.8
and I never tested with less than 5.6 ...

Can you try to fetch, build latest exctags and rerun please ?
( http://ctags.sourceforge.net/ )

from http://ctags.sourceforge.net/news.html I see a fix was done in 5.6 , but on the other hand our sql analyzer could have problems too, bringing kah to the party :)

On 2009-08-06 21:04:56 +0000, Harshit wrote:

(In reply to comment # 15)

umh, giving that exctags are currently on 5.8
and I never tested with less than 5.6 ...

Can you try to fetch, build latest exctags and rerun please ?
( http://ctags.sourceforge.net/ )

Sure.. but I had a tough time last when I tried building exctags from source. I wanted to use the latest exctags but lack of proper documentation is what made me to use the one which was there in my Solaris box (5.5.4).

If you have any pointers on building them, please pass on.

Thanks for all the help.

On 2009-08-07 04:01:50 +0000, Lubos Kosco wrote:

$ uname -a
SunOS helius 5.10 Generic_141414-07 sun4u sparc SUNW,Sun-Fire-V240

fully installed (whole package cluster)

$ /usr/sfw/bin/wget http://downloads.sourceforge.net/project/ctags/ctags/5.8/ctags-5.8.tar.gz?use_mirror=puzzle

note that the zip file seems to be broken

$ /usr/sfw/bin/gtar xzvf ctags-5.8.tar.gz

$ cd ctags-5.8 ; ./configure

it found and started using /usr/sfw/bin/gcc

$ make

and local binaries ./ctags are there :)
refer to INSTALL for more info

for completeness, my path:
PATH=/bin:/sbin:/usr/bin/:/usr/sfw/bin:/usr/sbin:/usr/ucb/:/usr/ccs/bin/:/usr/openwin/bin:/usr/java/bin

On 2009-08-07 04:24:16 +0000, Trond Norbye wrote:

Why do you have /usr/ucb in your path???? :-O

On 2009-08-07 04:52:06 +0000, Harshit wrote:

(In reply to comment # 17)
Thanks Lubos!

I will give it a shot on Monday. Will update on how it goes.

Thanks.

On 2009-08-07 05:46:49 +0000, Lubos Kosco wrote:

(In reply to comment # 18)

Why do you have /usr/ucb in your path???? :-O

:)

ok, correction, my path:

PATH=/opt/mercurial:/bin:/sbin:/usr/bin:/usr/sbin:/usr/bin:/opt/sfw/bin:/usr/sfw/bin:/usr/ccs/bin/:/usr/openwin/bin

;)

On 2009-08-12 03:03:28 +0000, Harshit wrote:

(In reply to comment # 19)

(In reply to comment # 17)
Thanks Lubos!

I will give it a shot on Monday. Will update on how it goes.

Thanks.

Lubos/Trond,

All is normal now.. We are seeing the processes getting terminated properly.
I didn't even upgrade the exuberant-ctags nor did I change any configuration!

Closing.

Thanks for all the help.

If a directory name is changed, all links to its previous versions and diffs are broken.

Obfuscated example:
Lets have a component foo-1.0 in a mercurial repo and a file foo-1.0/README.
As a part of upgrade process to 1.1, user does 'hg rename foo-1.0 foo-1.1'.

Viewing contents of foo-1.1/README works fine, and list of changes for the file looks sane too.

But all links to README revisions before the rename are broken,
https://example.com/source/xref/foo-1.1/README?r=1423%3A65a016eaa866 has just the following text:
README revision 1423:65a016eaa866
and a "1" denoting the first line of code. Hacking in URL doesn't help

Diffs between two selected revisions are broken in a similar way. https://example.com/source/diff/foo-1.1/README?r2=foo-1.1%2FREADME%40763%3A45da4d38492e&r1=foo-1.1%2FREADME%40745%3A09fd85317532 shows:
No differences found!
But in this case hacking helps, fixing version numbers in the link shows the correct page:
https://example.com/source/diff/foo-1.1/README?r2=foo-1.0%2FREADME%40763%3A45da4d38492e&r1=foo-1.0%2FREADME%40745%3A09fd8531753

Unable to parse your query:
You might try to enclose your search term in quotes, escape special characters with , or read the Help on the query language. Error message from parser:
Cannot parse '/apps/abc/xyz/': Lexical error at line 1, column 41. Encountered: after : ""

The File Path search fails for all cases, so does the 'Only in this Directory' search in xref fail. The only way to get it working is escaping each forward slash by backward slash.

So if file path is like '/apps/abc/xyz/', it works fine - but it is not the way users would prefer to do it.

status CLOSED severity minor in component analyzer for 0.8
Reported in version unspecified on platform ANY/Generic
Assigned to: Trond Norbye

On 2009-08-04 05:08:40 +0000, Lubos Kosco wrote:

Aug 3, 2009 6:55:31 AM org.opensolaris.opengrok.index.IndexDatabase indexDown
WARNING: Failed to add file /tank/opengrok-sources/n1aa-HEAD/tools/jdk142_04_i386/usr/j2se/jre/lib/i386/libjmutil.so
java.lang.IndexOutOfBoundsException
at java.nio.Buffer.checkIndex(Buffer.java:438)
at java.nio.DirectByteBuffer.get(DirectByteBuffer.java:209)
at org.opensolaris.opengrok.analysis.executables.ELFAnalyzer.getName(ELFAnalyzer.java:166)
at org.opensolaris.opengrok.analysis.executables.ELFAnalyzer.parseELF(ELFAnalyzer.java:111)
at org.opensolaris.opengrok.analysis.executables.ELFAnalyzer.analyze(ELFAnalyzer.java:79)
at org.opensolaris.opengrok.analysis.AnalyzerGuru.getDocument(AnalyzerGuru.java:246)
at org.opensolaris.opengrok.index.IndexDatabase.addFile(IndexDatabase.java:458)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase.java:578)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase.java:561)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase.java:561)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase.java:561)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase.java:561)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase.java:561)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase.java:561)
at org.opensolaris.opengrok.index.IndexDatabase.indexDown(IndexDatabase.java:561)
at org.opensolaris.opengrok.index.IndexDatabase.update(IndexDatabase.java:251)
at org.opensolaris.opengrok.index.IndexDatabase$1.run(IndexDatabase.java:139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:417)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:269)
at java.util.concurrent.FutureTask.run(FutureTask.java:123)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:650)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:675)
at java.lang.Thread.run(Thread.java:595)

On 2009-08-05 06:04:57 +0000, Trond Norbye wrote:

The specified file doesn't contain a string section in the ELF file. I fixed the bug by skipping ELF files without sections or empty string section :-)

Fixed in:
changeset: 804:d9a0f44786b4

On 2010-04-18 12:35:25 +0000, Knut Anders Hatlen wrote:

*** Bug 6229 has been marked as a duplicate of this bug. ***

From discuss mailing list (Knut):

OpenGrok's analyzers read the entire file into a byte buffer before they
start analyzing it, so they may require a lot of memory if some of the
files are big. That is, if individual files are big. The total size of
the code base shouldn't have much impact on the memory requirements, as
far as I know.

Additionally, the analyzers keep the potentially very big byte buffers
around (one per analyzer instance, cached in a ThreadLocal field) to
avoid allocating new ones later, so the buffer space is never freed
after reading a big file.

I have been planning to rewrite the analyzers to make them less
memory-hungry, but I haven't got around to it yet.

Jens/Knut:

Yepp, perhaps just using soft/weak-refs to the buffer or cached
Analyzers at all, so that - at least in theory - on memory pressure
they get automatically purged/gced may fix it with little effort?

Yes, that might help reclaiming the space. We would still see spikes in
memory usage when processing big files, though. I was hoping we could
solve the latter (or both) by passing file contents to the analyzers as
streams rather than byte arrays.

SSIA: error message and stack trace to follow:

There was an error!

Cannot parse '/foo.xml': Lexical error at line 1, column x. Encountered: after : "/foo.xml"

org.apache.lucene.queryparser.classic.ParseException: Cannot parse '/foo.xml': Lexical error at line 1, column x. Encountered: after : "/foo.xml"
at org.apache.lucene.queryparser.classic.QueryParserBase.parse(QueryParserBase.java:131)
at org.opensolaris.opengrok.search.QueryBuilder.buildQuery(QueryBuilder.java:285)
at org.opensolaris.opengrok.search.QueryBuilder.build(QueryBuilder.java:200)
at org.opensolaris.opengrok.index.IndexDatabase.getDefinitions(IndexDatabase.java:1110)
at org.apache.jsp.list_jsp._jspService(list_jsp.java:640)
at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:70)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:388)
at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:313)
at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:260)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:602)
at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:722)
Caused by: org.apache.lucene.queryparser.classic.TokenMgrError: Lexical error at line 1, column 35. Encountered: after : "/foo.xml"
at org.apache.lucene.queryparser.classic.QueryParserTokenManager.getNextToken(QueryParserTokenManager.java:1133)
at org.apache.lucene.queryparser.classic.QueryParser.jj_scan_token(QueryParser.java:601)
at org.apache.lucene.queryparser.classic.QueryParser.jj_3R_2(QueryParser.java:483)
at org.apache.lucene.queryparser.classic.QueryParser.jj_3_1(QueryParser.java:490)
at org.apache.lucene.queryparser.classic.QueryParser.jj_2_1(QueryParser.java:476)
at org.apache.lucene.queryparser.classic.QueryParser.Clause(QueryParser.java:227)
at org.apache.lucene.queryparser.classic.QueryParser.Query(QueryParser.java:213)
at org.apache.lucene.queryparser.classic.QueryParser.TopLevelQuery(QueryParser.java:171)
at org.apache.lucene.queryparser.classic.QueryParserBase.parse(QueryParserBase.java:121)
... 22 more

code.metager.de's OpenGrok implements the ability to search by "Programming language". What they probably mean is searching by the analyser.

Not sure who coded it up or whether the source code is available, but this feature might be useful for other OpenGrok installations, too.

There doesn't seem to be any command-line option to turn off historycache. The most relevant documentation says this:

-H
        Generate history cache for all repositories
-h /path/to/repository
        just generate history cache for the specified repos (absolute path from source root)
…
-k /path/to/repository
        Kill the history cache for the given repository and exit. Use '*' to delete the cache for all repositories.

The TMPDIR on OpenBSD is /tmp by default, and my /tmp is on a separate logical disc (e.g. sd4, which is a softraid volume).

I've noticed that during OpenGrok indexing, /tmp gets a lot of traffic; as much as 50% of disc traffic are some writes to /tmp. However, systat iostat 1 never reports any reads for the disc, only the writes.

If I start OpenGrok with TMPDIR set to an mfs partition, then the writes to /tmp stop.

I'm not conclusive of what writes those are, but it might be related to hsperfdata_opengrok, or, probably more likely, it seems like a lot of files might be created and removed directly from TMPDIR, because /tmp changes its modification time all the time.

Why OpenGrok causes so many writes for TMPDIR?