Code Monkey home page Code Monkey logo

dxr's Introduction

DXR

DXR is a code search and navigation tool aimed at making sense of large projects like Firefox. It supports full-text and regex searches as well as structural queries like "Find all the callers of this function." Behind the scenes, it uses trigram indices, elasticsearch, and static analysis data collected by instrumented compilers to make searches faster and more accurate than is possible with simple tools like grep. DXR also exposes a plugin API through which understanding of more languages can be added.

image

dxr's People

Contributors

abbeyj avatar alberskib avatar anmoljagetia avatar brucestephens avatar clementdagneau avatar ehsan avatar erikrose avatar evmar avatar garnacho avatar gicmo avatar gvenkat avatar jamonation avatar jbradberry avatar jcranmer avatar jlmarr10824 avatar jonasfj avatar kleintom avatar lonnen avatar markoschuetz avatar mook avatar mvcisback avatar nave-mvn avatar nibrahim avatar nickolay avatar nimak avatar nrc avatar pelmers avatar rhelmer avatar rricard avatar vedadux avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

dxr's Issues

search.cgi sqlite3.OperationalError

(Using a883d2 from master)

After building a large in-house project using clang, I get the following when doing the following search:

callers:MyClassName::OverloadedMemberFunction

(apologies that the formatting doesn't look as good here as it does on the web browser)

<class 'sqlite3.OperationalError'> Python 2.6.6: /usr/bin/python
Tue Apr 17 14:51:18 2012
A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred.

/var/www/html/search.cgi in ()
460 print '

'
461 kwargs = dict((k,form[k]) for k in optargs if k in form)
462 dispatch(form[param], *kwargs)
463 break
464 else:
dispatch = , form = {'callers': 'MyClassName::OverloadedMemberFunction', 'funcid': '15199', 'tree': 'exelerate-trunk'}, param = 'callers', kwargs = {'funcid': '15199'}
/var/www/html/search.cgi in processCallers(caller='MyClassName::OverloadedMemberFunction', path=None, funcid='15199')
364 "UNION SELECT functions.
FROM functions LEFT JOIN callers " +
365 "ON (callers.callerid = functions.funcid) LEFT JOIN targets USING " +
366 "(targetid) WHERE targets.funcid=?", (funcid, funcid)):
367 if not path or re.search(path, info['file_path']):
368 print '

%s

' % info['fqualname']
funcid = '15199'
<class 'sqlite3.OperationalError'>: SELECTs to the left and right of UNION do not have the same number of result columns
args = ('SELECTs to the left and right of UNION do not have the same number of result columns',)
message = 'SELECTs to the left and right of UNION do not have the same number of result columns'

Npm errors when running make on Centos7

This is from the es.zip snapshot downloaded yesterday. I'll upload the logs shortly.

npm install
npm WARN package.json [email protected] No repository field.

> [email protected] preinstall /scratch/dxr-es-based/dxr-es
> lockdown

NPM Lockdown is here to check your dependencies!  Never fear!
npm WARN package.json [email protected] No repository field.

> [email protected] preinstall /scratch/dxr-es-based/dxr-es
> lockdown

npm ERR! Linux 3.10.0-123.el7.x86_64
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "install"
npm ERR! node v0.12.2
npm ERR! npm  v2.7.4
npm ERR! code ETARGET

npm ERR! notarget No compatible version found: phantomjs@'>=1.9.2-6 <1.10.0'
npm ERR! notarget Valid install targets:
npm ERR! notarget ["1.9.7-1"]
npm ERR! notarget 
npm ERR! notarget This is most likely not a problem with npm itself.
npm ERR! notarget In most cases you or one of your dependencies are requesting
npm ERR! notarget a package version that doesn't exist.
npm ERR! notarget 
npm ERR! notarget It was specified as a dependency of 'uncss'
npm ERR! notarget 

npm ERR! Please include the following file with any support request:
npm ERR!     /scratch/dxr-es-based/dxr-es/npm-debug.log

npm ERR! Linux 3.10.0-123.el7.x86_64
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "install"
npm ERR! node v0.12.2
npm ERR! npm  v2.7.4
npm ERR! code ELIFECYCLE
npm ERR! [email protected] preinstall: `lockdown`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] preinstall script 'lockdown'.
npm ERR! This is most likely a problem with the DXR package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     lockdown
npm ERR! You can get their info via:
npm ERR!     npm owner ls DXR
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /scratch/dxr-es-based/dxr-es/npm-debug.log
make: *** [node] Error 1

And then

npm owner ls DXR
npm ERR! owner ls Couldn't get owner data DXR
npm ERR! Linux 3.10.0-123.el7.x86_64
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "owner" "ls" "DXR"
npm ERR! node v0.12.2
npm ERR! npm  v2.7.4
npm ERR! code E404

npm ERR! 404 Not Found: DXR
npm ERR! 404 
npm ERR! 404 'DXR' is not in the npm registry.
npm ERR! 404 You should bug the author to publish it (or use the name yourself!)
npm ERR! 404 
npm ERR! 404 Note that you can also install from a
npm ERR! 404 tarball, folder, http url, or git url.

npm ERR! Please include the following file with any support request:
npm ERR!     /scratch/dxr-es-based/dxr-es/npm-debug.log

Make html-ified file extensions configurable in dxr.config

xref-tools/cxx-clang/indexer.py has a list of file extensions that get "HTML-ified". While the current distribution has several common extensions, there will always be some place out in the wild that has yet another better internal naming convention...that does not conform with what is in there.

This actually caused DXR to not properly index and display files on my initial installation.

The list of file extensions should be configurable inside dxr.config (or somewhere).

readme lacks a definition

Coming from https://blog.mozilla.org/webdev/2013/06/13/dxr-digests-the-firefox-codebase/ I tested the website, I read the wiki page and the readme, and I still didn't know what DXR actually is.
You may want to copy this description to the top of the readme: «DXR is designed to generated a source code index for one more trees offline, preferably on a dedicated build server, the generated index is then uploaded to a webserver for hosting» from https://github.com/mozilla/dxr/blob/master/docs/deployment.mkd
After skimming configuration.mkd and architecture.mkd I still don't know what sort of repositories it works with, so I go away without an answer to my question "could this be used with our git (gerrit+gitblit) repo" and I wasted 15 min of my time.

Get rid of magical namespace plugin magic

It's confusing to have two different ways to register plugins, especially when one of them is magical and auto-detects names. Plus, I ran into an issue recently where the config_schema property was not added to the from_namespace method of registering a plugin.

I think we should kill this magic with fire and require all plugins to explicitly create a Plugin instance.

The python plugin is broken

Hello guys,

It seems the 348026a has broken the python plugin because it can no longer handle function calls, e.g. an indexing of the following example:

import collections
Person = collections.namedtuple('Person', 'name age gender')

produces next output:

Traceback (most recent call last):
  File "/home/vagrant/venv/bin/dxr", line 9, in <module>
    load_entry_point('dxr==2.0', 'console_scripts', 'dxr')()
  File "/home/vagrant/dxr/dxr/cli/__init__.py", line 29, in main
    return dxr.invoke(ctx)
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/click/core.py", line 991, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/click/core.py", line 837, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/vagrant/venv/local/lib/python2.7/site-packages/click/core.py", line 464, in invoke
    return callback(*args, **kwargs)
  File "/home/vagrant/dxr/dxr/cli/index.py", line 26, in index
    index_and_deploy_tree(tree, verbose=verbose)
  File "/home/vagrant/dxr/dxr/build.py", line 64, in index_and_deploy_tree
    index_name = index_tree(tree, es, verbose=verbose)
  File "/home/vagrant/dxr/dxr/build.py", line 275, in index_tree
    index_files(tree, tree_indexers, index, pool, es)
  File "/home/vagrant/dxr/dxr/build.py", line 628, in index_files
    swallow_exc=False)
  File "/home/vagrant/dxr/dxr/build.py", line 581, in index_chunk
    index_file(tree, tree_indexers, path, es, index)
  File "/home/vagrant/dxr/dxr/build.py", line 482, in index_file
    refses.append(file_to_index.refs())
  File "/home/vagrant/dxr/dxr/plugins/python/indexers.py", line 216, in refs
    return self.visitor.refs
  File "/home/vagrant/dxr/dxr/plugins/python/indexers.py", line 195, in visitor
    self._visitor.visit(syntax_tree)
  File "/usr/lib/python2.7/ast.py", line 241, in visit
    return visitor(node)
  File "/usr/lib/python2.7/ast.py", line 249, in generic_visit
    self.visit(item)
  File "/usr/lib/python2.7/ast.py", line 241, in visit
    return visitor(node)
  File "/usr/lib/python2.7/ast.py", line 251, in generic_visit
    self.visit(value)
  File "/usr/lib/python2.7/ast.py", line 241, in visit
    return visitor(node)
  File "/home/vagrant/dxr/dxr/plugins/python/indexers.py", line 100, in visit_Call
    start, end = self.file_to_index.get_node_start_end(node)
  File "/home/vagrant/dxr/dxr/plugins/python/indexers.py", line 295, in get_node_start_end
    start, end = self.call_start_table[loc]
KeyError: (10, 9)

Make clang compiler path dxr.config variable (with make time default)

The clang plugin almost always needs to be run against the same version of the compiler it was built against. This probably should be specified at compile time, so that invoking dxr index will use the correct clang compiler.

Alternatively, dxr should just bundle the appropriate version of clang that the plugin is designed for.

Odd formatting in deployment.rst

I see a lot of odd formatting in the file dxr/docs/source/deployment.rst.

For example:
DXR is designed to generate a :term:source code index for one or more source trees offline, preferably on a dedicated build server.

I can't figure out if :term:source code index is supposed to format the term source code index in a specific way. I also don't understand why it's linked. I can't find references to :term: syntax in Github Flavored Markdown.

Issues compiling firefox in docker

The makefile says "Presently, nothing outside the source checkout will be preserved on exit." and that's my experience, but the docs say to not put source to be compiled in ~/dxr - is there another way to do this?

Also, a new m-c install requires a ./mach bootstrap, which asks for the root password (in lieu of sudo) to install what it needs - the docker password exists in the config files, but not in the docs.

I can take a shot at updating the docs (unless you'd rather), but just wanted to check on how this should actually be done. Thanks!

Permanent i.e. explicit-revision links

I would like very much the ability to get permanent DXR links, i.e. DXR links to a specific revision of the source code.

I need that to be able to paste DXR links in bugs. At the moment, I have to go back to hg.mozilla.org for that.

Note: this was raised in the "Poll: What do you need in MXR/DXR?" thread on dev-platform, and +1'd by someone else.

Apologies if this feature already exists and I didn't find it.

check requirements file vs. virtualenv in dxr-serve.py

The conversation was in pr #314. Relevant bits:

@willkg said:

I had some problems switching branches and dealing with requirements in the vagrant environment. With fjord, we create a virtual environment and then activate it in .bashrc when you log in. Here, it looks like everything is installed at the system level, so when I go to pick up pyelasticsearch which is a new requirement in this or a related branch, it gets a little fussy.

When you switch branches are you rebuilding your vagrant vm? Is there a good way for dealing with requirements changing between branches?

Then @erikrose thoughtfully replied:

I like the venv idea! Right now, you have to use sudo to get anywhere installing requirements. (And yes, you do have to either throw away your vagrant box or manually run sudo ./peep.py install -r requirements.txt each time you switch to a branch with different requirements. We haven't historically had many of these, so we haven't felt the pain to motivate us to fix it.)

The venv thing would be an easy, welcome improvement. I'm less sure how to make switching branches lower-touch.

Then @willkg offered in an offhanded fashion:

I'll look at adding a venv patch. In Fjord we do this fancy thing with manage.py where it compares what you've got installed with the requirements file and tells you if things are out of sync.

https://github.com/mozilla/fjord/blob/master/fjord/manage_utils.py#L21

We could kick that off in dxr-serve.py or somewhere else that devs are likely to use but isn't in use in prod.

Want me to create a bug for this to track it (and move the conversation to)?

To which @erikrose stated in a calm manner:

Actually, we could attach that warning of yours to dxr-serve, and all would be fine. That's meant to be used just for development; only a crazy person would deploy with it.

This issue covers adding a thing like fjord has to dxr and having it kick off in dxr-serve.

Vagrant pip fails and setup.py develop breaks environment

I don't quite have all the details but I wanted to leave some notes about ways I've broken the dev environment in a vagrant box:

  1. pip or peep.py fails with:

    vagrant@vagrant-ubuntu-trusty-64 ~/dxr (python-inheritance*) $ sudo ./peep.py install -r    requirements.txt
    Traceback (most recent call last):
     File "./peep.py", line 66, in <module>
       import pip
     File "/usr/lib/python2.7/dist-packages/pip/__init__.py", line 11, in <module>
       from pip.vcs import git, mercurial, subversion, bazaar  # noqa
     File "/usr/lib/python2.7/dist-packages/pip/vcs/mercurial.py", line 9, in <module>
       from pip.download import path_to_url
     File "/usr/lib/python2.7/dist-packages/pip/download.py", line 25, in <module>
       from requests.compat import IncompleteRead
    ImportError: cannot import name IncompleteRead
    

    (Note that even when this isn't breaking I can't install libraries without sudo, which is a bit weird.)

    sudo easy_install -U pip will fix this.

  2. After re-running sudo python setup.py develop to reset the entry points after modifying them, attempting to run the tests fails:

    vagrant@vagrant-ubuntu-trusty-64 ~/dxr (python-inheritance*) $ nosetests --nologcapture dxr/plugins/python/tests
    E
    ======================================================================
    ERROR: Failure: ImportError (cannot import name ConnectionError)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
     File "/usr/local/lib/python2.7/dist-packages/nose/loader.py", line 414, in loadTestsFromName
       addr.filename, addr.module)
     File "/usr/local/lib/python2.7/dist-packages/nose/importer.py", line 47, in importFromPath
       return self.importFromDir(dir_path, fqname)
     File "/usr/local/lib/python2.7/dist-packages/nose/importer.py", line 94, in importFromDir
       mod = load_module(part_fqname, fh, filename, desc)
     File "/home/vagrant/dxr/dxr/plugins/python/__init__.py", line 5, in <module>
       from .indexers import mappings, TreeToIndex
     File "/home/vagrant/dxr/dxr/plugins/python/indexers.py", line 8, in <module>
       from dxr.build import file_contents, unignored
     File "/home/vagrant/dxr/dxr/build.py", line 21, in <module>
       from pyelasticsearch import (ElasticSearch, ElasticHttpNotFoundError,
     File "/usr/local/lib/python2.7/dist-packages/pyelasticsearch/__init__.py", line 3, in <module>
       from pyelasticsearch.client import ElasticSearch
     File "/usr/local/lib/python2.7/dist-packages/pyelasticsearch/client.py", line 18, in <module>
       from pyelasticsearch.exceptions import (ElasticHttpError,
     File "/usr/local/lib/python2.7/dist-packages/pyelasticsearch/exceptions.py", line 1, in <module>
       from urllib3.exceptions import TimeoutError as Timeout, ConnectionError
    ImportError: cannot import name ConnectionError
    
    ----------------------------------------------------------------------
    Ran 1 test in 0.002s
    
    FAILED (errors=1)
    

Source files with TABs don't display very well

By the looks of it there's no TAB expansion, so the TABs end up in HTML <code> blobs, and the TABs are (typically) displayed as single spaces.

I guess this might be non-trivial; I was imagining just expanding the TABs when producing the HTML, but if the column information is taken from clang then that's probably not going to work.

clang plugin has narrow view of build environment

The clang indexer plugin integrates well and easily in certain types of build environments--specifically, those that do not specify compilers in the CC and CXX environment variable and which use the executable name 'clang'. Sadly, in the real world, not all build environments behave that way.

To make it easier to integrate with the real world, it would be best if the plugin ALSO exported the following information:

  1. the same information that's in CC and CXX, but in DXR-specific environment variables that aren't at risk of being clobbered
  2. the flags that the plugin adds to the clang command line in their own environment variable so that a build environment/system can use that to integrate into its commands

Context menu doesn't work when clicking on folder name

I have a dxr server running (through Vagrant) and I am not seeing the context menu when I click on a folder name in the search result page (I am referring to the context menu that says "Browse folder contents", "Limit search to folder" ...).

When I click on the folder name I see the following message in the server log
[07/Aug/2014 14:32:42] "GET /<project>/dxr/static/templates//context_menu.html?s=1407421962679 HTTP/1.1" 404 -

The beginning of the path looks a bit dubious. All the other files are served directly from the static folder.

Is anyone else seeing the same problem? The context menu works fine on dxr.mozilla.org so I am probably doing something wrong. My git repository is up to date.

shell scripts in clang plugin contain botched paths

In the 'testing' branch, the shell scripts in the clang plugin contain botched paths. The refer to the path used in the master branch rather than that used in the testing branch.

Since they are wrong, and according to "blame" they've been wrong for 5 months, I'm wondering if maybe they're not even used any longer. If they aren't used, then what is the new process for setting up the environment properly in order to generate all the necessary data?

[clang plugin] members and callers query doesn't seem to work with virtual methods

If I create the two traditional classes Base and Derived, with a virtual void method Base::function, then asking for the methods of Base or Derived doesn't show anything. The SQLite database looks like it does contain suitable information.

Additionally (and quite possibly for different reasons) if I have a function caller calling Base::function through a reference or pointer (so it might be calling Derived::function) then asking for the callers of Base::function just shows the file containing the calling function. By the looks of it the callers table contains a negative integer for the called method and the targets table maps back to something displayable, so I speculate that something's going wrong with that indirection.

clang callgraph incomplete (testing branch)

In a large project (which cannot be shared), the callgraph is not being generated properly. More specifically, the callgraph is incomplete, lacking data on calls to many functions. These are, indeed, called, as I can set a breakpoint in a debugger and examine a stack trace, or I can verify based on personal knowledge.

In attempting to debug this enough to create a small publishable test case, I cannot figure out where the plugin generates its data or, more importantly, which bit of code DELETES all the data generated by the plugin.

Finally, while I realize that the goal is to make dxr such that it can run completely unattended as a cron job, that goal appears to be achieved at the expense of the uninitiated being able to inject themselves into the process and see what's going on and, potentially, debug/fix. Maybe something like a "--dry-run" option wherein the script would simply tell you everything it would do would facilitate that? I'm not sure since I don't know all the steps involved.

Import error when running dxr through mod_wsgi | 404 when trying to access files with built-in web server

I did a fresh install of dxr today on a Fedora 20 machine, it worked smoothly but I encountered a few issues:

Trying to run dxr through mod_wsgi results in a "ImportError: No module named site" on error_log, the vhost:

WSGIPythonHome /home/gnomeweb/dxr_venv

<VirtualHost *:80>
    ServerName dxr-test.gnome.org

    CustomLog /var/log/httpd/dxr-test.gnome.org/access_log combined
    ErrorLog /var/log/httpd/dxr-test.gnome.org/error_log

    # Serve static resources, like CSS and images, with plain Apache:
    Alias /static/ /home/gnomeweb/dxr/dxr/static/

    # We used to make special efforts to also serve the static pages of
    # HTML-formatted source code from the tree via plain Apache, but that
    # tangle of RewriteRules saved us only about 20ms per request. You can do
    # it if you're on a woefully underpowered machine, but I'm not maintaining
    # it.

    # Tell this instance of DXR where its target folder is:
    SetEnv DXR_FOLDER /home/gnomeweb/dxr_output/

    SetEnv PYTHONHOME /home/gnomeweb/dxr_venv

    WSGIScriptAlias / /home/gnomeweb/dxr/dxr/wsgi.py
</VirtualHost>

<Directory /home/gnomeweb/dxr/dxr>
   Require all granted
</Directory>

Running dxr from dxr-serve.py works just fine but another issue arises.

All the files are indexed properly and searching for code through the search bar works as expected and every single file content is properly showed when a string matches the search but I can seem to access the files content when clicking on them, for example:

http://dxr-test.gnome.org/git-test/source/gtk+ works OK
http://dxr-test.gnome.org/git-test/source/gtk+/AUTHORS returns a 404

The directories are accessible by the web server and I can't seem to find any permission error on the setup, my dxr_output folder only shows directories but no files at all, an excerpt of a tree run:

|-- build
|   `-- win32
|       |-- crypt
|       |-- vs10
|       |-- vs11
|       |-- vs12
|       `-- vs9
|-- demos
|   |-- gtk-demo
|   |   `-- data
|   |       |-- 16x16
|   |       |-- 22x22
|   |       |-- 24x24
...

Thanks for all the help in advance!

Errors running tests on es branch

I'm running into some issues running the tests via make test on the es branch in Vagrant:

  • With no modifications, I get this a bunch:

    ERROR
    Warning: $jobs is not used in build_command for 'clang'
    Build command for 'code' failed, exited non-zero.
    Log follows:
    | /bin/sh: 1: clang++: not found
    

    Which leads to a bunch of BuildErrors when the actual testcases are run.

  • If I add a symlink from clang++ to clang++-3.5, I get segfaults when attempting to build the code.

Any tips on how to debug this issue?

Wildcard support in filters

It would be great to be able to use wildcards in the filters, such as function-decl:* to list all function declarations

"no such group" error in buglink plugin

Rust, rustfmt, and servo have all thrown the following error (eg https://jenkins-dxr.mozilla.org/job/servo/9/console):

15:21:29 A worker failed while indexing /builds/dxr-build-env/src/servo/servo/tests/wpt/css-tests/css21_dev/html4/border-bottom-applies-to-009.htm:
15:21:29 Traceback (most recent call last):
15:21:29 File "/builds/dxr-build-env/dxr/dxr/build.py", line 572, in index_chunk
15:21:29 index_file(tree, tree_indexers, path, es, index)
15:21:29 File "/builds/dxr-build-env/dxr/dxr/build.py", line 542, in index_file
15:21:29 for chunk in bulk_chunks(docs(), docs_per_chunk=300, bytes_per_chunk=10000):
15:21:29 File "/builds/dxr-build-env/venv/local/lib/python2.7/site-packages/pyelasticsearch/utils.py", line 31, in bulk_chunks
15:21:29 for action in actions:
15:21:29 File "/builds/dxr-build-env/dxr/dxr/build.py", line 514, in docs
15:21:29 chain.from_iterable(regionses)))):
15:21:29 File "/builds/dxr-build-env/dxr/dxr/lines.py", line 472, in finished_tags
15:21:29 key=nesting_order)
15:21:29 File "/builds/dxr-build-env/dxr/dxr/lines.py", line 338, in tag_boundaries
15:21:29 for start, end, data in tags:
15:21:29 File "/builds/dxr-build-env/dxr/dxr/plugins/buglink/init.py", line 15, in refs
15:21:29 bug = m.group(1)
15:21:29 IndexError: no such group

favicon.ico isn't being set

This traceback happened the first time I loaded the toy dxr-serve.py server in ~/dxr/tests/test_basic. Refreshing the page didn't cause the problem to be reproduced.

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1701, in __call__
    return self.wsgi_app(environ, start_response)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1689, in wsgi_app
    response = self.make_response(self.handle_exception(e))
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1687, in wsgi_app
    response = self.full_dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1360, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1358, in full_dispatch_request
    rv = self.dispatch_request()
  File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1344, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/vagrant/dxr/dxr/app.py", line 192, in browse
    index=config['ES_ALIASES'][tree],
KeyError: u'favicon.ico'

Link to DXR project info in HTML output

I really like it when projects contain a link to the source code or a "getting involved" page in their output. This both lets people know they can scratch itches and also tells them how to do so. AFAICT dxr.mozilla.org doesn't have such an advertisement. I think we should put one in.

Undefined symbol clang::~PPCallbacks in libclang-index-plugin.so

When I try to make in the tests/test_basic directory, the compilation of the hello world program fails with this error.

I am building on a local machine, using the virtualenv. I updated the makefiles in the dxr/plugins/clang/ directory to point to the clang header files directories. I also tried a few things, like explicitly linking libclang-index-plugin.so with -lclang but I continue to see this error.

Please note, the symbol name above in the Title is the demangled name of the symbol.

Any help would be appreciated. Thanks!

build.py does not index files generated by the build system

build.py first indexes files (to populate trigraph tables, and build the files table), and then invokes the build system. If the build system generates any code, everything in those generated files is lost because there is no file_id to tie them to. (this is why we cannot search any of the binding code on dxr.mozilla.org)

Unable to clone repository on Windows due to invalid folder name

I am getting the following error when I clone the repository on my Windows machine:

G:\New folder>git clone --recursive https://github.com/mozilla/dxr.git
Cloning into 'dxr'...
remote: Counting objects: 13230, done.
remote: Compressing objects: 100% (137/137), done.
remote: Total 13230 (delta 111), reused 27 (delta 19)
Receiving objects: 100% (13230/13230), 8.37 MiB | 1.67 MiB/s, done.
Resolving deltas: 100% (5003/5003), done.
Checking connectivity... done.
fatal: cannot create directory at 'tests/test_markup/code/': Invalid argument
warning: Clone succeeded, but checkout failed.
You can inspect what was checked out with 'git status'
and retry the checkout with 'git checkout -f HEAD'

I think I am seeing this error message because I am working on Windows and angle brackets are not valid characters for folder names. I am guessing it is ok on Linux/MAC.

So far I have been downloading a zip file of the project but it is a bit tedious because it doesn't include submodules. Could the folder be renamed?

SVN support

I would like to use DXR in a subversion project.

Any ideas on how to start and add this?
I would be willing to implement this myself.

Npm error when running make

I am seeing the following error when running make in the project root folder:

npm install
npm WARN package.json [email protected] No repository field.

> [email protected] preinstall /home/vagrant/dxr
> lockdown

/home/vagrant/dxr/node_modules/.bin/lockdown: 1: /home/vagrant/dxr/node_modules/.bin/lockdown: ../lockdown/lockdown.js: not found
npm ERR! weird error 127
npm WARN This failure might be due to the use of legacy binary "node"
npm WARN For further explanations, please read
/usr/share/doc/nodejs/README.Debian

npm ERR! not ok code 0

I tried to investigate but I didn't really go anywhere. I am new to npm and any help would be appreciated.

My project is at sha 6ea7641. I am running dxr on a Windows machine through Vagrant. I've had problems in the past with that setup (they are now fixed) so it could be the source of the problem.

Should I be posting bug reports on GitHub or Bugzilla? They seem to be both active at the moment.

Highlight .idl files as c++

I imagine that this could be done with a similar hack to the parsing .h files as .cpp, Most of the time .idl looks pretty good as c++ (in fact, in many mozilla files, emacs is told to use c++-mode for .idl files).

[UI] provide information about number of lines

I really like how github's file browser shows info at the top about lines of code:

screen shot 2015-12-23 at 9 29 55 am

Granted, the user could always scroll to the bottom to find out, but it's not practical for HUGE files.

It would be cool for DXR to show similar stats for a file.

table 'fts' is missing

(Using a883d2 from master)

Performing a basic search for the text "boat" in my project, I get the following page returned:

<class 'sqlite3.OperationalError'> Python 2.6.6: /usr/bin/python
Tue Apr 17 14:57:44 2012
A problem occurred in a Python script. Here is the sequence of function calls leading up to the error, in the order they occurred.

/var/www/html/search.cgi in ()
460 print '

'
461 kwargs = dict((k,form[k]) for k in optargs if k in form)
462 dispatch(form[param], **kwargs)
463 break
464 else:
dispatch = , form = {'request_time': '1334692663956', 'string': 'boat', 'tree': 'exelerate-trunk'}, param = 'string', kwargs = {}
/var/www/html/search.cgi in processString(string='boat', path=None, ext=None, regexp=None)
190 printHeader = True
191
192 for row in queries.getFileMatches(conn, string):
193 if printHeader:
194 print '
Files
    '
    row undefined, global queries = <module 'dxr.queries' from '/home/ccleeland/localbuilds/dxr/dxr/queries.py'>, queries.getFileMatches = , global conn = <sqlite3.Connection object>, string = 'boat'
    /home/ccleeland/localbuilds/dxr/dxr/queries.py in getFileMatches(conn=<sqlite3.Connection object>, match_string='boat')
    6 def getFileMatches(conn, match_string):
    7 for row in conn.execute('SELECT (SELECT path from files where ID = fts.rowid), fts.basename ' +
    8 'FROM fts where fts.basename MATCH ?', ('"%s"' % match_string,)).fetchall():
    9 yield row
    10
    match_string = 'boat', ).fetchall undefined
    <class 'sqlite3.OperationalError'>: no such table: fts
    args = ('no such table: fts',)
    message = 'no such table: fts'

Index XPIDL

It would be rad if DXR could index XPIDL. MXR does this already.

The canonical code for parsing XPIDL files lives at https://hg.mozilla.org/mozilla-central/file/668f29cd71b3/xpcom/idl-parser/xpidl.py. See high-level usage at https://hg.mozilla.org/mozilla-central/file/668f29cd71b3/python/mozbuild/mozbuild/action/xpidl-process.py#l27.

Initially, basic support is all that is needed. But it would be really cool if DXR could perform interface inheritance and dump full API details for a given interface, including relationship graphs.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.