goto() import following about jedi HOT 59 CLOSED

I just noticed that completion also fails on names in intra-package imports.

from jedi.

1. Yeah, I haven't put much effort into the goto function... It may be buggy. There's only about forty tests. I will have a look at your issue.
2. Would you like to have the old get_definition() method back? That's possible. I removed it, because I didn't think someone would need it. Or, do you only want a method, that follows the imports, but not normal statements (a=1; b=a; c following c would still result at the b=a statement)? This would require an additional option for Jedi, which is certainly possible. What do you think?
3. intra-package imports? Can you give me an example? (the best example would be one out of the standard library)

Thank you for the bug report!

from jedi.

1. fixed: acd6488 -> git pull

from jedi.

1. That would be very nice. I used that feature for two purposes: Jumping to the definition of something, regardless of whether it is an instance, class name etc., omitting import statements, by testing the scopes for inheritance of the classes in evaluate.py. The other use case would be resolving the full dotted name for calling pydoc on it.
   I guess I could just call goto repeatedly on the imports. I'll just have to figure out my import problem (3).
2. This seems to be some sort of heisenbug on my side. It only occurs when I call goto from within vim when editing a file from the package that import refers to.
   Setup would be following:
   The current file is in the directory /base/foo. PYTHONPATH contains /base. I edit /base/foo/bar.py, which contains a statement 'from foo.baz.bam import Name'. If I call jedi#goto() or my own goto function with the cursor on Name, I get "The module you searched has not been found".
   If I run an external program, containing the same call to goto, from within vim with ':! somescript.py thisfile.py 20 15' (passing the correct parameters), it works.
   If I run goto on an import to a directory also in the /base dir, it works, too.
   If I setup a trivial test case, it works. If I strip down a nonworking package dir to the bare essentials, it fails. Really confusing. I guess it's got to be something with my vim environment.

from jedi.

Oh, and I noticed that the directory of the current file is added to sys.path when failing, I think it comes from within jedi.

from jedi.

Weird, my enumeration in the first comment was supposed to be 2. and 3., but github seems to have fixed that to 1. and 2.

from jedi.

alright, forget about it. I just deactivated all plugins but jedi, tested, reactivated them, it works. Maybe some stale .pyc files were interfering or whatever. Still, weird.

from jedi.

FYI, I hacked this piece of code to realize my desires. It seems to work for class names, method calls and assigned identifiers.
https://gist.github.com/3600539

from jedi.

Sorry, if I'm not sure if I understand. Is only (2) still relevant?
As for (2) get_definition is not gone. It's still available (functions.py). But: I removed it from the VIM functions, because the VIM interface was not good enough. get_definition is heavily used for testing (Most test cases use it).

The question is much more: What should we do with it? How should it look in VIM? Should it be working like the goto function now? Or do you want something different? Feel free to contribute to the VIM plugin!

For me Jedi is primarily an auto-completion library, not a VIM plugin. (It's just also a VIM plugin, because I use VIM). So my efforts are mostly directed to improving the library, not plugin/jedi.vim. But I really hope to see people extending the VIM plugin.

from jedi.

Because my comment didn't cover your last comment:
Is this also possible with evaluate.get_definition() or not?

from jedi.

whoops, simultaneous posts ;)
yes, this is only about (2) now (changed the issue title, too).
If you like, you could incorporate my resolution method into jedi#goto() as an optional method. You can probably find a cleaner way to realize it.
I'm gonna check out get_definitions, just a sec.

from jedi.

Alright, it seems that get_definitions() does at least what my functions do. I don't know why I didn't see that, because I tested it with get_definitions() yesterday. Maybe it was your latest patch. So I guess this issue has been resolved from the start ;)

from jedi.

Still, two things I have noticed.
What I said in the fifth comment, maybe it's relevant to you.
As you can see in the gist, I increment the column by one for goto(), this seems to be a bug.

from jedi.

First, You mean showing PyDoc for some statement?

Second, I think I know where this is coming from. I'm working on it, thanks.

from jedi.

What do you think in the following case (goto):

a = 3
b = a
c = b

I start goto on b (line 3). Where should the cursor land? On b? or on a (both line 2)? Because, if I follow it again it should hit a.

from jedi.

First: Not pydoc, I meant that when calling jedi#goto(), the current file's dir is prepended to sys.path, every time. Not a problem, but maybe unnecessary.

I don't really know what would be sensible in your example case. Do you want to be able to chain goto calls to reach the final assignment value?

from jedi.

Second, I have an idea, but I need some time ;-)

from jedi.

knock yourself out :) I'm very happy with get_definitions() as resolution to this issue.
Also, I cannot emphasise enough how amazing jedi is in combination with AutoComplPop and the preview window for function doc.

from jedi.

Btw: You don't need AutoComplPop, jedi-vim does that for you. (It's on by default, but you can turn it off: let g:jedi#popup_on_dot = 0).

from jedi.

The goto stuff is not finished yet. But there is a brand new feature on "K" which is the help key in vim. PyDoc is shown (with syntax highlighting, you can call it on anything, even operators). And get_definitions is back -> use d.

from jedi.

@davidhalter
not using AutoComplPop isn't an option for me ;)
Alright, my original problem seems to be back as of the last commit, and this time it doesn't disappear when I de- and reactivate my plugins. And this time, it seems even worse.
I think it might result of where and when what part of PYTHONPATH jedi utilizes.
Scenario:
I work in a dir ~/code/python, where I have two subdirs, myproject and mylib. mylib is also installed in /usr/lib/python. Both of them are present in PYTHONPATH and sys.path, setup by my vimrc when editing. When I have a statement

from mylib import something

then, when running :make from within vim, the version from my home is used, not the system one.
But when I use jedi#goto() or get_definition() on anything that leads to mylib, jedi finds the file in /usr.
When I run it on something that leads to a different file in the same dir, myproject, nothing is found.
This is similar to what I described in an earlier comment, but then, at least mylib was found in my home, and every other time when I deactivated all plugins temporarily, even that worked perfectly for some time.
And still, when I run an external file that calls get_definition() from the vim commandline using ":! somescript.py", the local myproject definition is found.
The file setting up PYTHONPATH et al. is executed before jedi, as listed in :scriptnames.
Do you have any idea what could be the reason for this?

from jedi.

@tek: By not using AutoComplPop I meant that Jedi by default automatically does what AutoComplPop does (but just for Python).
I have to look into your problem, but that's not possible for the next few days, probably until Monday. I'm pretty busy. If you discover something else let me know!

from jedi.

I know, but ACP provides additional features that I use.
Alright, if I can spare the time, maybe I'll dig through jedi code myself and see if I can pin the problem down.

from jedi.

So I found out why I have been experiencing the heisenbug, it's pretty embarassing…when testing jedi#goto() isolated from other plugins, I exported PYTHONPATH manually before starting vim, as my module-path plugin file wouldn't be loaded. After reactivating everything, I naturally didn't unset PYTHONPATH, and I have just confirmed that this is the cause of the failure to find definitions in my local code repo.
But the issue still remains somehow; as jedi is being loaded after my path setup and both :echo $PYTHONPATH and :python print sys.path contain the very directory I manually exported as described above and it Used to work™ at some point…I wonder if this can be fixed.

from jedi.

So, is :python print sys.path wrong? I don't get it.

However, a little comment on how jedi uses the sys.path:
It takes the default sys.path and removes the first entry (probably because that is the jedi directory, I don't know, it always worked for me, but actually I don't know if this is the best way...):
https://github.com/davidhalter/jedi/blob/master/builtin.py#L16
After loading jedi, sys.path is left untouched. This means that if you change it, jedi won't feel a thing...

Does this explain anything? I would be open for changes (about the whole sys.path thing. Because I really don't know what you mean by manually exporting. Maybe you could explain that. Otherwise you could just dump the builtin.module_find_path at any given point (and print it here) which might help to understand the problem.

from jedi.

Alright, looking at module_find_path helped me fix it. I set the path in two different files and I only checked one of them :-| The second one was being read after jedi. Pretty obvious actually.
Thanks for the input, I think this issue is done now ;)
Btw, by manually exporting I meant executing export PYTHONPATH=/my/project/basedir in the shell just before starting vim (as opposed to being set from within my vim config).

from jedi.

ah, all right! Was the lacking support for sys.path modifications inside of python files also a problem?

from jedi.

Don't know if I understand, do you mean the fact that sys.path is only read once? Given a case where sys.path is manipulated in a file that is being operated on using jedi?

from jedi.

Let us suppose we have a file:

import foo
sys.path.insert(0, '/path')  # path to second foo module
import foo as bar
# access foo and bar...

This is just one example, that jedi is currently not covering.

from jedi.

No, I don't have any cases like that. The only path manipulation I do is that of the vim environment.

from jedi.

All right. I just asked, because I'm interested how sophisticated sys.path modifications are in practice.

from jedi.

I would assume that they are being frowned upon heavily ;)

from jedi.

One more thing…
get_definition() fails when invoked on import statements, because the scopes variable then isn't being constructed as a set, which is necessary. Even if this is fixed, construction of Definition fails, because the imports aren't being followed. I managed to get it to work with this, but you'll probably want to implement it more elegantly.
https://gist.github.com/3680010
boosted this from goto() below.

from jedi.

Yeah, you're right. There are multiple issues with imports (also still with the completion and its context). I'm working on it. The goto function is currently being changed to a more "general" function, because I need it for renaming (which is the next and probably last big thing). However, don't expect get_definition to work on these imports in the next few days or weeks. I need clean long-term solutions. There are already too many hacks ;-)

from jedi.

Alright. What's 'renaming' about?

from jedi.

Refactoring. Renaming variables / functions / classes consistently.

from jedi.

Excellent! Any thoughts about vim integration? I'd been always annoyed with rope refactoring destroying my undo history…though I don't know if this is still an issue with 'undofile'. Still, I think a vim plugin wouldn't be too complex.

from jedi.

There is now a rename feature on <leader>r and a related name feature on <leader>n. The latter isn't important. But I really like your opinion on the former. I'm not shure how to handle import renames. Should I rename the file? Also, what to do with files that are not opened, but lie in the same directory and point to the same name? Do the renames there and save the file? Is that a good practice? Or should there be some additional user input to ACK the changes?

Regarding rope: The basic problem with rope is, that it handles your files. Jedi is a code analysis tool, it doesn't write any files. Rope is a refactoring library and therefore has much to do with files. I think the current implementation is pretty much as "vimy" as you can get. With undo/redo ability.

from jedi.

wow! Really amazing. I guess one possibility would be to just open all files and do the changes in vim. That way it wouldn't even be necessary to differentiate the two cases. As for import renaming, what would be the alternative to renaming the file? You could ask the user if it should be done.
I think the biggest problem could come from something going wrong and clobbering everything up. The sanest way to do it would probably be commiting all changes to source control before using the feature…you can never account for all eventualities (like broken code).

from jedi.

Thanks for the feedback! I'm now trying to implement something like that. By the way, do you use the jedi#use_tabs_not_buffers option?

from jedi.

No, I almost never use tabs. I especially can't see an advantage to use them for this purpose…a split would make sense to me.
One other issue: When renaming fails when leaving insert, temp_rename remains set and thus the next call to jedi#rename() tries it again. My suggestion is to call the function with a parameter from the autocmd and check for its existence instead of temp_rename.

from jedi.

How would you like to use your buffers? What about names, that occur in 10 buffers?
However, I set the jedi#use_tabs_not_buffers = 1 option as a default, because this option has a much better support as long as I'm using no buffers to work with. I guess you have to disable it now in your vimrc. The current renaming solution is just ugly for buffers... Save and open a new one. Please feel free to do modify it as you like.

The other issue: Why didn't I come up with this. The temp_rename hack is ugly anyways... Thanks!

from jedi.

Writing the buffer before editing another one would be necessary if you would edit with 'nohidden' set, which is highly unusual. Even if the target file was open and its buffer was hidden in modified state, :edit could open it without the bang. Nevertheless, I would save all buffers before doing this anyway.
By the way, as start_renaming is a parameter that shouldn't be set by the user, I'd recommend setting it by making the function parameters varargs (function jedi#rename(...)) and testing for the value of a:0, so you can keep mapping to jedi#rename(). Just a little syntactic sugar ;)

from jedi.

I'm not an experienced VIM script writer. Didn't think of varargs. Thanks for the hint!

Also, please change the stuff you don't like about buffers and send a pull request I'm really open to solutions.

from jedi.

I fixed a small error. But when testing, it never tried to open other files. Is this supposed to work yet?
I also took the liberty of making two other small changes – the a:0 parameter contains the number of arguments, not the first argument, and thus is guaranteed to be set. So there is no need to catch the error.

from jedi.

yes, opening other files should already work!
good changes!
why did you change to execute edit? however, the opened files seem to ignore the filetype. at least with tabs. any ideas?

from jedi.

The :execute was the error I fixed. Running :edit a:path would create a new file called 'a:path'.
How do you determine the files containing the related names? Do you search cwd recursively? When I call jedi#related_names(), I only get results from the current file. I'd have to get this to work to examine the filetype issue.

from jedi.

Oh and just FYI, the intended purpose of tabs in vim is to have multiple window layouts. I suspect you are using them to have your buffer names visible while editing. A more canonical way to have that feature would be to use a buffer explorer/switcher plugin. Just a hint, I don't want to say your way is invalid or something ;) but I had been doing the same before realizing that.

from jedi.

Yeah, I know what you mean. And I tried. But the buffer switchers are visually not as comfortable as tabs. I really didn't like the way they are working.
I even reported a bug once: fholgado/minibufexpl.vim#39 But after a while I switched back to tabs.

from jedi.

I see. I have used MBE for a while, but at some point I decided that I only needed to see the buffer list when switching. So now I use quickbuf (got a fork here), which opens a list at the bottom when invoked from a mapping. There you can cycle with jk, open with <cr> or the buffer number, delete buffers etc. I think its very convenient.

from jedi.

Maybe I'll try it once, but I really like my current solution (switch tabs with , ).

Awesomeness is coming! :-) As a last feature I implemented something that helps you if you execute classes / functions. It is definitely the biggest hack I ever did in my whole software life, but totally worth it. I really love it! Please try and leave feedback. Now I will just try to fix a few bugs, there will be no new features. There are just a few to-dos left until I reach the beta stage.

from jedi.

Very interesting feature! Did you think about adding parameter names to the completion items? Looks very nice.
Though it only works correctly when I use it below an emtpy line. If there is text above the line, it is oddly incorporated into the first delimiter string and the background color and underlining isn't displayed.
I'd imagine, it would be pretty nice to display this in the preview window, alongside the docstring.
Another thing I wanted to ask for some time: Any reason why you don't use ftplugin?

from jedi.

I added param names this morning. I hope it works. I didn't test it very well.

Can you send me a picture of the code that is not working? I haven't seen any highlighting related bugs.

Adding this to the preview window would be an option. Maybe I do it once. But I like it much more "as is".

Using ftplugin instead of plugin? That would of course be possible and probably better.

from jedi.

http://imgur.com/a/YwkXo Seems not to happen when I put the code into global scope.

from jedi.

haha, thank you! really helpful, I guess I just never tested if it also worked within python string literals. I'll try to fix it, hopefully that's possible :-)

from jedi.

It doesn't look like it has anything to to with the string. If I delete that line, so that the function def is the displaying line, the same corruption occurs.

from jedi.

Really strange, because for me your example works (even the one with a string). Can you tell me what VIM version you are using, which operating system and so on?

from jedi.

Found what's causing this – I override some syntax items in my vim config to be able to fold docstrings. There I had a region that matches all classes and methods. After adding jediFunction to its contains parameter, it worked.

from jedi.

great! (and please don't pull for the next 1-2 days, we're trying to make it work with pypi).

from jedi.