I think there is one bug, and one potential improvement:

1. When I flip through the list of symbols in a file, the yellow bar always appears one line above the relevant symbol. My guess is this is another 0 vs 1 based indexing problem.
2. For methods, it would be nice if the yellow bar covered the whole body of code, that is how it is done for e.g. Typescript. I guess this would require us to know where the method body ends, and then returning a Location that has a range that potentially spans multiple lines?

Libc.free(outbuf); return convert(Csize_t, 0)

get's chopped to
return convert(Csize_t, 0)

When I make some edits to an open file, then the show symbols command shows an empty list.

I think what is happening is that the didChange events invalidate the block information, but then the symbol command doesn't trigger a reparsing?

When I save the file and then open the show symbols command, I do see the full list.

struct Foo
    x::Bool
    Foo() = new(false)
end

results in

file: 'untitled:Untitled-1'
severity: 'Warning'
message: 'Constructor name does not match type name'
at: '3,5'
source: 'CSTParser.Diagnostics.Diagnostic'

It would be nice if the latex completion UI showed the actual Unicode character in the completion UI somehow. Not sure whether it should just be returned as the detail or documentation or some other field in the LS protocol.

Is the new linter always running? This is related to #60, if I make this setting false that bug goes away.

If one positions the mouse cursor just after the last character in a file (and that is not a line break), then our ls will receive a hover request that has a position that doesn't exist in the document and our ls crashes. So we need to cover this.

Add a codeaction to list a declared functions return points (through both explicit return statements and possible last evaluated expressions), may be useful to add a lint check for returns of Void not matching other return types.

Ideally we would not ever load any user code into the LS process. This makes it non-trivial to get at documentation, symbols, methods etc. from packages that user code might load. I'm not entirely sure how to best handle this, here are some rough ideas:

1. span a new julia process, that julia process loads the package, and returns all the necessary information to the LS process, say in JSON format. This could be done async, so maybe some of the symbols and info from the package would only show up after a while.
2. run our static analysis on the package that a user loads. This will probably miss some things...

In general it might also make sense to cache symbol/method/function/doc information about packages on file or something...

A constructor like so:

immutable Gridded{T}
...
    function (::Type{Gridded{T}}){T}(...)
        ...
    end
end

gets a warning:

severity: 'Warning'
message: 'Constructor name does not match type name'
at: '70,5'
source: 'CSTParser.Diagnostics.Diagnostic'

Presumably here:
https://github.com/JuliaEditorSupport/LanguageServer.jl/blob/4ae5fdcdeb9413deb9a2a7787b22c03e4362d93e/src/trav/lint.jl#L326

The biggest problem I think is still the on disc cache. I think at the end of the day we have a massive race condition there, as soon as someone is using two instances of vscode at the same time I think that can trigger, right? So my preference would still be that we do away with that, and instead build the cache anew in every instance of the LS and only in-memory.

With #36 it takes about 10 seconds on my system to extract all the information from base, and I assume it should be much faster to extract stuff from packages, so maybe we can just say the cache is rebuilt in-memory for every LS instance?

I guess one problem is how to move the doc info from the child process to the LS. Maybe we could serialize things, then encode them as text and just stream them via STDOUT to master? Or another idea would be to create a temporary file (with a temporary filename) that is used to transfer just one set of results back to the LS process, but we never write anything merged out to disc.

Another idea might be to actually spawn the process that does all the work, and let the LS continue to run while the child process runs. Once the child process is done, its results could be merged into the in memory cache. This would mean that the LS could still respond to messages while the child process extracts stuff. If a request for a doc comes in that hasn't been added to the cache, it would just return empty, which still seems better than just blocking the LS?

Not an issue (as in problem) but I have managed to get the Julia LanguageServer working with neovim :)

This is the minimal config I use to get it working (installed/managed via vim-plug):

call g:plug#begin()
  Plug 'JuliaEditorSupport/julia-vim'
  Plug 'autozimu/LanguageClient-neovim', { 'do': ':UpdateRemotePlugins' }
  Plug 'roxma/nvim-completion-manager'
call g:plug#end()

" julia
let g:default_julia_version = '0.6'
let g:julia_syntax_highlight_deprecated = 1

" language server
let g:LanguageClient_autoStart = 1
let g:LanguageClient_serverCommands = {
\   'julia': ['julia', '--startup-file=no', '--history-file=no', '-e', '
\       using LanguageServer;
\       server = LanguageServer.LanguageServerInstance(STDIN, STDOUT, false);
\       server.runlinter = true;
\       run(server);
\   '],
\ }

nnoremap <silent> K :call LanguageClient_textDocument_hover()<CR>
nnoremap <silent> gd :call LanguageClient_textDocument_definition()<CR>
nnoremap <silent> <F2> :call LanguageClient_textDocument_rename()<CR>

Note: JuliaEditorSupport/julia-vim is required otherwise Neovim thinks .jl files are lisp and the client won't start. Plus, some syntax highlighting is always nice :)

Linting from the LanguageServer doesn't seem to work so I'm currently using NeoMake/Lint.jl for that (not shown).

I think we switched that off, right? We should probably go back at some point.

Right now diagnostic messages only show up in the PROBLEMS panel in VS Code once a file has been opened. Could we instead just send all diagnostics back as soon as we have them, even if the file was never opened? Those messages then still show up in the PROBLEMS panel, and I think that is generally nicer and more standard in the VS Code universe.

@tkelman, re this I was going to add some new linting checks. Do you want any more rules added?

New Linting rules

• Extended external methods should reference at least one package specific type.
• Check module imports against REQUIRE.
• Check julia REQUIRE version against Travis.
• Check pkg REQUIRE versions against Pkg.available(pkg)
• Check for min REQIURE version of Compat.

See https://discourse.julialang.org/t/warning-vscode-formatter-shredders-code/4663.

@ZacLN we should push a bug fix release for this problem ASAP. Main question is: should we fix this on master here and then release a new version that has all the other recent stuff you did, or should we instead do a pure bug-fix release that just includes a fix for this issue? I don't have a good understanding of the things you did since the last official release, so probably better if you make that call.

Cannot tag a new version "v0.0.1" preceding all existing versions.
cc: @davidanthoff

I'm adding this as a medium term aim. The linter seems to cause more trouble than it's worth, I can't do any work without having it disabled and more often than not I'm uninterested in it's messages.

I've begun to strip/rewrite JuliaParser so that it's a bit less opaque and have started on implementing the lexer/parser that retains whitespace. It'll take a while but when it's done the parser in conjunction with the language server protocol 3.0 should be able to:

• provide a CST that can regenerate the source file.
• return specific (actionable) information on parse errors (i.e. error("expected 'end') -> insert end)
• be able to return some formatting hints (i.e. indent rules, spaces after commas, etc)
• keep track of the name space (a la namespace.jl) as it parses to warn of conflicting names, use of globals, etc

Eventually this should be able provide all the non-evaluated functionality that Lint.jl does.

Trawl through deprecated.jl and add linter warnings where possible.

Right now they are shown as info messages, which seems not ideal.

Could they just be shown like all other diagnostics as well? Maybe if we use a different string in the source field (something like "julia package linter") those messages won't be removed once the standard linter (who might send messages with a source value of "julia linter") sends a new set of diagnostics? I'm not sure whether that is the case, though.

0.7-beta

https://github.com/JuliaEditorSupport/LanguageServer.jl/blob/50bb880bceb2acb1605e82b4e00b02196ac06f94/src/protocol.jl#L165

According to the spec:

The range and the rangeLength fields do not have to be provided.

• An event describing a change to a text document. If range and rangeLength are omitted
• the new text is considered to be the full content of the document.

I guess this never came up before because VScode would automatically fill in those fields with 0 and length(document) ?

My work around, at least to let me continue testing is:

  function TextDocumentContentChangeEvent(d::Dict)
      if length(d)==1
          text = d["text"]
          rangeLength = length(text)
          lines = split(text, "\n")
          TextDocumentContentChangeEvent(Range(0,0, length(lines)-1, length(lines[end])-1), rangeLength, text)
      else
          TextDocumentContentChangeEvent(Range(d["range"]), d["rangeLength"], d["text"])
      end
  end

Currently we only analyse files that get opened in the editor, but I think the recommended way of operating is that provide diagnostics for all files in the current workspace. Plus it probably makes sense to parse all files in the workspace so that symbol resolution etc. can work on this.

I'll tackle this once #4 is merged. The general strategy would be this: if rootPath in the initialize request is not empty, open all *.jl files, lint them and run the parser on them. In the VS Code extension register a file watcher for those files, so that we get file change notifications for changes etc.

It is interesting because the docs are found.

To note is that ALL exported variables in the package are warned as undefined.

I see two paths: 1) we wait until @ZacLN's JuliaParser rewrite enables code formatting or 2) someone tries to use https://github.com/KristofferC/Tokenize.jl to add something in the meantime.

My guess is that neither @ZacLN nor I will get to 2) anytime soon, but if someone wants to contribute something along those lines, let us know here and we'll help you get started!

In a module with something similar to:

module MM
   using Example
   export hello
   Example.hello(x::Int) = "hello"
end

there will be a warning that hello is exported even though it is not defined. However, extending a method also defines the method itself in the module.

This is on LanguageServer.jl master:

The request that the atom language client sends seems to be well-formed, so this seems to be a LanguageServer.jl problem.

The list is really cool that shows up, but when I go through the list with the up and down arrow and look at the yellow highlights, it always highlights too many lines. I think it always seems to highlight everything up to the next code block, including the first line of the next code block. Ideally it would only highlight e.g. the function itself, i.e. everything up to and including the end statement.

Here is an example:

@ZacLN Here is one idea how we might be able to avoid a JuliaParser fork, but clearly you need to vet this whether it makes sense. This is based largely on what I see here.

Essentially the idea is to modify JuliaParser in such a way that by default it returns exactly the same things as without your modifications, but that we can add code here (or in another package) that returns either Expr objects with position and error info, or maybe in the future even a new Expr2 type or something like that.

Essentially the idea would be this. Say the top level parse function right now has this structure:

function parse(input)
    bla bla
    throw(D)
    bla bla
end

So somewhere in the parse method (or another method called by it) you now want to replace this throw(D) with a return Expr(:error,D). We could modify the signature of parse in this way:

function parse(input)
    return parse(Val{:default}, input)
end

function parse{T}(::Type{T}, input)
    bla bla
    return foo(T, D)
    bla bla
end

function foo(::Type{Val{:default}}, D)
    throw(D)
end

So this would be the code in JuliaParser. In LS, we would add this method:

function foo(::Type{Val{:lsparser}}, D)
    return Expr(:error,D)
end

I think if this kind of structure would work for all the modifications you need, the JuliaParser folks might be ok with merging this (I hope). The benefit here is this: a) I think there should be no perf penalty relative to the current JuliaParser code, b) none of the location code would actually be in JuliaParser, i.e. we could keep this here in the LS package, experiment and change it quickly etc., c) JuliaParser should look and behave exactly like it does right now for other users, like the Gallium debugger etc.

Do you think something like that could work?

I think this issue julia-vscode/julia-vscode#108 can be solved by using code lens provider.

My idea is to create a set of (codelens actions) buttons, like: run block, run file, etc. at the beginning of each top-level block. User can use mouse or just use key shortcut to run the code block

I believe this line of code need to handle the exception caused by removing the file before reading. Otherwise, an error would make the language server fail, and restart in vscode.

https://github.com/JuliaEditorSupport/LanguageServer.jl/blob/master/src/provider_misc.jl#L48

struct S end

struct M
    y
end

(m::M)(x) = m.y + x

gives that both m in the function declaration are potentially undefined.

I've started to play with this in Emacs (using lsp-mode.el), and was finally able to get both sides to talk to each other. Shocked and delighted that jump to symbol works! Here are a couple of questions/issues that I've run into.

1. I ran into a problem with an empty or malformed message taking down the server right at the start. I solved this by wrapping the process(request, server) line in languageserverinstance.jl with a try-catch block, ignoring any errors. I'm guessing there's a better approach, but I'm obviously not too familiar with the inner workings of this package.

2. Emacs refuses to quit, because it cannot shut down the language server. It looks like the required message is not implemented here? https://github.com/JuliaEditorSupport/LanguageServer.jl/blob/eff7b98c8a3a3834dc3111ed96abaef0b9e414bd/src/provider_misc.jl#L68

Can that be added? It does make hacking on this somewhat difficult.

Thanks

Currently shifting branches can mess up the state of LS workspace

See screenshot. Basically, module Foo is being tagged as "possible use of undeclared variable Foo".

This is v0.6.2 with Julia v0.6.0.

There are still some calls to that function in the code, but the function itself is no longer there.

At the end of each line, it could remove any trailing whitespaces.

sbromberger/LightGraphs.jl#687

Thank you :)

I think this can wait a bit, might just make sense to start thinking about this.

I think for the linter we can just put all the markdown into comments and then send that stuff to the linter.

Less sure about the rest...

This starts with this observation from @ZacLN:

We need to have a think about how we want to handle workspace wide parsing and user included Modules, I'm a bit wary of just parsing everything and including it across all files. I think we need some set of rules that determine whether some arbitrary file is related to the one currently open. Maybe based on include(..)'s across the project?

Yes, this is a tricky one... Even in one file this is not simple. Take this file:

# EDIT ZONE START

# EDIT ZONE END

function foo()
end

Should the completion provider actually offer foo if one starts to type in the edit zone? I think for now that is probably the only sensible answer...

Ok, but on to the real problem. I think ideally we would store the stuff that the parser finds in a file as part of that Document, not as part of some global symbol list. And then the question is, what symbols from other files should the completion provider/doc etc. use in addition to the local symbols. There is one case that is relatively easy: if file A includes file B, then while editing A the LS should also show completions for things defined in B, doc help for things defined in B etc. I think the more complicated situation is that we edit file B, and say file A looks like this:

module FileA
include("file_C.jl")
include("file_B.jl")
end

This is really tricky, because it means that a) actually all the symbols in file B are in the module FileA, but there is no way to learn that from just looking at file_B.jl, and second we should actually show all the symbols/docs etc. from file_C while editing file_B...

I'm not entirely sure how to resolve this... Here are some options:

1. We could build a graph that represents all the includes etc, and while editing file_B we would show symbols from any file that follows the structure of the example above. But one problem is this: file_B might be included by multiple "parent" files, and in different modules. Say there is another file that includes file_B, but here file_B is included in a different module. In what module should the symbols in file_B live now? The module in file_A, or the module in the new file?
2. A more conservative approach would be that we special case this for packages. I.e. we build up a graph, and we detect the case where there is one file in the src folder that has the same name as root folder and starts with a module with the same name, and if all of that is the case, we scope all the included files accordingly into that modules namespace.

In both of these cases we would of course have to do everything recursively, i.e. follow multiple includes etc... Plus using, import, importall etc..

I don't feel I fully comprehend the whole thing at this point, this is certainly a tricky issue... I do think (hope) it is distinct from how we handle using statements that load other packages...

See here. Might be quite tricky to do, but maybe sometimes down the road.

Format document puts space in between the curly braces. Fixing a deprecation does not put space between them.

Right now if you start a docstring with triple quotes, until you put the closing quotes in, the entire file will be underlined in red as a parsing error. (Same goes for any block without a corresponding end.) This is really distracting.

Would it be possible to just highlight the section of code that is "in error"? (or, as an alternative, when a block starts, automatically insert end statements or closing quotes as necessary?)

When I package lint this package, I get two info messages. Both complain that I have packages declared in REQUIRE that I don't seem to be using. The complaint about DataValues I understand, because that one is used via the @reexport macro and so can't really be detected. But the complaint about Reexport seems incorrect, there is a using Reexport here?

function foo()
    a = ones(3, 3)
    b = a'
    return b
end

yields

file: 'untitled:Untitled-1'
severity: ''
message: 'Unexpected white space around operator'
at: '3,10'
source: 'CSTParser.Diagnostics.Diagnostic'

for the b = a' line.

Would be cool if the format document command would automatically indent everything properly.

There are two problems with the current linter approach: 1) it uses all user packages, i.e. it loads user code into the LS process, which is not ideal, and 2) once we open all files in the workspace (#9) things might get slow.

Here is how I propose to solve this: start another julia process from the LS process that is used for linting. Communicate with that process via the lintserver method. Submit all the lint requests to that process, but return from any document handling request right away in the LS process (e.g. incremental updates). As responses from the lint process come back in, we can send the diagnostic notifications back to the extension async.

Using vscode 0.7 beta.

Smallest example (ignore the quotes in top left corner):

Actually "$a" is a smaller example that also shows the squiggly wrong.

A bit larger example where there is a larger desynchronization:

Here are some interesting reads about the C# Roslyn design:

• https://blogs.msdn.microsoft.com/ericlippert/2012/06/08/persistence-facades-and-roslyns-red-green-trees/
• http://stackoverflow.com/questions/25963328/how-does-the-release-version-of-roslyn-implement-immutable-trees
• https://roslyn.codeplex.com/discussions/541953
• http://hillside.net/plop/2013/papers/Group2/plop13_preprint_22.pdf
• https://robinsedlaczek.com/2015/04/29/inside-the-net-compiler-platform-performance-considerations-during-syntax-analysis-speakroslyn/

The VS Code UI now shows things like "[CSTPasrer.Diagnostics.Diagnostic] MissingWS" or "[CSTPasrer.Diagnostics.Diagnostic] ExtraWS". I think these are the enum values?

I think we should change this so that a) the source of the message is just something like "julia linter" or something short like that and b) we should have short one sentence descriptions of the problem that will show up in the UI, something like "Whitespace missing" or something like that.