Code Monkey home page Code Monkey logo

sourcebrowser's Introduction

SourceBrowser

NuGet package

Source browser website generator that powers https://referencesource.microsoft.com, http://sourceroslyn.io, https://source.dot.net, and others.

Create and host your own static HTML website to browse your C#/VB/MSBuild/TypeScript source code. Note that it does require an ASP.NET Core website for hosting (symbol index is kept server-side), so without ASP.NET Core the search function doesn't work.

Of course Source Browser allows you to browse its own source code: http://sourcebrowser.azurewebsites.net

Now also available on NuGet: https://www.nuget.org/packages/SourceBrowser

Instructions to Build (requires Visual Studio 2019):

  1. git clone https://github.com/KirillOsenkov/SourceBrowser
  2. cd SourceBrowser
  3. Build.cmd

Instructions to generate and run a test website

  1. GenerateTestSite.cmd
  2. RunTestSite.cmd

In Visual Studio 2019:

  1. Open SourceBrowser.sln.
  2. Set HtmlGenerator project as startup and hit F5 - it is preconfigured to generate a website for TestCode\TestSolution.sln.
  3. Pass a path to an .sln file or a .csproj file (or multiple paths separated by spaces) to create an index for them
  4. Pass /out: to HtmlGenerator.exe to configure where to generate the website to. This path will be used in step 6 as your "physicalPath".
  5. Pass /in: to pass a file with a list of full paths to projects and solutions to include in the index
  6. Pass /root: if you want to preserve relative .sln folders rather than merging all solutions. This folder must contain all specified .sln or .csproj paths.
  7. Set SourceIndexServer project as startup and run/debug the website.

Note: Visual Studio 2019 is required to build Source Browser.

Conceptual design

At indexing time, C# and VB source code is analyzed using Roslyn and a lot of static hyperlinked HTML files are generated into the output directory. There is no database. The website is mostly static HTML where all the links, source code coloring etc. are precalculated at indexing time. All the hyperlinks are hardwired to be simple links bypassing the server.

The only component that runs on the webserver is a service that given a search query does the lookup and returns a list of matching types and members, which are hyperlinks into the static HTML. The webservice keeps a list of all declared types and members in memory, this list is also precalculated at indexing time. All services, such as Find All References, Project Explorer, etc. are all pre-rendered.

The generator is not incremental. You have to generate into an empty folder from scratch every time, and then replace the currently deployed folder with the new contents atomically (using e.g. Azure Deployments, robocopy /MIR to inetpub\wwwroot, etc). For smaller projects, deploying to Azure using Dropbox or Git would work just fine.

Limitations and known issues

  1. Indexing more than one project with the same assembly name is currently unsupported. Only the first project wins. This is due to a fundamental design decision to only reference an assembly by short name. Customizers should add a means to filter "victim" projects out in their forks to pick the best single project for inclusion in the index.
  2. The generated website can only be hosted in the root of the domain. Making it run from a subdirectory is non-trivial and unlikely to be supported.

Features

  • Solution Explorer - contents of projects merged into single tree on the left
  • coloring for C#, VB, MSBuild, XAML and TypeScript
  • Go To Definition (click on a reference)
  • Find All Reference (click on a definition)
  • Project Explorer - in any document click on the Project link at the bottom
  • Namespace explorer - for a project view all types and members nested in namespace hierarchy
  • Document Outline - for a document click on the button in top right to display types and members in the current file
  • http://<URL>/i.txt for the entire solution and /AssemblyName/i.txt (for an assembly) displays source code stats, lines of code, etc
  • http://<URL>/#EmptyArrayAllocation finds all allocations of empty arrays (this feature is one-off and hardcoded and not extensible)
  • Clicking on the partial keyword will display a list of all files where this type is declared
  • MSBuild files (.csproj etc) have hyperlinks
  • TypeScript files (*.ts) are indexed if they're part of a C# project. Work underway to allow an arbitrary array of TypeScript files.
  • Search for GUIDs in C#/VB string literals is supported

Project status and contributions

This is a reference implementation that showcases the concepts and Roslyn usage. It comes with no guarantees, use at your own risk. We will consider accepting high-quality pull requests that add non-trivial value, however we have no plans to do significant work on the application in its current form. Any significant rearchitecture, adding large features, big refactorings won't be accepted because of resource constraints. Feel free to use it to generate websites for your own code, integrate in your CI servers etc. Feel free to do whatever you want in your own forks. Bug reports are gratefully accepted.

For any questions, feel free to reach out to @KirillOsenkov on Twitter. Thanks to @v2_matveev for contributing TypeScript support! Thanks to numerous other contributors for various fixes and contributions!

sourcebrowser's People

Contributors

adelecourt avatar alexperovich avatar ashmind avatar atifaziz avatar bernd5 avatar citizenbean avatar controlflow avatar database64128 avatar dependabot[bot] avatar dotnet-bot avatar drose-d2l avatar dustincampbell avatar jaredpar avatar jcorbett313 avatar jnm2 avatar joerobich avatar jrc313 avatar kirillosenkov avatar ltrzesniewski avatar m-akinc avatar mrahhal avatar reegeek avatar sebazzz avatar stephentoub avatar tiesmaster avatar trevordarcyevans avatar vladima avatar weihanli avatar yaakov-h avatar yuvalitzchakov avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sourcebrowser's Issues

Have a way to turn the Git blame feature on or off and make it off by default

We shouldn't assume that the majority of the solutions are in Git and even if so, that users want to pay for this feature by default.

I think it should be opt-in. We should also advertise this so that it's obvious to users that this feature exists and have a Wiki page on how to turn it on and how to use it.

No js, css support makes it unusable for web projects

Just want to say that this is a great project and it could be used by many more people if it would do at-least some basic indexing of js,css to find symbols.

I think that unsupported file types should not be ignored and instead be treated like text and when a search is performed then the appropriate line can be opened in them.
This will give at least a basic handling of unsupported files like js,css.

Or can go even further and to add a configuration of "textual file extensions" which will be indexed by symbols and lines.

Remove a first-chance exception in Git blame

Currently when indexing any project there's a first-chance exception from libgit2 followed up by an exception log entry:

16:32:49 1/30/2017 4:32:49 PM: First chance exception while processing assembly: WpfApplication1
LibGit2Sharp.NotFoundException: the path 'obj' does not exist in the given tree
   at LibGit2Sharp.Core.Ensure.HandleError(Int32 result) in c:\Git\libgit2sharp\LibGit2Sharp\Core\Ensure.cs:line 160
16:32:49 Couldn't blame C:\SourceBrowser\TestCode\WpfApplication1\obj\Debug\Page1.g.cs
16:32:49 Couldn't blame C:\SourceBrowser\TestCode\WpfApplication1\obj\Debug\GeneratedInternalTypeHelper.g.cs
LibGit2Sharp.NotFoundException: the path 'obj' does not exist in the given tree
   at LibGit2Sharp.Core.Ensure.HandleError(Int32 result) in c:\Git\libgit2sharp\LibGit2Sharp\Core\Ensure.cs:line 160
   at LibGit2Sharp.Core.Ensure.ZeroResult(Int32 result) in c:\Git\libgit2sharp\LibGit2Sharp\Core\Ensure.cs:line 179
   at LibGit2Sharp.Core.Proxy.git_blame_file(RepositorySafeHandle repo, FilePath path, GitBlameOptions options) in c:\Git\libgit2sharp\LibGit2Sharp\Core\Proxy.cs:line 106
   at LibGit2Sharp.BlameHunkCollection..ctor(Repository repo, RepositorySafeHandle repoHandle, String path, BlameOptions options) in c:\Git\libgit2sharp\LibGit2Sharp\BlameHunkCollection.cs:line 45
   at LibGit2Sharp.Repository.Blame(String path, BlameOptions options) in c:\Git\libgit2sharp\LibGit2Sharp\Repository.cs:line 831
   at LibGit2Sharp.RepositoryExtensions.Blame(IRepository repository, String path) in c:\Git\libgit2sharp\LibGit2Sharp\RepositoryExtensions.cs:line 631
   at GitGlyph.GitBlameVisitor.GetBlame(String path) in C:\SourceBrowser\src\GitGlyph\GitBlameVisitor.cs:line 88
16:32:49 Couldn't blame C:\SourceBrowser\TestCode\WpfApplication1\obj\Debug\GeneratedInternalTypeHelper.g.cs

It would be nice to see if libgit2 allows a way to probe for a given file whether it's a part of the repo and don't try to annotate it if not, so that we can avoid the exception.

If that's not possible, Plan B is to whitelist the first-chance exception so that it doesn't get logged.

What to deploy?

It's likely me, but I have a hard time figuring out what to deploy. I've generated the index output using HtmlGenerator and created a webapp on IIS on our local server to test what to do on the remote server. When browsing to this webapp I see the familiar interface and things work when browsing. However search throws a missing method exception, which is likely due to the fact I do something wrong. So I see the SourceIndexServer, but to get that started according to the docs, I have to alter a line in a .vs config file. But what do I do in my own webserver? I have no deep knowledge of IIS configs other than setting up an app, application pool etc. So I'm a bit lost what to deploy (the Index folder generated by HtmlGenerator and SourceIndexBrowser, or just Htmlgenerator?).

TIA and sorry if this is documented already somewhere and I overlooked it.

Source Code Location vs. Type Assembly

I was just using source.dot.net and searched System.String. It works as expected, I get the source code for the string in System.Private.CoreLib (finally without getting lost in the CoreCLR project and the mscorlib sub folder). However, as a normal consumer I would have suspected the type in System.Runtime where the normal compilation links against. This is a planned difference between source.dot.net and apisof.net.

I would ask for a also integrating the TypeForwarders (well knowing that most of them are build artifacts ;)) into the source browser, so this detail of the source code is also highlighted.

@terrajobst

NullRef

System.NullReferenceException: Object reference not set to an instance of an object. 
at Microsoft.SourceBrowser.SourceIndexServer.Models.SortedSearch.FindBounds(String word, Int32& low, Int32& high) 
at Microsoft.SourceBrowser.SourceIndexServer.Models.SortedSearch.FindItem[T](IList`1 list, String word, Func`2 keySelector) 
at Microsoft.SourceBrowser.SourceIndexServer.Models.Index.FindAssembly(String assemblyName) 
at Microsoft.SourceBrowser.SourceIndexServer.Models.Index.GetReferencingAssembliesCount(String assemblyName) 
at Microsoft.SourceBrowser.SourceIndexServer.Models.ResultsHtmlGenerator.GetNumberOfReferences(String assemblyName, Index index) 
at Microsoft.SourceBrowser.SourceIndexServer.Models.ResultsHtmlGenerator.<>c__DisplayClass14_0.b__1(IGrouping`2 g) 
at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext() 
at System.Linq.Buffer`1..ctor(IEnumerable`1 source) 
at System.Linq.OrderedEnumerable`1.d__1.MoveNext() 
at Microsoft.SourceBrowser.SourceIndexServer.Models.ResultsHtmlGenerator.WriteSymbolResults(Index index) 
at Microsoft.SourceBrowser.SourceIndexServer.Models.ResultsHtmlGenerator.Generate(Stopwatch sw, Index index, String usageStats) 
at Microsoft.SourceBrowser.SourceIndexServer.Controllers.SymbolsController.GetHtmlCore(String symbol, String usageStats) 
at Microsoft.SourceBrowser.SourceIndexServer.Controllers.SymbolsController.GetHtml(String symbol)

HTTP 404 on generated site

Hi,

I've been using Microsoft.SourceBrowser from NuGet, but it doesn't support the new project format with MSBuild 15.0, or new syntax with Roslyn 2.0.

I've managed to build the latest code from source.

I used the following command-line to generate the index and website:

C:\Git\SourceBrowser\src\HtmlGenerator\bin\Debug\net46\HtmlGenerator.exe  /force /out:__SourceBrowser__ /in:ProjectList.txt

The generated site appears to mostly work, but there's no main page. URLs such as /i.txt work, but / just returns a HTTP 404.

At first I thought this might be an IIS issue, so I used dotnet to run Kestrel and got the same result.

C:\CW\__SourceBrowser__>dotnet .\Microsoft.SourceBrowser.SourceIndexServer.dll --help
Hosting environment: Production
Content root path: C:\CW\__SourceBrowser__
Now listening on: http://localhost:5000
Application started. Press Ctrl+C to shut down.
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
      Request starting HTTP/1.1 GET http://localhost:5000/
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
      Request finished in 170.0031ms 404
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
      Request starting HTTP/1.1 GET http://localhost:5000/
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
      Request finished in 0.7488ms 404

Am I missing a step, or is the generated site actually incomplete?

Provide return code on error

Please provide a return code for the HtmlGenerator when an error occurs and return only 0 when it completes successfully.

HtmlGenerator.exe fails with HTTP proxy authentication error (407)

Suppose you are in the directory C:\Temp.

Install Microsoft.SourceBrowser (version 1.0.7 at time of reporting this issue) using NuGet:

nuget install Microsoft.SourceBrowser -Version 1.0.7

Clone this project on which HtmlGenerator will be invoked:

git clone https://github.com/KirillOsenkov/SourceBrowser

Now run:

Microsoft.SourceBrowser.1.0.7\tools\HtmlGenerator.exe SourceBrowser\SourceBrowser.sln

You should see the program fail with a HTTP proxy authentication (407) error:

13:36:14 Generating website
13:36:14 Reading assembly names from C:\Temp\SourceBrowser\SourceBrowser.sln
13:36:14 Reading assembly names from C:\Temp\SourceBrowser\SourceBrowser.sln complete. Took: 00:00:00.0266311
13:36:14 09/10/2015 13:36:14: First chance exception
System.EntryPointNotFoundException: Unable to find an entry point named 'EventSetInformation' in DLL 'advapi32.dll'.
   at Microsoft.Win32.UnsafeNativeMethods.ManifestEtw.EventSetInformation(Int64 registrationHandle, EVENT_INFO_CLASS informationClass, Void* eventInformation, Int32 informationLength)
   at System.Diagnostics.Tracing.EventProvider.SetInformation(EVENT_INFO_CLASS eventInfoClass, Void* data, Int32 dataSize)
13:36:14 09/10/2015 13:36:14: First chance exception
System.Net.WebException: The remote server returned an error: (407) Proxy Authentication Required.
   at System.Net.HttpWebRequest.GetResponse()
13:36:14 09/10/2015 13:36:14: First chance exception
System.Net.WebException: The remote server returned an error: (407) Proxy Authentication Required.
   at System.Net.HttpWebRequest.GetResponse()
13:36:14 09/10/2015 13:36:14: First chance exception
System.Net.WebException: The remote server returned an error: (407) Proxy Authentication Required.
   at System.Net.WebClient.DownloadDataInternal(Uri address, WebRequest& request)

Unhandled Exception: System.Net.WebException: The remote server returned an error: (407) Proxy Authentication Required.
   at System.Net.WebClient.DownloadDataInternal(Uri address, WebRequest& request)
   at System.Net.WebClient.DownloadString(Uri address)
   at Microsoft.SourceBrowser.HtmlGenerator.Federation..ctor(String[] servers)
   at Microsoft.SourceBrowser.HtmlGenerator.Federation..ctor(IEnumerable`1 servers)
   at Microsoft.SourceBrowser.HtmlGenerator.Federation..ctor()
   at Microsoft.SourceBrowser.HtmlGenerator.Program.IndexSolutions(IEnumerable`1 solutionFilePaths)
   at Microsoft.SourceBrowser.HtmlGenerator.Program.Main(String[] args)
13:36:17 Generating website complete. Took: 00:00:03.2944425

Performance: Incremental build and directory structure

I'm looking into the problem of scaling SourceBrowser right now. Currently, running our entire codebase through the indexer takes ~28 hours on a pretty decent machine. I see two options:

  1. Federation
  2. Incremental build

The former has the problem that federated text search is messy. It also requires some reasonable decisions about how to break up a codebase, and maintenance on all individual instances. Given this, I'm focused on the second option.

My analysis is that there are two halves to this issue:

  1. Which assemblies should be re-indexed? I see this as a separate question. Some other tool should either feed this information to SourceBrowser, or signal it by deleting indices that should be rebuilt.
  2. How to rebuild an assembly's index? This is actually quite complex, because indices of assemblies modify each other's folders in ways that are not always easy to trace.

An intermediate goal I am considering, then, is to deepen the division between SourceBrowser's indexers two phases (Pass1-Generation and Pass2-Finalization) by introducing two invariants:

  1. The ProjectGenerator at the heart of Pass1, as well as anything it creates (DocumentGenerators, etc.), shall write ONLY to the relevant assembly directory.

  2. The ProjectFinalizer at the heart of Pass2, as well as anything it creates, shall read from assembly directories, and write only to a new set of "output" directories, from which the website is actually run.

In my mind, this has a few benefits:

  1. Deleting an assembly directory signals to Pass1 that regeneration is needed. Pass2 can be re-run each time, deleting and rebuilding its output directories as it does so.

  2. Intermediary files (i.e. the txt files) can be left in place without intermingling with the HTML that comprises the website. This means cleaning these up is less pressing, and can make debugging easier.

  3. Because of the "index" and "website" structure separation, each can be specialized to be easier to work with for its purpose.

  4. Parallelism becomes simpler, because during Pass1 no two ProjectGenerators need ever interact.

Given that this is a fairly significant re-factor, however, I wanted to gather feedback from others who have worked on the project. Am I missing some danger of this approach? Is there a better approach to gaining incremental builds? Is there a better approach to improving performance? Alternative ideas that might save me a lot of refactoring would be greatly appreciated.

"SolutionExplorer" view contents with multiple solutions

(sorry for the mess if you follow this via E-Mail; but I figured it would be better to rephrase and update the original question)

Hi,

first of all, thanks for putting this out in the wild :-)

I have a source tree that looks structurally like this:

\<toplevel>
     solution1.sln
     solution2.sln
     solution3.sln
\<toplevel>\solution1-projects\src\<project1>
\<toplevel>\solution1-projects\src\<project2>
\<toplevel>\solution2-projects\src\<project3>
\<toplevel>\solution2-projects\src\<project4>
\<toplevel>\solution3-projects\src\<project5>
\<toplevel>\solution3-projects\src\<project6>

Each solution depends on the previous. That is, and depend on the assemblies build by solution1.sln, etc.

Now, when I invoke the HtmlGenerator.exe tool like so, HtmlGenerator.exe /out:<somedir> Solution1.sln Solution2.sln Solution3.sln I get indexed sources for all projects, but a "SolutionExplorer" that only reflects the last solution file specified on the command line.

Furthermore, the files between the different solutions are not cross-referenced. For example, a type from a project in solution1, that appears in a project in solution2, is no link to that source file.

I can understand if you don't have the time to debug into this (I will try that myself over the next week or so), but maybe you have an idea what could be going wrong here.

Thanks,
Christian

Latest Roslyn version changed Glyph numbers

Running the generator with Roslyn 2.3.1 results in unrecognized symbol types in the generated index. For example interfaces are not searchable and show up as "symbols". Most probably the Glyph IDs changed in Roslyn, and IndexEntry.GetKind returns the default symbol kind text for them.

Host outside of IIS

I tried host generated website with a simple python webserver

python -m http.server 8000

Search doesn't work in this case.

Publish to nuget

Publish the tool to nuget so we can use it during build to publish our code without the need to add the binary to the repo.

Build failed for XML namespace

It seems Project cannot build with VS2015/2017 since yesterday 30th/Mar/2017.

Issue reproduce with downloaded latest zip build with MSBuild/VS2015 with instruction on README. Here's error message.

> SourceBrowser-master\src\BuildLogParser\BuildLogParser.csproj(1,1): error MSB4041: The default XML namespace of the project must be the MSBuild XML namespace. If the project is authored in the MSBuild 2003 format, please add xmlns="http://schemas.microsoft.com/developer/msbuild/2003" to the <Project> element. If the project has been authored in the old 1.0 or 1.2 format, please convert it to MSBuild 2003 format.
> SourceBrowser-master\src\Common\Common.csproj(1,1): error MSB4041: The default XML namespace of the project must be the MSBuild XML namespace. If the project is authored in the MSBuild 2003 format, please add xmlns="http://schemas.microsoft.com/developer/msbuild/2003" to the <Project> element. If the project has been authored in the old 1.0 or 1.2 format, please convert it to MSBuild 2003 format.

Is there any additional work should I do before build?

Could not generate the web site from 1.0.15 binaries

Hello,
I need to generated the web site for two source code repositories. For that i passed to the HtmlGenerator.exe two .sln files, i tried with the 1.0.14 version, the web site is generated but in the repo explorer bar i can see only one repository, i have to navigate through assmeblies to get to the second one.

So i wanted to try the latest version 1.0.15, but the web site isn't generated (i don't see any index.html ) , it fails with the exception bellow.

Is there a change between binaries of the two versions ? because i cannot find all .css/.js/.html files that used to be located in "\tools\web".

I have VS 2015 installed, i have both MSBuild 12 and 14. I am on Windows 7.

Thank you

15:57:11: First chance exception
System.Runtime.InteropServices.COMException (0x80040154): Retrieving the COM class factory for component with CLSID {177F0C4A-1CD3-4DE7-A32C-71DBBB9FA36D} failed due to the following error: 80040154 Class not registered (Exception from HRESULT: 0x80040154 (REGDB_E_CLASSNOTREG)).
   at System.RuntimeTypeHandle.CreateInstance(RuntimeType type, Boolean publicOnly, Boolean noCheck, Boolean& canBeCached, RuntimeMethodHandleInternal& ctor, Boolean& bNeedSecurityCheck)
   at System.RuntimeType.CreateInstanceSlow(Boolean publicOnly, Boolean skipCheckThis, Boolean fillCache, StackCrawlMark& stackMark)
```

```
15:57:11: First chance exception
System.DllNotFoundException: Unable to load DLL 'Microsoft.VisualStudio.Setup.Configuration.Native.dll': The specified module could not be found. (Exception from HRESULT: 0x8007007E)
   at Microsoft.Build.Shared.VisualStudioLocationHelper.GetSetupConfiguration(ISetupConfiguration& configuration, IntPtr reserved)
   at Microsoft.Build.Shared.VisualStudioLocationHelper.GetQuery()
```

```
15:57:12: First chance exception
System.ArgumentNullException: Value cannot be null.
Parameter name: o
   at System.Runtime.InteropServices.Marshal.GetIUnknownForObjectNative(Object o, Boolean onlyInContext)
   at Microsoft.Build.Tasks.InteropUtilities.RCWForCurrentContext`1..ctor(T rcw)
```

```
15:57:12 Workspace failed: Msbuild failed when processing the file 'D:\repo1\repo1.csproj' with message: C:\Program Files (x86)\MSBuild\12.0\bin\amd64\Microsoft.CSharp.CurrentVersion.targets: (448, 9): Invalid command line switch for "Csc.exe". Value cannot be null.
Parameter name: o
15:57:12 Msbuild failed when processing the file 'D:\repo1\repo1.csproj' with message: C:\Program Files (x86)\MSBuild\12.0\bin\amd64\Microsoft.CSharp.CurrentVersion.targets: (448, 9): Invalid command line switch for "Csc.exe". Value cannot be null.
```

Queries that contain \ crash

http://source.roslyn.io/#q=msbuild%20src%5C

System.NullReferenceException: Object reference not set to an instance of an object. at SourceIndexServer.Models.Query.FilterProjects(DeclaredSymbolInfo symbol) at SourceIndexServer.Models.Query.Filter(DeclaredSymbolInfo symbol) at System.Linq.Enumerable.<>c__DisplayClass01.b__1(TSource x) at System.Linq.Enumerable.WhereEnumerableIterator1.MoveNext() at System.Linq.Enumerable.d__11.MoveNext() at System.Collections.Generic.List1..ctor(IEnumerable1 collection) at System.Linq.Enumerable.ToList[TSource](IEnumerable1 source) at SourceIndexServer.Models.Index.FindSymbols(Query query, Interpretation interpretation) at SourceIndexServer.Models.Index.FindSymbols(Query query) at SourceIndexServer.Models.Index.Get(String queryString) at SourceIndexServer.Controllers.SymbolsController.GetHtmlCore(String symbol, String usageStats) at SourceIndexServer.Controllers.SymbolsController.GetHtml(String symbol)

Provide details on the last index run

It may be useful to provide details in the overview page (or in the header) about the last index run (time, server, sourcebrowser version, number of projects, number of assemblies, etc ...)
We use sourcebrowser to index our code base multiple times a day, so this info is useful for us.

Path issues

(i know i've been opening a lot of issues, sorry)

When hosting a generated SourceBrowser site on a path instead of main-domain/subdomain, many URLs are broken and redirect to the site root

Camel case search

Search feature now finds only all words starting with given prefix.
It would be nice to support:

  • camel case search
  • do not search only for prefix, but match at any position

So for example search string:
NeCl
would find:
MyNewClass
NewClass

Project conditional compilation symbols ignored

I'm noticing that project conditional compilation symbols are being ignored such that declaring "HELLO" as project settings compilation symbols does not enable the code within "#if HELLO". However, code within "#if !HELLO" does get enabled.

I didn't see anything in the documentation stating whether or not this was expected. Is it expected?
Thanks for the help, SourceBrowser is an excellent project!

SourceBrowser Error

Hi,

HTMLGenerator seems to writing the below error message on every solution I run it against. It continues with building the site and successfully completes it but I'm not sure what the below error is meant to be. Any help on this would be appreciated.
PS: The tool is just awesome and thanks for sharing it with the public.

First chance exception
System.EntryPointNotFoundException: Unable to find an entry point named 'ReadFile' in DLL 'api-ms-win-core-file-l1-2-0.dll'.
at System.Reflection.Internal.FileStreamReadLightUp.NativeMethods.ReadFileModern(SafeHandle fileHandle, Byte* buffer, Int32 byteCount, Int32& bytesRead, IntPtr overlapped)
at System.Reflection.Internal.FileStreamReadLightUp.TryReadFile(Stream stream, Byte* buffer, Int64 start, Int32 size)

HtmlGenerator can't deal with multiple csproj's with same assembly name

I have the following scenario:
One solution, with two projects, P1 and P1.Extended. Both use the namespace 'P1' and the assembly name 'P1'. P1.Extended contains all of P1's code (through file links in the csproj file), and extra classes. I use this in my work to have a .NET 4.5+ build of our runtime with extra classes for async/await support. This way I get multiple dlls with the same name (but stored in different folders), and it's a drop-in replacement to add async. If the assembly name would change, it would be more work for our users as they then had to change the references in their own code to the new file. (the same scenario is also used by e.g. Entity Framework, where they have two builds with the same assembly name, one with and one without async, as one is for .NET 4.0 and the other for .NET 4.5+)

If P1.Extended is the first project in the .sln file, it gets processed by HtmlGenerator and P1 is skipped (not mentioned at all in the output), if P1 is the first file in the sln, P1.Extended is skipped and not mentioned in the output at all.

I reproduced this with 2 projects with the names above in a simple sln file and both with their own class (class1 and class2) and I added class1 as a file link to P1.Extended. Based on the order in the sln file one is skipped over the other.

I think it's related to the fact that you can have assembly browsing so the name of the assembly is key here, but that would also skip a set of classes (namely the ones in the Extended assembly if it's lower in the sln file order). Not sure if this is fixable. the workaround is to have the extended project (which should contain all the code of the normal project) first in the sln file.

I had a .zip ready but GitHub doesn't want me to attach .zip files.

TargetFrameworks

Hi,

I'm not sure if this a bug/known issue/feature but:
In the new csprojs format it possible to mention target frameworks by two attributes:
TargetFrameworks and TargetFramework

https://docs.microsoft.com/en-us/dotnet/core/tools/csproj

It seems that if i used the plural property (TargetFrameworks) classes' html are not generated, but it do works for the singular property.

Thanks

Projects using new .csproj format unsupported?

I seem to be getting an error when my solution contains projects using the new .csproj format.
The error is sort of obscure, and doesn't point me to which project is failing (there's around 300 in the solution)

Is this a known issue? Or am I doing something wrong?

Microsoft.Build.Exceptions.InvalidProjectFileException: The default XML namespace of the project must be the MSBuild XML namespace. If the project is authored in the MSBuild 2003 format, please add xmlns="http://schemas.microsoft.com/developer/msbuild/2003" to the <Project> element. If the project has been authored in the old 1.0 or 1.2 format, please convert it to MSBuild 2003 format.
   at Microsoft.Build.Shared.ProjectErrorUtilities.ThrowInvalidProject(String errorSubCategoryResourceName, IElementLocation elementLocation, String resourceName, Object[] args)
   at Microsoft.Build.Construction.ProjectParser.Parse()
   at Microsoft.Build.Construction.ProjectParser.Parse(XmlDocumentWithLocation document, ProjectRootElement projectRootElement)
   at Microsoft.Build.Construction.ProjectRootElement.Create(XmlReader xmlReader, ProjectCollection projectCollection)
   at Microsoft.CodeAnalysis.MSBuild.ProjectFileLoader.<LoadProjectAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.CodeAnalysis.MSBuild.ProjectFileLoader.<LoadProjectFileAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.CodeAnalysis.MSBuild.MSBuildProjectLoader.<LoadProjectAsync>d__23.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.CodeAnalysis.MSBuild.MSBuildProjectLoader.<GetOrLoadProjectAsync>d__22.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.CodeAnalysis.MSBuild.MSBuildProjectLoader.<ResolveProjectReferencesAsync>d__28.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.CodeAnalysis.MSBuild.MSBuildProjectLoader.<LoadProjectAsync>d__23.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.CodeAnalysis.MSBuild.MSBuildProjectLoader.<GetOrLoadProjectAsync>d__22.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.CodeAnalysis.MSBuild.MSBuildProjectLoader.<ResolveProjectReferencesAsync>d__28.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.CodeAnalysis.MSBuild.MSBuildProjectLoader.<LoadProjectAsync>d__23.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.CodeAnalysis.MSBuild.MSBuildProjectLoader.<GetOrLoadProjectAsync>d__22.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.CodeAnalysis.MSBuild.MSBuildProjectLoader.<ResolveProjectReferencesAsync>d__28.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.CodeAnalysis.MSBuild.MSBuildProjectLoader.<LoadProjectAsync>d__23.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.CodeAnalysis.MSBuild.MSBuildProjectLoader.<GetOrLoadProjectAsync>d__22.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.CodeAnalysis.MSBuild.MSBuildProjectLoader.<LoadSolutionInfoAsync>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.CodeAnalysis.MSBuild.MSBuildWorkspace.<OpenSolutionAsync>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.SourceBrowser.HtmlGenerator.SolutionGenerator.CreateSolution(String solutionFilePath, ImmutableDictionary`2 properties) in C:\Source\SourceBrowser\src\HtmlGenerator\Pass1-Generation\SolutionGenerator.cs:line 394

Could not load file or assembly 'System.Threading.Tasks.Dataflow' when running from build server.

I've wrote a small script that download SourceBrowser from NuGet and runs it daily on our build server - a Windows 7 server with .Net 4.6 installed (Windows 2008 R2 didn't work either).
I get this exception while it runs:

09:49:48 Failed to open solution: D:\Jenkins\Slave\job\solution.sln
System.IO.FileNotFoundException: Could not load file or assembly 'System.Threading.Tasks.Dataflow, Version=4.5.24.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
File name: 'System.Threading.Tasks.Dataflow, Version=4.5.24.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'
   at Microsoft.Build.BackEnd.Logging.LoggingService.ProcessLoggingEvent(Object buildEvent, Boolean allowThrottling)
   at Microsoft.Build.BackEnd.Logging.LoggingService.LogInvalidProjectFileError(BuildEventContext buildEventContext, InvalidProjectFileException invalidProjectFileException)
   at Microsoft.Build.Evaluation.Project.ReevaluateIfNecessary(ILoggingService loggingServiceForEvaluation)
   at Microsoft.Build.Evaluation.Project.Initialize(IDictionary`2 globalProperties, String toolsVersion, String subToolsetVersion, ProjectLoadSettings loadSettings)
   at Microsoft.CodeAnalysis.MSBuild.ProjectFileLoader.<LoadProjectAsync>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.CodeAnalysis.MSBuild.ProjectFileLoader.<LoadProjectFileAsync>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.CodeAnalysis.MSBuild.MSBuildWorkspace.<LoadProjectAsync>d__40.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.CodeAnalysis.MSBuild.MSBuildWorkspace.<GetOrLoadProjectAsync>d__39.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.CodeAnalysis.MSBuild.MSBuildWorkspace.<OpenSolutionAsync>d__34.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at Microsoft.SourceBrowser.HtmlGenerator.SolutionGenerator.CreateSolution(String solutionFilePath, ImmutableDictionary`2 properties)

I solved this issue in another server by installing Visual Studio 2015 - is this required?

Need to add support for Shared Folders

  • Create a simple test solution with shared folder(s) and check it in under https://github.com/KirillOsenkov/SourceBrowser/tree/master/TestCode
  • Set HtmlGenerator project as startup (may need to unload SourceIndexServer project if running for the first time), pass the path to the test solution as argument and F5
  • Investigate the generated solution as to what is going wrong and whether it's SourceBrowser's fault or Roslyn MSBuildWorkspace's fault (or both). If there's work in Roslyn MSBuildWorkspace required, file a bug against https://github.com/dotnet/roslyn. Otherwise if it's just a source browser limitation, may file another more detailed bug here against Source Browser with the analysis and I can take a stab at fixing it then.

MSBuild fails with VS 2017 installed

When I try to run msbuild, I get the following errors:

Microsoft (R) Build Engine version 4.7.2046.0
[Microsoft .NET Framework, version 4.0.30319.42000]
Copyright (C) Microsoft Corporation. All rights reserved.

Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch.
Build started 6/1/2017 2:36:46 PM.
Project "C:\Users\Carson\Documents\sourcebrowsin\SourceBrowser\SourceBrowser.sln" on node 1 (default targets).
ValidateSolutionConfiguration:
  Building solution configuration "Debug|Any CPU".
Project "C:\Users\Carson\Documents\sourcebrowsin\SourceBrowser\SourceBrowser.sln" (1) is building "C:\Users\Carson\Docu
ments\sourcebrowsin\SourceBrowser\src\HtmlGenerator\HtmlGenerator.csproj" (2) on node 1 (default targets).
C:\Users\Carson\Documents\sourcebrowsin\SourceBrowser\src\HtmlGenerator\HtmlGenerator.csproj(2,1): error MSB4041: The d
efault XML namespace of the project must be the MSBuild XML namespace. If the project is authored in the MSBuild 2003 f
ormat, please add xmlns="http://schemas.microsoft.com/developer/msbuild/2003" to the <Project> element. If the project
has been authored in the old 1.0 or 1.2 format, please convert it to MSBuild 2003 format.
Done Building Project "C:\Users\Carson\Documents\sourcebrowsin\SourceBrowser\src\HtmlGenerator\HtmlGenerator.csproj" (d
efault targets) -- FAILED.

Project "C:\Users\Carson\Documents\sourcebrowsin\SourceBrowser\SourceBrowser.sln" (1) is building "C:\Users\Carson\Docu
ments\sourcebrowsin\SourceBrowser\src\GitGlyph\GitGlyph.csproj" (3) on node 1 (default targets).
C:\Users\Carson\Documents\sourcebrowsin\SourceBrowser\src\GitGlyph\GitGlyph.csproj(2,1): error MSB4041: The default XML
 namespace of the project must be the MSBuild XML namespace. If the project is authored in the MSBuild 2003 format, ple
ase add xmlns="http://schemas.microsoft.com/developer/msbuild/2003" to the <Project> element. If the project has been a
uthored in the old 1.0 or 1.2 format, please convert it to MSBuild 2003 format.
Done Building Project "C:\Users\Carson\Documents\sourcebrowsin\SourceBrowser\src\GitGlyph\GitGlyph.csproj" (default tar
gets) -- FAILED.

Project "C:\Users\Carson\Documents\sourcebrowsin\SourceBrowser\SourceBrowser.sln" (1) is building "C:\Users\Carson\Docu
ments\sourcebrowsin\SourceBrowser\src\SourceIndexServer.Tests\SourceIndexServer.Tests.csproj" (4) on node 1 (default ta
rgets).
C:\Users\Carson\Documents\sourcebrowsin\SourceBrowser\src\SourceIndexServer.Tests\SourceIndexServer.Tests.csproj(1,1):
error MSB4041: The default XML namespace of the project must be the MSBuild XML namespace. If the project is authored i
n the MSBuild 2003 format, please add xmlns="http://schemas.microsoft.com/developer/msbuild/2003" to the <Project> elem
ent. If the project has been authored in the old 1.0 or 1.2 format, please convert it to MSBuild 2003 format.
Done Building Project "C:\Users\Carson\Documents\sourcebrowsin\SourceBrowser\src\SourceIndexServer.Tests\SourceIndexSer
ver.Tests.csproj" (default targets) -- FAILED.

Done Building Project "C:\Users\Carson\Documents\sourcebrowsin\SourceBrowser\SourceBrowser.sln" (default targets) -- FA
ILED.

This has also happened to me before installing VS 2017 aswell.

can't build

Everytime I try to build the project, there is always 6 errors no matter what and it is bugging me so bad

add user/pass

so we can all stop worrying about our source code open in the web :D

Feature Request

Thank you for this excellent project.
Would it be possible to implement an autocomplete feature for the search box?

Subdirectory support

Hey,

first of all I would like to thank you for your nice work. My question or request is more a pull request but I was not able to create them. Is it possible to add subdirectory support? I plan to use your tool on an internal source server. But it's not possible to get a new IP-Address for each generated source view.

I tried it by my own but I think it's easier if you can arrange that because I first have to understand the whole code beforer I can do it by myself.

Thank you very much

Regards
Raphael

Add ability to disable warnings in the index logs

When running the index, we get a bunch of warnings like this one below. It would be great if the warnings can be disabled (maybe through command line), to make the logs easier to read.

System.InvalidCastException: Unable to cast object of type 'MSBuild.CheckRootFileTask' to type 'Microsoft.Build.Framework.ITask'. at Microsoft.Build.Shared.TaskLoader.CreateTask(LoadedType loadedType, String taskName, String taskLocation, Int32 taskLine, Int32 taskColumn, LogError logError, AppDomainSetup appDomainSetup, Boolean isOutOfProc, AppDomain& taskAppDomain)

Generate custom examples for overview.html

Currently the examples on overview.html are hard-coded .NET reference source examples (BitConverter, Enumerable.cs, etc.), and so the example links don't work as expected. Can this be changed to use examples from the analyzed project instead?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.