bpmct / jetbrains-indexer Goto Github PK

View Code? Open in Web Editor NEW

24.0 2.0 11.0 173 KB

Generate & package JetBrains shared indexes with a Docker container.

Home Page: https://coder.com/blog/faster-jetbrains-ides-with-shared-indexes

License: MIT License

Dockerfile 53.88% Shell 46.12%

jetbrains ide intellij-idea pycharm goland indexing shared-indexes

jetbrains-indexer's Introduction

jetbrains-indexer

Generate & package JetBrains shared indexes with a Docker container.

Shared indexes are often hosted on a CDN and used by IDEs to speed up loading (indexing) time for JetBrains IDEs (IntelliJ IDEA, PyCharm, GoLand, etc). Blog post: https://coder.com/blog/faster-jetbrains-ides-with-shared-indexes

Basic usage

Generate indexes for your project

cd your-project/

docker run -it --rm \
    -v "$(pwd)":/var/project \
    -v "$HOME/indexes-out":/shared-index \
    -e INDEXES_CDN_URL=http://localhost:3000/project \
    bencdr/indexer:idea-2021.3
    
# you may need to fix the file permissions for the generated indexes
sudo chown -R $(id -u):$(id -g) $HOME/indexes-out

Upload indexes to CDN (or test locally)
```
# test with local Python server
cd $HOME/indexes-out/
python3 -m http.server 3000
```
this URL must be the same as INDEXES_CDN_URL in step 1.

Add intellij.yaml to your project if you don't have one

sharedIndex:
  project:
    - url: http://localhost:3000/project

Open your IDE and test (use File → Invalidate Caches to load indexes for the first time again)

IDE support

By default, this project indexes version 2021.3 of your IDE. Specify the IDE name by using the appropriate tag (e.g bencdr/indexer:[ide-name]-2021.3). You can verify with DockerHub.

If an IDE/version is not on DockerHub, we recommend you manually pulling and building the image yourself using these build arguments.

Docs and examples

⚠️ Many of these are a work in progress

Troubleshooting

See ./docs/troubleshooting.md

If you just want to generate raw shared indexes with Docker, you can use damintsew/idea-shared-index-dockerfile. This project is based on that.

jetbrains-indexer's People

Contributors

Stargazers

Watchers

Forkers

mortenscheel janisz duda-co wuwenyao mafio69 tupywebteam icassell dockergiant sunznx julianwki wikia

jetbrains-indexer's Issues

Currently not working

`Status: Downloaded newer image for bencdr/indexer:idea-2021.3

Generating indexes

2022-01-25 13:04:16,296 [ 63] WARN - llij.ide.plugins.PluginManager - id redefinition ([row,col,system-id]: [2,3,"product classpath"])
Jan 25, 2022 1:04:17 PM java.util.prefs.FileSystemPreferences$1 run
WARNING: Couldn't create user preferences directory. User preferences are unusable.
Jan 25, 2022 1:04:17 PM java.util.prefs.FileSystemPreferences$1 run
WARNING: java.io.IOException: No such file or directory
Jan 25, 2022 1:04:17 PM java.util.prefs.FileSystemPreferences$6 run
WARNING: Prefs file removed in background ?/.java/.userPrefs/prefs.xml
Dump Shared Index command IU-213.5744.223

Failed to parse commandline: /shared-index/output

Usage:
[idea] dump-shared-index [command] [options]

The "project" command help:
Generates shared index for a given project

Usage:
[idea] dump-shared-index project [options]

options:
--output= Location of the indexes output directory
it will be updated with generated new indexes

--project-dir=<value>         Project home directory

--commit=<value>              [optional] Source Version Control revision

--compression=<value>         [optional] Compress shared index using: xz, gzip, plain

--configure-project=<value>   [optional] Call IntelliJ project configurators

--convert-project=<value>     [optional] Call IntelliJ version converters

--project-id=<value>          [optional] Project identifier

--use-xz-compression=<value>  DEPRECATED. Please use 'compression=xz' or 'compression=plain instead
                              [optional] Shared index can be additionally compressed with xz

2022-01-25 13:04:18,504 [ 2271] WARN - l.NotificationGroupManagerImpl - Notification group CodeWithMe is already registered (group=com.intellij.notification.NotificationGroup@5ec78e22). Plugin descriptor: PluginDescriptor(name=Code With Me, id=com.jetbrains.codeWithMe, descriptorPath=plugin.xml, path=/opt/idea-IU-213.5744.223/plugins/cwm-plugin, version=213.5744.223, package=null, isBundled=true)

Formatting indexes

Running the 'cdn-layout-tool' tool 0.8.65
Collected 0 index.json groups
Collected 0 index.json groups
Adding 0 items...
mv: cannot stat '/shared-index/output': No such file or directory

Creating intellij.yaml file

Wrote the following to intellij.yaml:

sharedIndex:
project:
- url: http://localhost:3000/project

Be sure to commit this file to source control so others can use it.
`

Files owned by root

I know it probably goes without saying, but maybe you could add a note to the readme mentioning that the files created by the container will be owned by root, including some files in the project .idea folder.

I just ran sudo find . -user root -exec chown $USER:$USER {} \; in the project folder to fix it.

guide for shared indexes with CDN

Is that possible

Chexk this please AndroidIDEOfficial/AndroidIDE#1677

guide for shared indexes on filesystem

You can use the file:// schema to access a shared index tarball locally :)

guide for shared indexes with Coder

build & push images for different IDEs with GitHub actions

e.g bencdr/indexer:intellij-2021.3, bencdr/indexer:webstorm-2021.3 etc

Final index compression time

Hello,

We have final index *.ijx.xz about 330Mb. Its compression time takes about 10mins. Compression runs in 1 thread.

17:13:34  Compressing                                                                                       0%
17:13:34  Compressing                                                                                       1%
17:13:44  Compressing                                                                                       2%
17:13:50  Compressing                                                                                       4%
...........
...........
17:22:55  Compressing                                                                    .................  98%
17:23:03  Compressing                                                                    .................  99%
17:23:05  Compressing                                                                    .................. 100%
17:23:05  Finalizing indexes

Could you pls optimise that?

Thank you in advance!

Pass -Xmx in convenient way