poloclub / cnn-explainer Goto Github PK

View Code? Open in Web Editor NEW

7.4K 164.0 1.1K 88.28 MB

Learning Convolutional Neural Networks with Interactive Visualization.

Home Page: https://poloclub.github.io/cnn-explainer/

License: MIT License

CSS 0.19% HTML 0.82% JavaScript 54.56% Python 4.09% Shell 0.17% Svelte 40.17%

deep-learning visualization interactive-visualizations visual-learning machine-learning

cnn-explainer's Introduction

CNN Explainer

An interactive visualization system designed to help non-experts learn about Convolutional Neural Networks (CNNs)

For more information, check out our manuscript:

CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualization. Wang, Zijie J., Robert Turko, Omar Shaikh, Haekyu Park, Nilaksh Das, Fred Hohman, Minsuk Kahng, and Duen Horng Chau. IEEE Transactions on Visualization and Computer Graphics (TVCG), 2020.

Live Demo

For a live demo, visit: http://poloclub.github.io/cnn-explainer/

Running Locally

Clone or download this repository:

git clone [email protected]:poloclub/cnn-explainer.git

# use degit if you don't want to download commit histories
degit poloclub/cnn-explainer

Install the dependencies:

npm install

Then run CNN Explainer:

npm run dev

Navigate to localhost:3000. You should see CNN Explainer running in your broswer :)

To see how we trained the CNN, visit the directory ./tiny-vgg/. If you want to use CNN Explainer with your own CNN model or image classes, see #8 and #14.

Credits

CNN Explainer was created by Jay Wang, Robert Turko, Omar Shaikh, Haekyu Park, Nilaksh Das, Fred Hohman, Minsuk Kahng, and Polo Chau, which was the result of a research collaboration between Georgia Tech and Oregon State.

We thank Anmol Chhabria, Kaan Sancak, Kantwon Rogers, and the Georgia Tech Visualization Lab for their support and constructive feedback.

Citation

@article{wangCNNExplainerLearning2020,
  title = {{{CNN Explainer}}: {{Learning Convolutional Neural Networks}} with {{Interactive Visualization}}},
  shorttitle = {{{CNN Explainer}}},
  author = {Wang, Zijie J. and Turko, Robert and Shaikh, Omar and Park, Haekyu and Das, Nilaksh and Hohman, Fred and Kahng, Minsuk and Chau, Duen Horng},
  journal={IEEE Transactions on Visualization and Computer Graphics (TVCG)},
  year={2020},
  publisher={IEEE}
}

License

The software is available under the MIT License.

Contact

If you have any questions, feel free to open an issue or contact Jay Wang.

cnn-explainer's People

Contributors

Stargazers

Watchers

Forkers

aihill alphasldiallo lilujunai deeplearning-machinelearning barrel-titor sts-sadr selfdrivinggoudan furionzg 1803643593 dreadlord1984 qwe3208620 zuohehe useric chenwangyong poet-libai yaoceyi b-xiang prabhanjan215 kanshichao hlyu368 zkwalt xujinjin1 milan-chicago xlulu-zer0 cqulun123 gm0515 amseej rabi3elbeji catsostrong miss-bug phillweston saqibmamoon ppdouble qhapper hongyunnchen killerwuhan awoziji royzon ht-dep duoergun0729 bruinxiong shuwang8951 lincong666 uestc-hjw nkuhealong guoxinfei ruofeidu sailorbj delay-xili dsp6414 losecolor justcode996 xwyangjshb edwardwenjue cdcy tianyuluan baodijun zggl ella2le jjcheer yyqgood trendingtechnology instantwindy blackjaxx shuangzixing89 cqlpman uusing mayinjin mldl stevenhailin maguangyan kxw123456 lingyi002 hq2013 giwayne xueqing888 csu-gh songdave 1026295417 wykxyz kk-12138 youmark superchong1987 panxiong-cn mysticaltech bobycv06fpm shilaifu hjffily eblancoh ipadawan jiaodaxiaozi sun-qianyu chaoyue729 hnexplorer kleistvonliu yuyouling qqyouhappy duzhanyuan daul0124 xjliu1204

cnn-explainer's Issues

A proposition

Add the "2D" suffix to the conv's headers, it is not so obvious.

CNN explainer

Did you ever think to make a plugin like this for pytorch or tensorflow

dear wang:
Your cnn-explainer is very impressive for my study,thank you sou much.However,did you ever think to make a plugin like this for pytorch or tensorflow.so the research could visualize their cnn work procedure clearly.This is only my suggestion,thank so much for your cnn-explainer.

Pop-up window is too small

Hello! Thank you for your contribution!
I find that the pop-up window of details is too small to see the details on my laptop screen. Can I make the window full screen or zoom it in?

poloclub

Would be cool to have an explainer for an object detection pipeline as well

Possible to support loading and introspecting via ONNX.js?

Relates #2 and #8:
Might it be possible to support loading / introspecting (a subset of) models loaded via ONNX.js?
https://github.com/Microsoft/onnxjs

Curious because we would like to visualize our own PyTorch model / networks.

I'm assuming that this may be closed as out-of-scope, which is totally fine! Just wanna put it out there ;)

Whether is possible to customize the CNN-Explainer

Hi.

Thank you for sharing this excellent work. The explainer is super helpful for understanding the CNN.
Since I am not familiar with the TF.js. I wonder if I finetune the tiny-VGG on other datasets for example dog vs cat, will explainer automatically change to show how the CNN works on dog vs cat? Or I have to rewrite the part of the js to make it works. Thank you.

how to see myself model

thank you very much, I can not find any turorial how to see myself model.

error in npm run dev

(base) yosunpeng@server:~/cnn-explainer$ npm run dev

[email protected] dev /home/yosunpeng/cnn-explainer
rollup -c -w

sh: 1: rollup: not found

npm ERR! Linux 5.3.0-28-generic
npm ERR! argv "/usr/bin/node" "/usr/bin/npm" "run" "dev"
npm ERR! node v8.10.0
npm ERR! npm v3.5.2
npm ERR! file sh
npm ERR! code ELIFECYCLE
npm ERR! errno ENOENT
npm ERR! syscall spawn
npm ERR! [email protected] dev: rollup -c -w
npm ERR! spawn ENOENT
npm ERR!
npm ERR! Failed at the [email protected] dev script 'rollup -c -w'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the svelte-app package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR! rollup -c -w
npm ERR! You can get information on how to open an issue for this project with:
npm ERR! npm bugs svelte-app
npm ERR! Or if that isn't available, you can get their info via:
npm ERR! npm owner ls svelte-app
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR! /home/yosunpeng/cnn-explainer/npm-debug.log

Why is the page empty

question about changing parameter 'height'

Hi @xiaohk, I successfully managed to make cnn-explainer show a model with more layers and now I'm trying to make it work with model that has a different value of filter. I set the filter to 20 and put this new model in the cnn-explainer, the intermediate view and detail view work just as expected, but nodes are overlapping with eachother.

I checked the code in overview-draw.js and it shows:

    vSpaceAroundGap = (height - nodeLength * curLayer.length) / (curLayer.length + 1);

I think an overlay maybe means a negativ vSpaceAroundGap, so what I can do is to change the parameter 'height' but what should I do?

HTTP ERROR 403

When I run the application npm run dev, open the localhost:5000 at edge, it return HTTP ERROR 403.How can I figure it out?

Layout messed up

Hello,

Kudos to this awesome project!!! I love it.

I just downloaded the repo and I was able to run it locally. But the layout seems completely messed up. It's not as organized as the live demo. I've tried it on Chrome, Safari and Firefox and they all have the same messed layout.

I wonder if there is a way to get it fixed. Thanks

See images below:

Error in running tiny-vgg.py

Hi,

I ran the tiny-vgg.py following the conda env (supposedly all dependencies are installed), and I got error like below.
ValueError: Input 0 of layer sequential is incompatible with the layer: its rank is undefined, but the layer requires a defined rank.
I'm seeking your thoughts on what is going on. No one else seems have this issue.
Thanks for your time answering this.

vue中无法运行

当我运行程序的时候就会报错，也不知道是什么原因，哪位大神帮忙指点迷津，谢谢！
D:\works\qianduan\ywgz-mini-program>npm run dev
npm ERR! Windows_NT 10.0.19042
npm ERR! argv "D:\software\node\node_new\node.exe" "C:\Users\86152\AppData\Roaming\npm\node_modules\npm\bin\npm-cli.js" "ru
n" "dev"
npm ERR! node v16.14.0
npm ERR! npm v3.8.6
npm ERR! path D:\works\qianduan\ywgz-mini-program\package.json
npm ERR! code ENOENT
npm ERR! errno -4058
npm ERR! syscall open

npm ERR! enoent ENOENT: no such file or directory, open 'D:\works\qianduan\ywgz-mini-program\package.json'
npm ERR! enoent ENOENT: no such file or directory, open 'D:\works\qianduan\ywgz-mini-program\package.json'
npm ERR! enoent This is most likely not a problem with npm itself
npm ERR! enoent and is related to npm not being able to find a file.
npm ERR! enoent

npm ERR! Please include the following file with any support request:
npm ERR! D:\works\qianduan\ywgz-mini-program\npm-debug.log

cnn _ explainer in local host

hi ... i tried to rub the package.jason in eclipse . the message console shows the local host port . but when i open the local host in chrome the CNN explainer remains blank.

Whether possible to use the explainer for the other dataset

Hi,

Thank you for sharing this great work. I am new to the Tf.js. I wonder if I finetune the tiny-VGG to the dataset dog vs cat, can the JS automatically to explain the dog vs cat model? Or I have to rewrite the js part. Thx.

Different settings of the legend ranges after updateCNN

Hi Wang,
Your work seems fabulous for me to learn from the scratch to deeper understanding of CNN. However, the ranges for the layers doubled after I switch from one image to annther. It makes me confused for a while...

Before Switching:

After:

I thus review your codes and try to find the reason. It is not a piece of cake for me because I am used to work more on the back-end.

[-range1/2, range1/2]

During the the code review, I find there are different settings of the ranges btw function drawLegends and updateCNN.

[-range1, range1]

It is not a big issue but I still want to confirm that the first one from -range1 /2 to range1 /2 is right, isn't it?

By the way, I wonder if there is a way to present RNN with LSTM like your CNN Explainer. Could I try it based on your code for a simple RNN in sentiment task?

Thanks for your job and Hope you good health.

Damein

detail view not work with new added convolution nodes

Hallo @xiaohk, thank you very much for your reply, it is very helpful. I now fixed several problems I have met, for example the detail view of existing convolution nodes now are displayed with their former nodes and bais normally. But I still cant make new added convolution nodes show their detail view animation.

and here are the codes I changed:

const layerIndexDict = {
'input': 0,
'conv_1_1': 1,
'relu_1_1': 2,
'conv_1_2': 3,
'relu_1_2': 4,
'max_pool_1': 5,
'conv_2_1': 6,
'relu_2_1': 7,
'conv_2_2': 8,
'relu_2_2': 9,
'max_pool_2': 10,
'conv_3_1': 11,
'relu_3_1': 12,
'conv_3_2': 13,
'relu_3_2': 14,
'max_pool_3': 15,
'output': 16
}

const layerLegendDict = {
0: {local: 'input-legend', module: 'input-legend', global: 'input-legend'},
1: {local: 'local-legend-0-1', module: 'module-legend-0', global: 'global-legend'},
2: {local: 'local-legend-0-1', module: 'module-legend-0', global: 'global-legend'},
3: {local: 'local-legend-0-2', module: 'module-legend-0', global: 'global-legend'},
4: {local: 'local-legend-0-2', module: 'module-legend-0', global: 'global-legend'},
5: {local: 'local-legend-0-2', module: 'module-legend-0', global: 'global-legend'},
6: {local: 'local-legend-1-1', module: 'module-legend-1', global: 'global-legend'},
7: {local: 'local-legend-1-1', module: 'module-legend-1', global: 'global-legend'},
8: {local: 'local-legend-1-2', module: 'module-legend-1', global: 'global-legend'},
9: {local: 'local-legend-1-2', module: 'module-legend-1', global: 'global-legend'},
10: {local: 'local-legend-1-2', module: 'module-legend-1', global: 'global-legend'},
11: {local: 'local-legend-2-1', module: 'module-legend-2', global: 'global-legend'},
12: {local: 'local-legend-2-1', module: 'module-legend-2', global: 'global-legend'},
13: {local: 'local-legend-2-2', module: 'module-legend-2', global: 'global-legend'},
14: {local: 'local-legend-2-2', module: 'module-legend-2', global: 'global-legend'},
15: {local: 'local-legend-2-2', module: 'module-legend-2', global: 'global-legend'},
16: {local: 'output-legend', module: 'output-legend', global: 'output-legend'}
} // I'm not sure if I did right here

let updatingLayerIndexDict = {
local: {
module: [1, 2, 8, 9, 10, 13, 14, 15],
global: [1, 2, 3, 4, 5, 8, 9, 10, 13, 14, 15]
},
module: {
local: [1, 2, 8, 9, 10, 13, 14, 15],
global: [1, 2, 3, 4, 5, 8, 9, 10, 13, 14, 15]
},
global: {
local: [1, 2, 3, 4, 5, 8, 9, 10, 13, 14, 15],
module: [1, 2, 3, 4, 5]
}
}; // about this part I dont really understand how this works and how can people change it to adjust to another model

and I have a question about the following part:

let detailedViewAbsCoords = {
1 : [600, 270, 490, 290],
2 : [500, 270, 490, 290],
3 : [700, 270, 490, 290],
4 : [600, 270, 490, 290],
5 : [650, 270, 490, 290],
6 : [775, 270, 490, 290],
7 : [100, 270, 490, 290],
8 : [60, 270, 490, 290],
9 : [200, 270, 490, 290],
10: [300, 270, 490, 290],
}
what does this part mean? should I change this part to adjust to my model? and how?

vpn

can't open https://poloclub.github.io/cnn-explainer/ if no vpn.

How we can explain neuron if we do use cnn?

How we can explain the neural networks when we do not apply cnn?
For example in Tensorflow Playground, how do they illustrate the neuron in each layer?

Update the number of layer

Hey @xiaohk, Thank you for your answer. The solution was simpler, I just had an error in the generation of my model.json file, with a shape at 1690 for 10 layers instead of 1183 for 7 ...
However, I had to adapt lines :
replace :

cnn-explainer/src/overview/flatten-draw.js

Line 859 in b9c1759

for (let i = 0; i < 10; i++) {

for (let i = 0; i < nodeCoordinate[curLayerIndex - 1].length; i++) {

cnn-explainer/src/overview/flatten-draw.js

Line 879 in b9c1759

textY = nodeCoordinate[curLayerIndex - 1][9].y + nodeLength / 2;

textY = nodeCoordinate[curLayerIndex - 1][nodeCoordinate[curLayerIndex - 1].length - 1].y + nodeLength / 2;

cnn-explainer/src/overview/flatten-draw.js

Line 345 in b9c1759

let loopFactors = [0, 9];

let loopFactors = [0, nodeCoordinate[curLayerIndex - 1].length - 1];

after add :

cnn-explainer/src/overview/flatten-draw.js

Lines 998 to 1002 in b9c1759

    
           export const drawFlatten = (curLayerIndex, d, i, width, height) => { 
        
             // Show the output legend 
        
             svg.selectAll('.output-legend') 
        
               .classed('hidden', false);

let maxLayer = nodeCoordinate[curLayerIndex - 1].length - 1;

replace :

cnn-explainer/src/overview/flatten-draw.js

Line 1096 in b9c1759

let bottomY = nodeCoordinate[curLayerIndex - 1][9].y + nodeLength -

let bottomY = nodeCoordinate[curLayerIndex - 1][maxLayer].y + nodeLength -

cnn-explainer/src/overview/flatten-draw.js

Line 345 in b9c1759

let loopFactors = [0, 9];

let loopFactors = [0, maxLayer];

cnn-explainer/src/overview/overview-draw.js

Lines 649 to 650 in b9c1759

    
                   svgPaddings.top + vSpaceAroundGap * (10) + vSpaceAroundGap + 
        
                   nodeLength * 10

svgPaddings.top + vSpaceAroundGap * (classLists.length) + vSpaceAroundGap +
nodeLength * classLists.length

It's ok it works perfectly. Good job !

Originally posted by @davy-blavette in #36 (reply in thread)

If I want to show my model, where should I change it

The explainer won't show up in the browser

Hi!
First of all I really do appreciate you and your team hard work!
But, a problem occurred in my browser, Well the rest of the page is working smoothly,
Unfortunately the explainer wouldn't show up.. is there anything that I could do?
I've also try to open it locally, but still the problem occurred :(

Relu layer bug

There is a bug in the max of the Relu layer here

Can I use cnn-explainer on auto-encoder model?

I'm currently using a deep auto-encoder model to learn state representation from the pixel, can I use this cnn-explainer to get a better understanding of the training process?

If so, what changes should be made? Could u please give me some guidlines~

Thanks 😁

run error

I fellow your steps,and got the error:

So,please help me to fix it,thx

[Feature] Shipping in a docker container

Making a docker file or submitting the project with dependencies to a container registry would improve easy usability.

更换图片重新训练后却没有变化

为什么更换了图片重新训练之后，生成的bin文件和model.json文件将原来的两个文件替换之后，用npm run dev打开网页之后却没有变化，不是图片没有变换而是训练的结果output没有变换

detail view problem

I added new layers to the tiny-VGG and got a new model. When I use CNN-Explainer to load this model, there came out the problem:

First there are no more connection lines between two nodes.

And after I click a convolutional node(for example conv_1_* and conv_2_* ), then no Bias and no former unit convoluntion animation are showed.

furthermore I finde that detail view juts doesn't work with those new added layers.

I don't know how to fix this problem can anyone help?

fatal: Could not read from remote repository.

Dear poloclub,
I run into this error msg. Could you let me know how to solve it? thanks a lot

G:\PY\cnnexplainer>git clone [email protected]:poloclub/cnn-explainer.git
Cloning into 'cnn-explainer'...
The authenticity of host 'github.com (140.82.118.3)' can't be established.
RSA key fingerprint is SHA256:nThbg6kXUpJWGl7E1IGOCspRomTxdCARLviKw6E5SY8.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'github.com,140.82.118.3' (RSA) to the list of known hosts.
[email protected]: Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

failed to create enviroment

For the image style transfer task

a suggestion for Deployment on server

Original package.json is not friendly for deploment. We need to use the IP (e.g., 64.69.41.183:3000) instead of localhost (e.g., localhost:3000) to allow accessing the server's webpage through a local browser. Specifically, it's crucial to modify the code in the package.json file.

--------before-------------------------------
"scripts": {
"build": "rollup -c",
"dev": "rollup -c -w",
"start": "sirv public --single",
"start:dev": "sirv public --single --dev --port 3000",
"deploy": "npx"
}

------after-------------------------------------------
"scripts": {
  "build": "rollup -c",
  "dev": "rollup -c -w",
  "start": "sirv public --single --host 0.0.0.0",
  "start:dev": "sirv public --single --dev --host 0.0.0.0 --port 3000",
  "deploy": "npx"
}

error in npm run dev

hi
I ran the project following the conda env (supposedly all dependencies are installed), and I got error like below.

[!] Error: Package subpath './compiler.js' is not defined by "exports" in D:\Users\WangST2\Desktop\workspace\study_dir\cnn-explainer-master\node_modules\svelte\package.json
Error [ERR_PACKAGE_PATH_NOT_EXPORTED]: Package subpath './compiler.js' is not defined by "exports" in D:\Users\WangST2\Desktop\workspace\study_dir\cnn-explainer-master\node_modules\svelte\package.json
at throwExportsNotFound (internal/modules/esm/resolve.js:290:9)
at packageExportsResolve (internal/modules/esm/resolve.js:513:3)
at resolveExports (internal/modules/cjs/loader.js:432:36)
at Function.Module._findPath (internal/modules/cjs/loader.js:472:31)
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:867:27)
at Function.Module._load (internal/modules/cjs/loader.js:725:27)
at Module.require (internal/modules/cjs/loader.js:952:19)
at require (internal/modules/cjs/helpers.js:88:18)
at Object. (D:\Users\WangST2\Desktop\workspace\study_dir\cnn-explainer-master\node_modules\rollup-plugin-svelte\index.js:11:4)
at Module._compile (internal/modules/cjs/loader.js:1063:30)

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] dev: rollup -c -w
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the [email protected] dev script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

I'm seeking your thoughts on what is going on. No one else seems have this issue.
Thanks for your time answering this.

How to display images on git pages with cnn explainer

I am training on tiny-vgg using a plant image dataset and am trying to display it in browser in cnn-explainer.
I could run npm run dev command and see it in my local environment, but when I run npm run build command and try to view it on github pages it doesn't work.
Only the layer part of the neural network is not displayed. What should i do? Please tell me how to solve it.
https://nikamasa51-d4c7cc0837.drafts.github.io/Study_CNN_Explainer/

创建环境失败

你好，请问这个创建环境失败该怎么解决呢？

Why is the interface blank?

Your work is really good!
I want to study it, but the browser interface is blank.How could I fix the problem? Thank you!!!

about retraining cnn explainer

Hello,it's really happy using cnn explainer.Yet I have a question,how could we use it to train a cnn for things like sword with around 200 kinds?

ReLu function max(0,a) gives 0

Using Ladybug pic as an example, the ReLU function after the top one layer (orange colour) gives 0 when the max value is not zero. For example, I got max(0,0.64)=0. Should it be 0.64? Thank you.

my interface is also blank

i check all issues involved such bug， but i still failure it.

run with static ip

HOST=ip npm run dev

is another way more secure or quick for setup?

no CNN Explainer running in broswer

Hi, I have download git and npm,
and the cnn-explainer was downloaded successfully,
then I used npm run dev, it showed "you application is ready" !

but when I input "localhost:5000" in my broswer, there is nothing in my broswer,

I don't know where was wrong.
Please help me, thank you.

What should I do if I want to change the model?

How can i change the input_image_size ?

I am trying to change input_image_size(the default value is 64). but I am not familiar with front-end module.
Is there anyone who can help me?

	export const drawFlatten = (curLayerIndex, d, i, width, height) => {
	// Show the output legend
	svg.selectAll('.output-legend')
	.classed('hidden', false);

	svgPaddings.top + vSpaceAroundGap * (10) + vSpaceAroundGap +
	nodeLength * 10

poloclub / cnn-explainer Goto Github PK

cnn-explainer's Introduction

CNN Explainer

Live Demo

Running Locally

Credits

Citation

License

Contact

cnn-explainer's People

Contributors

Stargazers

Watchers

Forkers

cnn-explainer's Issues

I fellow your steps,and got the error:

Recommend Projects

Recommend Topics

Recommend Org