ml5js / ml5-examples Goto Github PK
View Code? Open in Web Editor NEWA collection of ml5.js examples
Home Page: https://examples.ml5js.org
License: MIT License
A collection of ml5.js examples
Home Page: https://examples.ml5js.org
License: MIT License
This is the present code for the transfer learning example:
classifier = new ml5.ImageClassifier(video, modelLoaded);
Since we are planning to eventually have an empty ImageClassifier (see ml5js/ml5-library#121 and ml5js/ml5-library#128) I think we should require that you explicitly reference the pre-trained model. There is no default one. This is also more clear to an end-user as to what is going on.
This might change a bit if we go with the FeatureExtractor
idea, but for now the example should probably be:
classifier = new ml5.ImageClassifier('MobileNet', video, modelLoaded);
@cvalenzuela Is this already supported or do we have to add support for the above to the library?
I noticed in the LSTM Text Generator example that the text above the generate
button should change to "Generating..." but doesn't actually update until after the lstm.generate(data, gotData)
function finishes.
In the script: https://ml5js.org/docs/assets/scripts/example-lstm.js
I was able to get the "correct behavior" by wrapping the function call in a set timeout:
setTimeout(() => lstm.generate(data, gotData), 500)
I guess this gives the DOM a chance to update before the blocking call to .generate
is made? I'm not sure if this is an issue, but wanted to bring it to your attention.
I am trying to build a web-application based on the video classification example.
Following the google tensorflow-for-poets tutorial here I made a re-trained custom model based on [mobilenet_v2_1.0_224.pb].
It has 5 classes to differentiate and would really help if I can run that on the browser.
Now I have the [custom_model.pb] and the [custom_model.lite] version following these steps to optimize it for mobile usage.
The question is...
Is there a way to implement custom models in the video classification example?
I tried to dig into the original code of ml5.js but it was difficult to grasp where the models were loaded.
It seems now it only supports the default MobileNet/SqueezeNet.
Thanks for reading and lots of love to ML5.js :)
At present we have
javascript
--> example1
--> example2
p5js
--> example1
--> example2
with the idea that we'll also eventually have:
threejs
--> example1
--> example2
etc.
At the moment the p5js directory has many examples, maybe change to. . .
p5js
--> images
--> example1
--> example2
--> sound
--> example1
--> example2
--> text
--> example1
--> example2
?
Hello. I'm on working the web editor examples (as referenced in #6) here is my list so far:
Hello wonderful ml5 contributors,
ml5-examples/javascript
ml5-examples/p5js
We recently released ml5.js v0.2.3 - https://github.com/ml5js/ml5-library/tree/release - and we're in the process of getting all of our examples and website synchronized. We could some help going through to make sure that:
/p5js
and /javascript
release
branchIf you're testing the ml5-examples, you may have to change the index.html
file reference to the ml5 library. In this case, please be sure to use:
<script src="https://unpkg.com/[email protected]/dist/ml5.min.js" type="text/javascript"></script>
Thank you! π
Here you're calling a non-existent function. Should just be loop(classifier)
.
How would I go about hosting these examples on my own domain (purchased from google domains). e.g. www.example.com? rather than localhost?
An FYI that I am working on new charRNN
examples in conjunction with @memo's ml5js/ml5-library#221 over in the A2Z course. When we merge the changes we can bring the exmaples here.
https://github.com/shiffman/A2Z-F18/tree/master/week8-charRNN
In this page : https://ml5js.github.io/docs/style-transfer-image-example.html#
It is mentioned that you can train your own images but the link refers to the very same page. Is it an error or the tutorial is just not available yet ?
Found a bug in one of the examples. DevTools reporting possible memory leak.
I've used this example: https://github.com/ml5js/ml5-examples/tree/development/javascript/ImageClassification/ImageClassification_Video
and let it run overnight.
Other relevant information, if applicable
Here's some helpful information about my setup...
Thank you @joeyklee and @mayaman for the new PoseNet API and example β€οΈ
@joeyklee I think that a foreach()
loop and the =>
syntax can be confusing for beginners just used to the basics of p5. Also, I worry the drawPosition()
function (as nice as it is) might make it harder for users to be creative with the possibilities by messing around with custom drawing for different keypoints (connecting with lines, etc.) I would propose simplifying the example to something like. . .?
Let me know what you think!
let video;
let poseNet;
let poses = [];
function setup() {
createCanvas(640, 480);
video = createCapture(VIDEO);
video.size(width, height);
poseNet = ml5.poseNet(video);
poseNet.on('pose', function(results) {
poses = results;
});
video.hide();
}
function draw() {
image(video, 0, 0, width, height);
strokeWeight(2);
// For one pose only (use a for loop for multiple poses!)
if (poses.length > 0) {
let pose = poses[0].pose;
// Create a pink ellipse for the nose
fill(213, 0, 143);
let nose = pose['nose'];
ellipse(nose.x, nose.y, 20, 20);
// Create a yellow ellipse for the right eye
fill(255, 215, 0);
let rightEye = pose['rightEye'];
ellipse(rightEye.x, rightEye.y, 20, 20);
// Create a yellow ellipse for the right eye
fill(255, 215, 0);
let leftEye = pose['leftEye'];
ellipse(leftEye.x, leftEye.y, 20, 20);
}
}
When running the video classification example, I get this error :
Uncaught (in promise) Error: Requested texture size [0x0] is invalid.
Hi, any plans for person re-identification?
There is an error in LSTM examples on ml5js website that shows, which leads the example to crash:
Uncaught TypeError: ml5.LSTMGenerator is not a function
This error is appearing in two pages:
It is a problem caused by mismatching of library version (LSTMGenerator -> charRNN)?
See "Here is a complete example": https://ml5js.org/docs/Pix2Pix
I'm working on a new example for drawing with SketchRNN.
I tried https://ml5js.org/docs/word2vec-example in Chrome and FireFox.
There has been a problem loading the vocab: Unexpected token < in JSON at position 0
(anonymous) @ ml5.min.js:formatted:38924
Promise.catch (async)
value @ ml5.min.js:formatted:38923
e @ ml5.min.js:formatted:38905
t.default @ ml5.min.js:formatted:39008
(anonymous) @ example-word2vec.js:2
ml5.min.js:formatted:38975 Uncaught Error: Invalid input, vector not found for: rainbow,kitten
Just an issue to remind us to replace the "temp lib" folders with the cdn link once we push 0.1.0 to npm.
Hi there!
I'm currently working through the update to tfjs 1.0.2 1.1.2
Here is a running list of checks:
ml5-examples/javascript
ml5-examples/p5js
Mobilenet.js:374 Uncaught TypeError: this.mobilenet.infer is not a function
Mobilenet.js:374 Uncaught TypeError: this.mobilenet.infer is not a function
Mobilenet.js:374 Uncaught TypeError: this.mobilenet.infer is not a function
when I was trying to run the examples with webcam, after model loaded, this error pops up: Uncaught TypeError: Cannot read property 'length' of undefined (sketch: line 26) [copied from the web editor YOLO example]
and the PostNet example: Uncaught TypeError: ml5.PoseNet is not a constructor (sketch: line 23)
It happens when I tried to use imagenetClassification_Video, PostNET and YOLO locally. (running on chrome)
Many thanks for the work, this is a fantastic library!
As we build more examples, it would be great to automatically create an index page.
I built this page - https://joeyklee.github.io/sandbox/ - that uses the github API to get the contents of the directories in a repo and sets a URL to each example. Rather than listing all the URLs in the readme which creates overhead for maintenance, I think it makes sense to query the directories which showcase the examples in an index page.
I will make a PR on this.
Something like the screenshot below, but matching the ml5 website style.
Hi,
When I try pressing the first button ("is similar to..."), I get a message that says "Model has not loaded yet!", even after trying again, and waiting before trying as well. Thought I might point out.
Hello,
I'm getting errors when trying to add webcam images for either the FeatureExtractor_Image_Classification or FeatureExtractor_Image_Regression demos, both the p5js and javascript versions. The webcam displays in the canvas fine, but when I click on "Add Cat Image" in the classification demo, the console reports:
Uncaught (in promise) Error: pixels passed to tf.fromPixels() can not be null
at e.fromPixels (tf-core.esm.js:17)
at e.fromPixels (tf-core.esm.js:17)
at fromPixels_ (tf-core.esm.js:17)
at Object.fromPixels (tf-core.esm.js:17)
at imageUtilities.js:67
at tf-core.esm.js:17
at e.scopedRun (tf-core.esm.js:17)
at e.tidy (tf-core.esm.js:17)
at Object.t.tidy (tf-core.esm.js:17)
at t.imgToTensor (imageUtilities.js:66)
No images are added (when I click Train, I get "Error: Add some examples before training!")
This also happens to me for the online demos on the ml5 web site, https://ml5js.org/docs/custom-classifier and https://ml5js.org/docs/custom-regression
I see the problem on both Firefox and Chrome on Windows 7, using python -m http.server. Of note, the KNNClassification_Video demo does work for me.
Not sure but may be related to ml5js/ml5-library#260
Should we create a directory structure like this?
examples-p5/
examples-js/
I'm actually at a loss for what to call the "plain JavaScript" examples. Plain? Vanilla? ES2017? JavaScript? The problem of course being that p5 is also JavaScript.
Most of our image classification, style transfer examples are using <img>
elements. Would be nice to show loadImage()
and drawing image data to a canvas as well.
House cleaning - removing stale branches
The "Sentiment" example is missing the javascript file. See here.
Having to move back and forth between index.html
and sketch.js
for beginners causes confusion and errors. For the video classification example, just creating all the DOM elements within sketch.js
and/or drawing the video to the canvas instead might be a nice simplification.
(Note from #CCFestNYC)
Dear ml5 community,
I'm submitting a new issue. Please see the details below.
Did you find a bug? Want to suggest an idea for a new example?
In React, I have a simple setup for a charRNN. When I try to load my model in the src/models/mymodel, it somehow does not load the model, even though the callback triggers.
Here's some helpful screenshots and/or documentation of the new feature
Here's some example code or a demonstration of the issue in https://github.com/ml5js/ml5-examples OR in the https://editor.p5js.org or codepen/jsfiddle/etc...
clone the repo
npm i
npm start
[email protected]:zegenerative/ml5-lstm.git
Other relevant information, if applicable
Here's some helpful information about my setup...
A duplicate post for ml5js/ml5-homepage-demo#2, noting that the "I think this" and "My guess" is perhaps problematic for the classification examples. More thoughts and ideas at ml5js/ml5-homepage-demo#2, let's discuss there and then update here as well.
We used ml5js latest version '0.2.2' in which we are adding some images into classifier trained them and saved that model. After saving that model I got two files model.json and model.weights.bin in my local.
After that, we created another page where we are loading that model using 'classifier.load' method and after that we are giving one image to classify for which we used 'classifier.classify' which throws an exception
"TypeError: Cannot read property 'predict' of null
at https://unpkg.com/[email protected]/dist/ml5.min.js:106:298109
at https://unpkg.com/[email protected]/dist/ml5.min.js:35:27758
at e.scopedRun (https://unpkg.com/[email protected]/dist/ml5.min.js:35:27896)
at e.tidy (https://unpkg.com/[email protected]/dist/ml5.min.js:35:27656)
at Object.t.tidy (https://unpkg.com/[email protected]/dist/ml5.min.js:35:39151)
at e. (https://unpkg.com/[email protected]/dist/ml5.min.js:106:297986)
at x (https://unpkg.com/[email protected]/dist/ml5.min.js:106:348038)
at Generator._invoke (https://unpkg.com/[email protected]/dist/ml5.min.js:106:347826)
at Generator.e.(anonymous function) [as next] (https://unpkg.com/[email protected]/dist/ml5.min.js:106:348217)
at a (https://unpkg.com/[email protected]/dist/ml5.min.js:58:19990)"
Here is my code :
<head>
<title>Getting Started with ml5.js</title>
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.7.3/p5.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/p5.js/0.7.3/addons/p5.dom.min.js"></script>
<script src="https://unpkg.com/[email protected]/dist/ml5.min.js" type="text/javascript"></script>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
</head>
<body>
<h1>Image classification using MobileNet</h1>
<br/>
<input type="file" id="imgload" accept="image/*" />
<img src="#" id="imgshow" align="left" width="400" height="300" crossorigin="anonymous" />
<button id="upload-button" type="button" onclick="uploadButtonHandler()" reRender ="img">Predict</button>
</body>
<script>
$('document').ready(function () {
$("#imgload").change(function () {
if (this.files && this.files[0]) {
var reader = new FileReader();
reader.onload = function (e) {
$('#imgshow').attr('src', e.target.result);
}
reader.readAsDataURL(this.files[0]);
}
});
});
</script>
<script>
function uploadButtonHandler() {
let featureExtractor = ml5.featureExtractor('MobileNet');
const classifier = featureExtractor.classification();
console.log("setup classifier DONE", classifier);
const load1 ="{!URLFOR($Resource.ml5Model,'ml5Model/model.json')}";
console.log('load', load1);
setTimeout(function() {
const load = classifier.load(load1);
console.log('load1',load1);
},2000);
setTimeout(function(){
const classify = classifier.classify(document.getElementById('imgshow'), gotResults);
console.log('classify',classify);
}, 4000);
function gotResults(err, results){
if (err) {
console.error(err);
}
if (results && results[0]) {
console.log('#result',results[0].label);
console.log('#confidence',results[0].confidence);
}
}
}
</script>
Initially we were using '0.1.3' version of ml5 f, but we upgraded to '0.2.2' beaucse it giving label and confidance value. But look like there is some exception to classify.
Please look at this and suggest we write way to get the label and confidence value after loading the model.
I download the ml5-examples and unzip, then load CharRNN _Text index.html from its folder structure, not changing a thing, into the browser via my IIS web server and I get all these errors. I have tried to change the path to the models, but to no avail. Many of the other examples work fine, just not the RNN examples?
Failed to load resource: the server responded with a status of 404 (Not Found)
checkpointLoader.js:69 Uncaught Error: Not found variable Variable
at XMLHttpRequest.r.onload (checkpointLoader.js:69)
models/woolf/rnnlm_softmax_b:1 Failed to load resource: the server responded with a status of 404 (Not Found)
checkpointLoader.js:69 Uncaught Error: Not found variable rnnlm/softmax_b
at XMLHttpRequest.r.onload (checkpointLoader.js:69)
models/woolf/embedding:1 Failed to load resource: the server responded with a status of 404 (Not Found)
models/woolf/rnnlm_multi_rnn_cell_cell_0_basic_lstm_cell_bias:1 Failed to load resource: the server responded with a status of 404 (Not Found)
checkpointLoader.js:69 Uncaught Error: Not found variable embedding
at XMLHttpRequest.r.onload (checkpointLoader.js:69)
checkpointLoader.js:69 Uncaught Error: Not found variable rnnlm/multi_rnn_cell/cell_0/basic_lstm_cell/bias
at XMLHttpRequest.r.onload (checkpointLoader.js:69)
models/woolf/rnnlm_multi_rnn_cell_cell_0_basic_lstm_cell_kernel:1 Failed to load resource: the server responded with a status of 404 (Not Found)
checkpointLoader.js:69 Uncaught Error: Not found variable rnnlm/multi_rnn_cell/cell_0/basic_lstm_cell/kernel
at XMLHttpRequest.r.onload (checkpointLoader.js:69)
models/woolf/rnnlm_multi_rnn_cell_cell_1_basic_lstm_cell_kernel:1 Failed to load resource: the server responded with a status of 404 (Not Found)
checkpointLoader.js:69 Uncaught Error: Not found variable rnnlm/multi_rnn_cell/cell_1/basic_lstm_cell/kernel
at XMLHttpRequest.r.onload (checkpointLoader.js:69)
models/woolf/rnnlm_multi_rnn_cell_cell_1_basic_lstm_cell_bias:1 Failed to load resource: the server responded with a status of 404 (Not Found)
checkpointLoader.js:69 Uncaught Error: Not found variable rnnlm/multi_rnn_cell/cell_1/basic_lstm_cell/bias
at XMLHttpRequest.r.onload (checkpointLoader.js:69)
models/woolf/rnnlm_softmax_w:1 Failed to load resource: the server responded with a status of 404 (Not Found)
checkpointLoader.js:69 Uncaught Error: Not found variable rnnlm/softmax_w
at XMLHttpRequest.r.onload (checkpointLoader.js:69)
Great initiative with ML5. When trying to run the "PoseNet example with Single detection" on Chrome Version 66.0.3359.181 (Official Build) (64-bit) I get the following error:
Uncaught (in promise) Error: Requested texture size [0x0] is invalid.
at Ee (ml5.min.js:1)
at Qe (ml5.min.js:1)
at Ze (ml5.min.js:1)
at e.createMatrixTexture (ml5.min.js:1)
at e.acquireTexture (ml5.min.js:1)
at e.uploadToGPU (ml5.min.js:1)
at e.getTexture (ml5.min.js:1)
at e.fromPixels (ml5.min.js:1)
at e.fromPixels (ml5.min.js:1)
at e.fromPixels (ml5.min.js:1)
A few seconds later, I tried again and it worked. I have not changed anything aside from running the official posenet example provided by tensorflow.js in a another tab beforehand.
Both screenshots are attached...
Many thanks for your work!
Hi, i'm finding a big memory usage of the browser using the PitchDetection model. After starting pitch detection the memory usage go up slowly and it continue until i close the browser tab or the browser crashes.
Running the PoseNet example in vanilla js currently produces the following error:
Uncaught (in promise) TypeError: Failed to execute 'createObjectURL' on 'URL': No function was found that matched the signature provided.
This is possibly due to deprecated createObjectURL
method.
Following this alternative solution correctly returns the video stream, but results in a new error: the gotPoses()
callback returns undefined
, thus drawKeypoints()
can't iterate over the poses
array.
Uncaught TypeError: Cannot read property 'length' of undefined
at drawKeypoints (sketch.js:57)
at drawCameraIntoCanvas (sketch.js:37)
As suggested in #52 I've tried to debug it using different versions of ml5 (i.e. [email protected]
, [email protected]
, [email protected]
) provided by https://unpkg.com
, but to no avail.
Interestingly enough, the p5js
example seems to be woring fine...
Is this simply due to a video
element issue loading too slow like in ml5-library #137 ? Any ideas?
Thank you for the amazing library!
This is a continuation of #6 but opening a separate issue as it relates specifically to prepping material for the IMA course in ml5js/Intro-ML-Arts-IMA-F19#34.
For week 3 material, I'm providing examples for PoseNet, UNet, and BodyPix.
@joeyklee I can create web editor versions of UNet and BodyPix and put them in the ima_ml
if that's more expedient. Also are they perhaps not included b/c there is an issue with the model and the p5 web editor? I'll do some tests and report back here!
βοΈThis is the same error we ran into with image classification but since that example now works in theory we can maybe get this one to! (I might also suggest switching it to loadImage()
and use preload()
but that's a different discussion π€ͺ)
I'm working on an image classification example (like this one https://ml5js.github.io/docs/simple-image-classification-example.html) that uses a lot of images, and ideally saves the classifications to a json file.
What's the best way to read in a directory of images in p5, and iterate through them to get the image names? Is this something better suited for a regular js example, with a node server or something?
The demo is currently not working.
Looks like the API changed to featureExtractor.classification(video, videoReady);
I'm submitting a PR with a fix :)
python -m SimpleHTTPServer
codes only run on python 2,
but considering that many people are using version 3,
It's good to post the code for python3.
python -m http.server
A clear and concise description of what the issue is about. Let us know if you:
As a way to serve up our examples to the ml5-website-2, we should make sure to merge our release
branch to master
so that our latest release examples are being served up.
It looks like Github is starting to restrict from which branch we can host our github pages. Something worth noting.
We should document with the examples and in the README what will run offline and what will not. We should be clear about when a pre-trained model is being accessed via the cloud.
Hey all,
I just wrote a kmeans class for ml5-library I'll PR shortly. I have an example that uses d3 (interactive cluster assignments for different datasets) - should I create a new directory called /d3
in the PR here? Or should I just throw the example in the existing /javascript
dir?
Thanks!
Hi! Not sure if I should raise this issue here or in the library repo. The "BodyPix with Webcam" example doesn't work in p5.js instance modeβit throws ReferenceError: loadImage is not defined
.
release
<div id="p5sketch"></div>
to the html and run the modified sketch below.const s = (sketch) => {
let bodypix;
let video;
let segmentation;
let img;
const options = {
outputStride: 8, // 8, 16, or 32, default is 16
segmentationThreshold: 0.3 // 0 - 1, defaults to 0.5
}
sketch.setup = function () {
sketch.createCanvas(320, 240);
// load up your video
video = sketch.createCapture(sketch.VIDEO);
video.size(sketch.width, sketch.height);
// video.hide(); // Hide the video element, and just show the canvas
bodypix = ml5.bodyPix(video, modelReady)
}
function modelReady() {
console.log('ready!')
bodypix.segment(gotResults, options)
}
function gotResults(err, result) {
if (err) {
console.log(err)
return
}
// console.log(result);
segmentation = result;
sketch.background(0);
sketch.image(video, 0, 0, sketch.width, sketch.height)
sketch.image(segmentation.maskBackground, 0, 0, sketch.width, sketch.height)
bodypix.segment(gotResults, options)
}
}
let myp5 = new p5(s, document.getElementById('p5sketch'));
FYI the code on the ml5 website is different to the git examples. I'm assuming the git version is correct or am I missing something as to why they are not both charRNN π
https://ml5js.org/docs/lstm-interactive-example
A declarative, efficient, and flexible JavaScript library for building user interfaces.
π Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. πππ
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google β€οΈ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.