Comments (3)
Wow! It looks good
I can't wait to play with it!
I pull the development version and I tell you soon
Thx
OLivier
from pyss3.
Hi @oterrier!
Thanks for creating this issue. Yes, Issue #6 added the possibility to load multi-label datasets from disk using two common file structures. Now I'm currently working on Issue #5, which, once finished, will enable PySS3 to provide full support for multi-label classification. I've just finished working on Evaluation.test()
which now supports multilabel classification (0a897dd) 😊
Yes, it would be great to have the feature you're suggesting. Besides, I think that Live_Test.run()
should also work with no y_test
at all, sometimes you just want to test some documents manually, for instance:
docs = ["this is document 1", "this is document 2", "this is document 3"]
Live_Test.run(clf, docs)
It would also be a nice feature to have, what do you think?
In the case of multi-label classification, The only thing that I'm not entirely sure about yet is how to show the list of documents in the Live Test. So far, since each document belongs to only one category, documents are grouped by categories in the left panel, as shown in the following image:
Also, the % of hits (recall) and the "misclassified" icon (!) should be removed or replaced by something else when loading documents with multiple labels.
I think one possible solution is, divided into the following, from easiest to more complex, steps:
Step 1
When loading multilabel documents, remove everything from the left panel, and only show the plain list of all documents (no categories, no % of hits, no misclassification icon). Once a document is selected, and once classified, show the list of true category labels along with its name, in the location that is marked below:
There, we could even paint in green correctly predicted labels and in red misclassified ones.
Step 2
For each document, add the % of the total labels correctly classified. For instance, if we have the following case:
x_test = ["this is document 1", "this is document 2", "this is document 3"]
y_test = [["labelA"],
["labelB"],
["labelA", "labelB"]]
And the predicted labels are:
y_pred = [["labelB"], # misclassified
["labelB"], # hit!
["labelA"]] # 50% hit!
In the Live Test we could show the list with these 3 documents along with the % of label hits (recall), as follows:
[Live Test left Panel]
doc1 (0%)
doc2 (100%)
doc3 (50%)
Step 3
When the user moves the cursor over the %, the actual list of true labels is shown along with the predicted ones. Maybe we could use a Tooltip to accomplish this.
Step 4
Add a filter option at the top of the panel, allowing the user to filter the list of documents by category labels or even other options, like %0 of hits, for instance.
What do you think about this? It is not necessary to implement all these steps; step 1 alone will be enough to enable the user to load multilabel documents. Further steps only improve user experience.
Once I finish working on Issue #5 I'll start working on this one, however, feel free to submit any PR, since any kind of help would give me a "head start" and I'll greatly appreciate it 😎
from pyss3.
Hi again @oterrier! I've just finished implementing the multi-label support for the Live Test Tool (up to step 2 included). As described above, now, in the left panel, the list of documents is shown with no categories and when selected, the true labels are shown along with the predicted labels, as shown below:
Step two was also implemented, and each test document in the list is shown with a % corresponding to its label-based accuracy (aka hamming score). Besides, misclassified labels are shown colored in red, as "drama" below:
Let me know Oliver if this is what you needed/wanted, or if you find something that needs to be improved/changed/fixed. I'm performing the final checks before releasing the new version, which will include all the changes and improvements regarding multi-label classification 😄
from pyss3.
Related Issues (20)
- Divison by 0 HOT 4
- Initialization of sanction function HOT 6
- Custom metrics for evaluation HOT 5
- Use evaluation and explanation as a standalone package? HOT 2
- Partial learn HOT 10
- Data loading issues while train HOT 4
- [joss] update the changelog HOT 1
- [joss] update entry site of the documentation HOT 1
- [joss] feature request: accessible utility to import a dataset HOT 4
- [joss] software paper comments HOT 1
- [JOSS] comments on the paper
- AttributeError: type object 'Dataset' has no attribute 'load_from_url' HOT 3
- AttributeError: type object 'Dataset' has no attribute 'load_from_url' HOT 3
- PYSS3 support for multi-class classification
- Set custom Confidence Vectors
- Custom preprocessing in Live Test HOT 8
- Multilabel Classification Evaluation HOT 14
- Multilabel Classification Dataset Loading HOT 4
- Change of category name HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyss3.