An open-source, lightweight, and portable spam classifier for cNFTs on Solana with 96% accuracy.
Can run anywhere that webassembly runs: on a server, in a lambda function, and even running entirely in your browser:
demo.1.mp4
Also included is the model training code and data, so you can train and bring your own model if the default model is not performing well.
Feature extraction is done with a combination of on-chain data and OCR using the tesseract.js library. Classification is done with naive bayes and a hand-picked set of spam
and ham
cNFTs.
You can try a live (heavily rate limited) example of the library running on AWS Lambda here:
https://api.filtoor.xyz/classify?address=A1xhLVywcq6SeZnmRG1pUzoSWxVMpS6J5ShEbt3smQJr
Try a new cNFT by replacing the address={...}
parameter. The classifier will either spit out "spam" or "ham" (or "error" if something went wrong).
If you'd like to use this API in your production project, please DM me to get set up!
First, install the library:
npm i cnft-spam-filter
then import the requisite function:
const { extractAndClassify } = require("cnft-spam-filter")
or
import { extractAndClassify } from "cnft-spam-filter"
Finally, call the function wherever you want to classify:
const classification = await extractAndClassify(assetId, rpcUrl);
Note that you'll need to bring your own rpcUrl
that supports the DAS
api--I recommend Helius for their generous free plan https://www.helius.dev/.
You can find a few lightweight examples of how to use the library in different environments in the /examples folder of the repository.
cnft-spam-filter
aims to be portable, so you can run it in pretty much any environment that you want.
You can train your own model and pass it to classify(tokens, model)
. Code for this is in the /train folder.
You'll see spam_ids.json
and ham_ids.json
there; these are the cNFTs used to train the model.
You can test the accuracy of a model using the code in the /test folder. Make sure that your training set and test set do not overlap. It should spit out a confusion matrix as well as all of the mistakes made:
![10](https://private-user-images.githubusercontent.com/157436846/302474885-edade4d2-4d70-4768-a5a0-631bda8cd9fa.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjM5MTgwNzMsIm5iZiI6MTcyMzkxNzc3MywicGF0aCI6Ii8xNTc0MzY4NDYvMzAyNDc0ODg1LWVkYWRlNGQyLTRkNzAtNDc2OC1hNWEwLTYzMWJkYThjZDlmYS5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwODE3JTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDgxN1QxODAyNTNaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT1mM2E2OGQzNGU1ZDM4MTA2ZjNhN2JkZDQzMDg0MjQ5N2IzZDk3M2NiMGM5NzQzMzAzOTA4N2E2YWJiMDI1NjkyJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.fO2gaPxxPggKTevwVFUZY-0gRB6PRQNb_s0RF__NHKs)
If you want to use cnft-spam-filter
in production, I recommend setting up a caching layer so that you don't have to analyze each cNFT multiple times. This should be done at your own app level: you can use redis, a database, localstorage--whatever you want.
Feel free to open pull requests to contribute if you think this is interesting! I will try to get to them as best as I can. There are definitely some tasks that need to be implemented.
All code is released under the MIT license -- go crazy.
Solana/USDC donations are appreciated but not required by any means:
solarnius.sol