nsfwspy / nsfwspy.net Goto Github PK
View Code? Open in Web Editor NEWA .NET image and video classifier used to identify explicit/pornographic content written in C#.
License: MIT License
A .NET image and video classifier used to identify explicit/pornographic content written in C#.
License: MIT License
Hey, there's a minor issue like this, when i try to classify some "tough for AI neutral images":
here's the image used for this call:
here's my piece of code using your NuGet:
public NsfwSpyResult Classify(IFormFile file)
{
var nsfwSpy = new NsfwSpy();
using (var ms = new MemoryStream())
{
file.CopyTo(ms);
var fileBytes = ms.ToArray();
var result = nsfwSpy.ClassifyImage(fileBytes);
return result;
}
}
Thank you for this repo. Is there any possibility to get access to the dataset used for training the network?
Apparently it detects YouTube links as nsfw? And pretty much any imgur or tenor link
Hi,
Thanks for sharing this, it's pretty useful.
Could you advise how I can add additional classification?
I would like to add and classify gambling related images.
Also I've tried to run the Train project but getting an error on following line;
var trainedModel = trainingPipeline.Fit(trainSet);
And the error message is;
System.ArgumentOutOfRangeException: 'Only one class found in the LabelAsKey column. To build a multiclass classification model, the number of classes needs to be 2 or greater Arg_ParamName_Name'
Many thanks
Im running a discord bot on railway.app, The bot automatically disconnects and reconnects while classifying the image.
var nsfwImagedetector = new NsfwSpy(); var uri = new Uri(message.Attachments.First().Url); var result = nsfwImagedetector.ClassifyImage(uri); float resultFloat = result.Sexy + result.Hentai + result.Pornography; bool isImageNsfw = resultFloat > 0.992000 ? true : false; Console.WriteLine(resultFloat); Console.WriteLine(isImageNsfw); if (isImageNsfw) { var nsfwUpdate = Database.createUpdateSet("nsfw", true); await cardCollection.UpdateOneAsync(filter, nsfwUpdate); }
These are some errors i found while classifying image. although it gives result regardless of errors. Only issue is crashing the console application
`2023-01-07 13:07:39.898091: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory
2023-01-07 13:07:39.921942: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2023-01-07 13:07:40.576038: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-01-07 13:07:40.728653: I tensorflow/core/platform/profile_utils/cpu_utils.cc:104] CPU Frequency: 2200000000 Hz
2023-01-07 13:07:40.733302: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fbea0005c30 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2023-01-07 13:07:40.733356: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
2023-01-07 13:07:40.851229: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2023-01-07 13:07:40.851288: W tensorflow/stream_executor/cuda/cuda_driver.cc:312] failed call to cuInit: UNKNOWN ERROR (303)
2023-01-07 13:07:40.851336: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (railway): /proc/driver/nvidia/version does not exist
`
Thanks for a great work. I just tried to add this package to my test C# API and failed to run it on MacOS Monterey.
After successfully adding to our project, when the code hit NSFW Spy, it exits immediately without any error. Any tips would be appreciated.
SciSharp.TensorFlow.Redist
Runtime Environment:
OS Name: Mac OS X
OS Version: 12.4
OS Platform: Darwin
RID: osx.12-x64
Base Path: /usr/local/share/dotnet/x64/sdk/6.0.105/
Host (useful for support):
Version: 6.0.5
Commit: 70ae3df4a6
.NET SDKs installed:
3.1.419 [/usr/local/share/dotnet/x64/sdk]
5.0.408 [/usr/local/share/dotnet/x64/sdk]
6.0.105 [/usr/local/share/dotnet/x64/sdk]
On my system, I have nVidia Drivers installed, but no nVidia GPU at the moment. It crashes when it finds the driver but not apparently some magic files.
2022-05-27 22:50:07.900705: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcudart.so.10.1'; dlerror: libcudart.so.10.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /nix/store/gka59hya7l7qp26s0rydcgq8hj0d7v7k-gcc-11.3.0-lib/lib:/nix/store/p9d5i8nhx5g77h6mjfs8rqfl2cq33jrr-libsecret-0.20.5/lib:/nix/store/9757vpby0rqzlbm2p02dm812h28djrg5-e2fsprogs-1.46.5/lib:/nix/store/l55cmvhfrqyxhdvgcfxgq0jpizcdp7ld-libnotify-0.7.12/lib:/nix/store/m7d2jrrwprb4n48ddccsyg2ffqr6yj0m-pipewire-0.3.51-jack/lib
2022-05-27 22:50:07.900735: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2022-05-27 22:50:08.072930: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use the following CPU instructions in performance-critical operations: AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-05-27 22:50:08.091265: I tensorflow/core/platform/profile_utils/cpu_utils.cc:104] CPU Frequency: 3593110000 Hz
2022-05-27 22:50:08.091881: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f92340093e0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2022-05-27 22:50:08.091913: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
2022-05-27 22:50:08.093389: W tensorflow/stream_executor/platform/default/dso_loader.cc:59] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory; LD_LIBRARY_PATH: /nix/store/gka59hya7l7qp26s0rydcgq8hj0d7v7k-gcc-11.3.0-lib/lib:/nix/store/p9d5i8nhx5g77h6mjfs8rqfl2cq33jrr-libsecret-0.20.5/lib:/nix/store/9757vpby0rqzlbm2p02dm812h28djrg5-e2fsprogs-1.46.5/lib:/nix/store/l55cmvhfrqyxhdvgcfxgq0jpizcdp7ld-libnotify-0.7.12/lib:/nix/store/m7d2jrrwprb4n48ddccsyg2ffqr6yj0m-pipewire-0.3.51-jack/lib
2022-05-27 22:50:08.093408: W tensorflow/stream_executor/cuda/cuda_driver.cc:312] failed call to cuInit: UNKNOWN ERROR (303)
2022-05-27 22:50:08.093431: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (uGamingPC): /proc/driver/nvidia/version does not exist
Unhandled exception. HeyRed.Mime.MagicException: Could not find any valid magic files!
at HeyRed.Mime.Magic..ctor(MagicOpenFlags flags, String dbPath)
at HeyRed.Mime.MimeGuesser.GuessMimeType(Byte[] buffer)
at HeyRed.Mime.MimeGuesser.GuessFileType(Byte[] buffer)
at NsfwSpyNS.NsfwSpy.ClassifyImage(Byte[] imageData)
at NsfwSpyNS.NsfwSpy.ClassifyImage(Uri uri, WebClient webClient)
at IsItYiff.Program.Main(String[] args) in /home/krutonium/RiderProjects/IsItYiff/IsItYiff/Program.cs:line 11
Thanks for your great library but i use this library in web api project and it take a long time and at the end it cant predict anything. i run your test code and its not work.
Trying to write a small quick command line program to scan a disk for NSFW images and report what it finds, all very simple stuff using .NET 6.
"Directory.GetFiles" to get a list of file names to check, then new up an NsfwSpy object, then a foreach to classify each found image in the list.
Problem is, when the program gets to the "var nsfwSpy = new NsfwSpy();" line, it pauses for a few seconds then just exits.
I've single stepped in as far as I can and it seems to be in the file NsfwSpy.cs at around line 28 where it creates a new MLContext with the model zip file.
The model is present, all the required DLL's are present, no errors, no exceptions or anything are given, just a silent exit and the program stops executing at that point.
Win10 64 bit, developing using JetBrains rider with .NET V6 and NsfwSpy added using NuGet (Version 3.4.3)
Any advice?
I want to transfer it to tflite model , use it in android project,thanks
Created a dumb console app (.NET 6) with the following
static void Main(string[] args)
{
Console.WriteLine("Hello, World!");
var nsfwSpy = new NsfwSpy();
var result = nsfwSpy.ClassifyVideo(@"D:\movies for checking\Edit.mp4");
Console.ReadKey();
}
It seems that the code execution never comes out of using (var collection = new MagickImageCollection(video, MagickFormat.Mp4))
No exceptions or anything, i even left it running overnight thinking it might be taking long (the Edit.mp4 is first 10 minute clip of God Father Part 2), I cut it only to 10 min thinking it might be the size/time of all the movie being at (3h:22m run time and 1.4GB).
Here is what my console looks like as the program is running.
I have a 3070 running, with all the libraries installed properly (I think)
Hello, World!
2023-11-29 09:49:57.528567: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_101.dll
2023-11-29 09:49:57.727815: I tensorflow/core/platform/cpu_feature_guard.cc:142] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN)to use the following CPU instructions in performance-critical operations: AVX2
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-11-29 09:49:57.739943: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2602ddd4620 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2023-11-29 09:49:57.740026: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
2023-11-29 09:49:57.741758: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library nvcuda.dll
2023-11-29 09:49:57.758938: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1716] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: GeForce RTX 3070 computeCapability: 8.6
coreClock: 1.725GHz coreCount: 46 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 417.29GiB/s
2023-11-29 09:49:57.759034: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_101.dll
2023-11-29 09:49:58.152471: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cublas64_10.dll
2023-11-29 09:49:58.200779: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cufft64_10.dll
2023-11-29 09:49:58.228869: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library curand64_10.dll
2023-11-29 09:49:58.447674: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusolver64_10.dll
2023-11-29 09:49:58.677289: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cusparse64_10.dll
2023-11-29 09:49:58.928462: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudnn64_7.dll
>>>>>2023-11-29 09:49:58.928755:** I tensorflow/core/common_runtime/gpu/gpu_device.cc:1858] Adding visible gpu devices: 0
2023-11-29 09:51:52.426382: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1257] Device interconnect StreamExecutor with strength 1 edge matrix:
>>>>>>2023-11-29 09:51:52.426450:** I tensorflow/core/common_runtime/gpu/gpu_device.cc:1263] 0
2023-11-29 09:51:52.426524: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1276] 0: N
2023-11-29 09:51:52.426887: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1402] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6591 MB memory) -> physical GPU (device: 0, name: GeForce RTX 3070, pci bus id: 0000:01:00.0, compute capability: 8.6)
2023-11-29 09:51:52.429524: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x2606633c450 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2023-11-29 09:51:52.429562: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): GeForce RTX 3070, Compute Capability 8.6
I think u should be upgrading to .NET 6 at least (preferably 8 as 2.0 is deprecated), also note the time in bold in the above, it took a while for it to get to var collection = new MagickImageCollection(video, MagickFormat.Mp4)
Seems like the SciSharp, Magick, and Microsoft.ML can also be upgraded to later versions
I also wanted to do a PR where I add a progress event to the video processing so one can know how its going, below is the code if u wanna incorporate.
//Code Added
public delegate void UpdateVideoProgressDelegate(int currentFrame, int totalFrames);
public event UpdateVideoProgressDelegate UpdateVideoProgress;
public NsfwSpyFramesResult ClassifyVideo(byte[] video, VideoOptions videoOptions = null)
{
if (videoOptions == null)
videoOptions = new VideoOptions();
if (videoOptions.ClassifyEveryNthFrame < 1)
throw new Exception("VideoOptions.ClassifyEveryNthFrame must not be less than 1.");
var results = new ConcurrentDictionary<int, NsfwSpyResult>();
var watch = Stopwatch.StartNew();
using (var collection = new MagickImageCollection(video, MagickFormat.Mp4))
{
Console.WriteLine($"Loading video took {watch.Elapsed} to load video into collection of images");
collection.Coalesce();
var frameCount = collection.Count;
Parallel.For(0, frameCount, (i, state) =>
{
if (i % videoOptions.ClassifyEveryNthFrame != 0)
return;
if (state.ShouldExitCurrentIteration)
return;
var frame = collection[i];
frame.Format = MagickFormat.Jpg;
var result = ClassifyImage(frame.ToByteArray());
results.GetOrAdd(i, result);
//Code Added
if (videoOptions.EmitProgress)
{
UpdateVideoProgress?.Invoke(i, frameCount);
}
// Stop classifying frames if Nsfw frame is found
if (result.IsNsfw && videoOptions.EarlyStopOnNsfw)
state.Break();
});
}
var resultDictionary = results.OrderBy(r => r.Key).ToDictionary(r => r.Key, r => r.Value);
var gifResult = new NsfwSpyFramesResult(resultDictionary);
return gifResult;
}
Also how long does it take to process a 10 min vid in general.
When attempting to build the NsfwSpy web app when using v18.6.0+ / v16.17.0+ of Node.js, it will throw out a "ERR_LOADER_CHAIN_INCOMPLETE" error and get stuck trying to run 'npm install' on loop. Node.js v16.16.0 works perfectly fine due to "ERR_LOADER_CHAIN_INCOMPLETE" not existing as an error in previous versions.
I've used NsfwSpy successfully on a few test files , I'm now trying to set it up where the classification is written to a Dictionary.
Based on the results I want to run a function that moves the file into a corresponding directory based on the classification.
The issue I'm having currently is because I don't want to mess with the nuget package, all the images in a directory and classified and then the Dictionary is ready and files are moved. But if there is an error during classification the program crashes and the sorting function isn't used.
Question is , what is the best way to break up the lamda function so that I can sort files based on the classification? It will make it quiet a bit faster too so I can sort each file as it's classified instead of writing the entire list , then reading the entire list to sort.
Good thing, you could create compatibility with net framework 4.6.2
Straight up question I believe, but I'm interested in knowing if there are plans for a 'public' build availability
Hi there, I would like to host NsfwSpy inside a simple WebAPI controller to determine whether or not incoming images are NSFW. My question - if I register NsfwSpy in the DI system as Scoped, will new instances also consume resources? I.e. if one instance consumes ~600mb RAM, will 10 simultaneous incoming classification requests require ~6gb of RAM? Alternatively, can I safely register NsfwSpy as a DI Singleton without encountering threading issues or similar? What do you recommend as a best practice for this situation? Note - I am happy processing all incoming requests sequentially, my workload does not require parallel classification at this stage.
And thank you very much for providing this package!
I would like it to classify sexy / explicit images better
Are there any newer models?
Currently used model version is : 1.7.1+4cf622eb13c07947a38e6e3336221657007e635d
Here some incorrectly classified images. these are supposed to be sexy but they are not classified such and many more
for example this site much better at classification with best model : https://nsfwjs.com/
https://github.com/NsfwSpy/NsfwSpy.NET/assets/19240467/b410003d-1cbb-482f-ba00-744740aed93d
https://github.com/NsfwSpy/NsfwSpy.NET/assets/19240467/21653348-5ff8-4260-8f04-488ec19bc196
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.