cooperdk / yapo-e-plus Goto Github PK

I have a fairly large selection of content. I have manually renamed most of my content so that it’s fairly comprehensible. There are site-to-site inconsistencies with filename schemes, and some sites re-release or distribute to another site. Not only is manually managing file names a PITA, it’s also hard to find duplicates of the content as different files may have different resolutions, file sizes, encodings, or dating. This is further complicated by the fact that sometimes alternates are sometimes desirable (e.g. I may want to keep a VR version and non-VR version of the same scene, or a 5K version and a lower resolution version).

I would prefer to an automatic file managing scheme that I can adjust based on tags, to fit the renaming to my specific preferences.

I have files that are released by a particular site that I would generally like to use one filename scheme: <site> - <yy>.<mm>.<dd> - <optional scene name> - <actress1>, <actress2>, & <actress3>.ext
.\VR\<site> - <yy>.<mm>.<dd> - <optional scene name> - <actress1>, <actress2>, & <actress3>.ext

Whereas some other content that are movies with just a Title, that I would like to add information to the filename: .\Movies\<Title> (<YYYY>) - <actress1>, <actress2>, & <actress3>.ext

Ideally, the organizer could look at a specific folder to find content to organize into my library, and be triggered manually or watch the folder and work completely automatically.

It would be nice to be able to launch a scene in VLC from grid view without having to first open the page for the scene. Perhaps a VLC button could be overlaid in the corner of the thumbnail on hover.

It would be useful to see how many videos are in the library or how many results a search returned (for checking if the number of scenes actually matches the number of files in the library)

Sometimes I get lazy when adding scenes and forget to tag actors right away. I'd like to be able to search all scenes that are missing actors so I can make sure I tag them properly.

As above, hitting "Rescan folder" will add scenes that have been removed via the trash button. (guess you would need to keep track of hashes that have been removed)

Describe the bug
I think the software is checking duplicates wrong or in a wrong order in certain constellations. When it deletes the files from the drive, there is no other version of this file left.

To Reproduce
Steps to reproduce the behavior:

1. Add for example the whole drive "D:\" -> done
2. Delete D:\ from software without cleaning database.
   2.1. Moving some files around.
3. Add Sub-folder of D:\ instead of whole drive, because more flexible, e.g. "D\sorted".
4. Run duplicate check

Expected behavior
The software recognizes the new added scenes as "duplicates", deleting either the new ones or the old ones, but leaving none in the process, resulting in a data loss.

Recommendation besides looking into the feature again: Safety measure: moving to trash bin instead of deleting them completely.

Here a screenshot of my now empty folder for the lulz. :D

Describe the bug
YAPO e+ stops adding files after some (a lot) of files.

EDIT: It continues adding files after restarting YAPO and re-adding the folder.

Scanning...   --> A result was returned using method 1. Parsing JSON...
Internal Server Error: /add-items/
Traceback (most recent call last):
  File "django\core\handlers\exception.py", line 34, in inner
    response = get_response(request)
  File "django\core\handlers\base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "django\core\handlers\base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "django\views\decorators\csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "django\views\generic\base.py", line 71, in view
    return self.dispatch(request, *args, **kwargs)
  File "rest_framework\views.py", line 505, in dispatch
  File "rest_framework\views.py", line 465, in handle_exception
  File "rest_framework\views.py", line 476, in raise_uncaught_exception
  File "rest_framework\views.py", line 502, in dispatch
  File "views.py", line 1329, in get
  File "addScenes.py", line 64, in get_files
  File "addScenes.py", line 262, in create_scene
  File "scrapers\scanners.py", line 148, in tpdb
IndexError: list index out of range
"GET /add-items/?actorsToAdd=&createSampleVideo=false&folderToAddPath=D:%5C%5C&sceneTagsToAdd=&websitesToAdd= HTTP/1.1" 500 15555

Thanks for the info,

I tried the fix but I'm afraid it hasn't worked.

When I run the container the error message still appears and the container stops.

Originally posted by @jason4bury in #92 (comment)

Describe the bug
I can't pull docker image

To Reproduce
Steps to reproduce the behavior:
docker pull cooperdk/yapo-eplus Using default tag: latest Error response from daemon: manifest for cooperdk/yapo-eplus:latest not found: manifest unknown: manifest unknown

Expected behavior
A docker image should be pulled

When navigation to or reloading a page (whether that's scenes, actors etc.) the sort order resets to "Name Ascending" but the drop down list will still display your last sort option.

For example, if I sort my scenes by "Date Added Descending" and then click on the actors tab and then go back to the scenes tab, the sort setting dropdown will still say "Date Added Descending" but it's actually sorting by "Name Ascending". In order to re-sort by "Date Added Descending" I have to choose a different option in the dropdown then change it right back.

Support both landscape and portrait formats. Read the video metadata to determine display orientation

Describe the bug
api.metadataapi.net returns an unexpected reply code: 400

To Reproduce
Steps to reproduce the behavior:

1. install from yapo-eplus.exe
2. add the folder
3. check yapo terminal

Expected behavior
we should be able to set our own tpdb api key, so the error doesnt occur

Screenshots

Python version:
using the automated installer.

Getting an error when scanning:

"Generating Contact Sheet (1024 px wide, grid: 4x4...
Processing M:\Alice Bell - gush.iscreamgirls.14.08.06.alice.bell.mp4...
[ERROR] Could not find 'ffprobe' executable. Please make sure ffmpeg/ffprobe is installed and is in your PATH.
Error creating contact sheet!"

ffprobe.exe exists in C:\YAPO\videos\ffmpeg and lauches correctly from the command line (without any errors, i mean)

Describe the bug
Starting the program the second time it would give me the first 2 lines of the program (static files dir and media files dir) start and then the window just closes again.
running it in a static cli window throws an importerror: no module named _thread see screenshot.

For some reason it tries to run with my Python 2.7 installation and not with Python 3.6+ so I'm guessing the code isn't update to start with python3 by default.

To Reproduce

1. Start Yapo
2. Stop the process (seemingly can only be done in the task manager)
3. Start Yapo again

Expected behavior
That Yapo would start like it would the first time

Screenshots
http://tinyimg.io/i/OYOcW9G.png

Desktop (please complete the following information):

• OS: Win10

The Freeones scraper is currently broken.

Steps to reproduce the behavior:

1. Go to an actor or perform a full actor scrape from the Settings
2. No actor information will be found on Freeones.

The scrape should be performed, but Freeones.com just updated their website layout. For this reason. YAPO e+ will not scrape any information from Freeones until this is fixed.

A fix is on the way and expected to be pushed to Git on Monday 18th May in the evening CET.

Describe the bug
If an actor is born fx 22 September 1999 (Ella Cruz), she is reported as 21 years old already on 1 January 2020.

To Reproduce
Steps to reproduce the behavior:
Simply look at the actor in her view.

Expected behavior
The actor shouldn't be 21 years old until the day after the date of her birthday.

Feature:
Add support for seeking by clicking a certain thumbnail in a preview set.
FX, have 12 thumbnails tagged or named by location in the video's duration, and clicking that will jump to that location in the video. (video.mp4#t=00:20:30 will move to minute 20, second 30 of the video)

Importance:
Nice to have.

Whenever I try to scan a scene using the TPDB the following appears -

[WARN!] api.metadataapi.net returns an unexpected reply code: 400

• and the scan does not take place.

As the title says, the search in the Actor tab, should work when using actor's alias name too and not just with the main name. Thank you

Describe the bug
first start of yapo server, i receive error AttributeError: module 'memcache' has no attribute 'Client'
version downloaded using git: 0.7.6.4 (bugfix build 2) running under Ubuntu VERSION="18.04.6 LTS (Bionic Beaver)"

full error:
Performing system checks...

Exception in thread django-main-thread:
Traceback (most recent call last):
File "/usr/lib/python3.9/threading.py", line 973, in _bootstrap_inner
self.run()
File "/usr/lib/python3.9/threading.py", line 910, in run
self._target(*self._args, **self._kwargs)
File "/home/alfl/.local/lib/python3.9/site-packages/django/utils/autoreload.py", line 64, in wrapper
fn(*args, **kwargs)
File "/home/alfl/.local/lib/python3.9/site-packages/django/core/management/commands/runserver.py", line 118, in inner_run
self.check(display_num_errors=True)
File "/home/alfl/.local/lib/python3.9/site-packages/django/core/management/base.py", line 419, in check
all_issues = checks.run_checks(
File "/home/alfl/.local/lib/python3.9/site-packages/django/core/checks/registry.py", line 76, in run_checks
new_errors = check(app_configs=app_configs, databases=databases)
File "/home/alfl/.local/lib/python3.9/site-packages/django/core/checks/caches.py", line 63, in check_file_based_cache_is_absolute
cache = caches[alias]
File "/home/alfl/.local/lib/python3.9/site-packages/django/utils/connection.py", line 62, in getitem
conn = self.create_connection(alias)
File "/home/alfl/.local/lib/python3.9/site-packages/django/core/cache/init.py", line 44, in create_connection
return backend_cls(location, params)
File "/home/alfl/.local/lib/python3.9/site-packages/django/core/cache/backends/memcached.py", line 182, in init
super().init(server, params, library=memcache, value_not_found_exception=ValueError)
File "/home/alfl/.local/lib/python3.9/site-packages/django/core/cache/backends/memcached.py", line 28, in init
self._class = library.Client
AttributeError: module 'memcache' has no attribute 'Client'

To Reproduce
Steps to reproduce the behavior:
1 - create folder ~/projects
2 - cd into folder ~/projects and clone git repo
3 - run pip install -r requirements.txt ( no capital -R as described in readme as this doesn't work )
4 - open firewall port using: sudo ufw allow 8000/tcp
5 - start yapo using command: python manage.py runserver 127.0.0.1:8000

error received: AttributeError: module 'memcache' has no attribute 'Client'

Expected behavior
start of the yapo webserver

Screenshots

Python version:
Python 3.9.7 running under Ubuntu VERSION="18.04.6 LTS (Bionic Beaver)"

Additional context
tried to start with and without sudo rights ( so as std user or as root ). Error is the same

migration of an existing DB must happen in the startup, not in the docker buildfile.

best usage would be to write a startup shell script that manages migration and startup.

Requires testing. It seems that no information is actually stored (or might not be stored) when scanning multiple scenes (adding new scenes or re-scanning folders).
Information is stored if using the manual scanner within a scene view.

I use a fairly flat folder structure for a large collection of movies. Opening a scene straight from YAPO in VLC is functional, but I prefer the freedom to chose other players that are already available in Explorer’s Context Menu. The Open Folder button in the scene opens the flat folder, and then I have to manually find the file. I don’t use the rename feature, but I imagine if I did, I may also even have a hard time finding the file.

It would be preferable to bring the file for the referenced scene to focus. Chrome’s “View in folder” feature for downloaded files is a great example. The folder is opened in explorer and the file is pre-selected, which brings it into view even in a folder with a lot of files.

When scanning for website information and a logo is downloaded, the logo is not added to the videos_website table in the DB.

To Reproduce
Simply run a website scan in settings

Expected behavior
The website should be added to the db and shown on the website's page. This used to work.

A fix is imminent.

I'm not sure if emp has any actual API for this, but it would make things easier if tags and maybe the cover picture could be imported automatically using the torrent ID

At this point, YAPO blindly adds any folder it encounters.

Add function to disregard any trash or temp folders.

Setting the path is not necessary on Linux.

As above, when scanning for a title that has already been scanned (and misidentified), the console shows 'has already been scanned' and the web GUI shows 'not found'. clicking 'force' will replace existing info.

Currently the only way to know that the scanning of the directories has actually begun is by looking at the console

There is no guarantee of nicely formed filenames right now... But as we build correct metadata, renaming the original filename to include correct metadata to at least allow reidentification if needed makes sense.

Scraping already tries to use filename information like title, studio, stars or date if available. Extra title info is often a problem in scraping. Adjusting the title to be the correct metadata standardizes this.

At a minimum, renaming files to something with the major elements (studio/title/date/stars) in a fixed order would be great.

Allowing a flexible renaming (templated?) with all of those (and other scraped/collected info like resolution, duration, etc, and perhaps tags?) would allow edge case uses as well. Consider it a stretch goal.

Please add function for scan not only videos in one folder but search all videos with subfolders in selected folders.

In the future, YAPO will make use of YAML for other purposes. For this reason, we want to move the current configuration file from JSON to YAML.

• A function to convert the old configuration file to YAML
• A function to write configuration values by name and value
• A function to read configuration values by name

When running RunYAPO.exe, a modal displays stating the location of the static and media files, then shuts down. No GUI launches.

Currently only the file name shows next to each thumbnail.

Since directory names can contain useful information for tagging, it would be nice to see it displayed.

Also instead of the full path it is probably enough to display only the part relative to the root of the directory. Eg: for a video located in C:\videos\blonds\sitename\video.mp4 display blonds\sitename

Some websites use abbreviations which result in bad parsing.
For example, FF#### was used for Facial Fest and bb#### are used by Bang Bus.

These website abbreviations naturally cause a lot of bad parse results where any scene with "ff" or "bb" in the scene title result in a false positive for these websites.
It is possibly only if another website is not already found for scenes.

Still, it's necessary to parse website where this isssue is known to arise.

It's already possible by adding exclusions, but then the site won't be attached to the website it's supposed to.

Describe the solution you'd like

A parser that handles all known problematic abbreviations, They are probably already available in the PhoenixAdult code.

I have files of all kinds of resolutions, framerates, extensions, and bitrates in stereo 3D, VR and typical 2D files. I may have some duplicates across types that I want to keep, and some low-resolution duplicates that I might opt to remove.

Some information regarding the video files are already detected. Ideally, this information can be used to generate tags for a new category. I think this information is sufficiently different than the current categories of Actors, Website, and Tags to warrant a new category of tags.

Captured information should minimally include the resolution, framerate, duration, and file size.
Other highly useful information: video bitrate, codec, container/extension, display aspect ratio
Interesting, less broadly useful information: encoder profile/quality information, color space

Ideally a user can adjust automatic tagging based on the detects. I might want 1280x720p and 1920x1080p files to be tagged as HD, whereas another use might want the 1280x720p file tagged as “HD” and the 1920x1080p file as “FHD.”

Most users probably only care about files that have a framerate above the normal 30FPS tagged as “HFR”, but others may want 23.976 and 24 FPS files tagged as “24p”, 30 FPS files tagged “30p”, etc.

Display aspect ratio can help indicate a VR file, as the video stream resolution isn’t generally atypical.

The other video stream and file characteristics are primarily useful for sorting, understanding the content from the scene page, and duplicate detection and distinction.

Manual tagging in this category can be useful to distinguish other types of information that isn’t necessarily a great fit in with the rest of the tags.

Being able to search and sort the Scene list with all of this information, in addition to sites, actors (actresses) and tags would be very powerful. Perhaps a checkbox to enable viewing the whole category (could be disabled by default) to the scene page would be ideal.

New scene name is not always recognized by the TpDB scanner (due to the way it finds out how to rename it).
Needs to be modified slightly.

The title pretty much says it all.

Importing data from a CSV would be a great addition when you need to add/manage a lot of actors at once, expecially if you already have Names and Aliases for each Actors.

Provide a feature so YAPO will scan each video title with TpDB and add matching data to the scene, as well as add or update actor information

I'd like to suggest adding a button or link to the scene's contact sheet that is visible on both list and grid view. That way you could get a preview of the scene without having to navigate into it.

Operation breaks if an actor's measurements only include the first part, for example "32A".
For some reason, the parser I wrote fails to recognize if only parts of the measurement field are available.

This issue breaks operation if a new folder is added, and it makes it impossible to re-add the folder because I introduced a feature to not allow a user to add the same folder twice.
The only way to remedy this is to manually edit the database.

Therefore, KEEP BACKUPS for now.

To Reproduce
Steps to reproduce the behavior:

1. Go to 'Add Folders/Actors/Tags'
2. Add a new folder
3. Let YAPO work through the directory structure, allowing it to add new actors.
4. At some point, you may get this error:

03:07:35 [ERROR] local variable 'measurements' referenced before assignment while trying to add (folder) to folder list

Expected behavior
The measurements data should be parsed correctly, and only the first part added if that's the only part available. This worked earlier, but stopped working after I modified the scraper to work with XPATHs.

As this is a bug that will potentially make it impossible to re-add the same folder (without renaming it), this is a priority issue and will be fixed very soon.

I can only assume this is because you are redoing your website, but the initial run tries to pull FFMPEG and fails.

error:

"I am going to install a copy of FFPMEG (4.3.1 static, YAPO build).
It will be placed at C:\YAPO\videos\ffmpeg
Press enter to acknowledge... >
Getting https://porn-organizer.org/dl/ffmpeg-latest.zip...
Downloading...
Copying ffmpegC:\YAPO\videos\ffmpeg\https:\porn-organizer.org\dl\ffmpeg-latest\bin\ffmpeg.exe
An error occured, please download FFMPEG manually from the above link
and place it in C:\YAPO\videos\ffmpeg
YAPO will exit now. Press enter to acknowledge... >"

Would be nice if you include some information on how to set up a password on the app , I mean you can't have the kids accidentally stumble upon 2TB of homework. I tried reading the Django documentation and I talked with ChatGPT for hours and at this point it seems like this is too complicated for my simple mind , so i just have the server locally at 127.0.0.1 instead of it being accessible by the whole network. I know its probably like 2 lines of code , but I just can't figure it all , I'm sure a lot of people feel the same way.

Run the program, seems to setup fine.

When first run the following error appears on web page -

Internal Server Error

The server encountered an unexpected internal server error

(generated by waitress)

Error appears on console

IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!

Importing the numpy C-extensions failed. This error can happen for
many reasons, often due to issues with your setup or how NumPy was
installed.

We have compiled some common reasons and troubleshooting tips at:

https://numpy.org/devdocs/user/troubleshooting-importerror.html

Please note and check the following:

• The Python version is: Python3.8 from "C:\YAPO\yapo-eplus.exe"
• The NumPy version is: "1.20.1"

and make sure that they are the versions you expect.
Please carefully study the documentation linked above for further help.

Original error was: DLL load failed while importing _multiarray_umath: The specified module could not be found.

ERROR:waitress:Exception while serving /favicon.ico
Traceback (most recent call last):
File "numpy\core_init_.py", line 22, in
File "PyInstaller\loader\pyimod03_importers.py", line 531, in exec_module
File "numpy\core\multiarray.py", line 12, in
File "PyInstaller\loader\pyimod03_importers.py", line 531, in exec_module
File "numpy\core\overrides.py", line 7, in
ImportError: DLL load failed while importing _multiarray_umath: The specified module could not be found.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "django\core\handlers\exception.py", line 47, in inner
response = get_response(request)
File "django\utils\deprecation.py", line 113, in call
response = self.process_request(request)
File "django\middleware\common.py", line 53, in process_request
if self.should_redirect_with_slash(request):
File "django\middleware\common.py", line 71, in should_redirect_with_slash
not is_valid_path(request.path_info, urlconf) and
File "django\urls\base.py", line 154, in is_valid_path
resolve(path, urlconf)
File "django\urls\base.py", line 25, in resolve
return get_resolver(urlconf).resolve(path)
File "django\urls\resolvers.py", line 545, in resolve
for pattern in self.url_patterns:
File "django\utils\functional.py", line 48, in get
res = instance.dict[self.name] = self.func(instance)
File "django\urls\resolvers.py", line 589, in url_patterns
patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
File "django\utils\functional.py", line 48, in get
res = instance.dict[self.name] = self.func(instance)
File "django\urls\resolvers.py", line 582, in urlconf_module
return import_module(self.urlconf_name)
File "importlib_init_.py", line 127, in import_module
File "", line 1014, in _gcd_import
File "", line 991, in _find_and_load
File "", line 975, in find_and_load_unlocked
File "", line 671, in load_unlocked
File "PyInstaller\loader\pyimod03_importers.py", line 531, in exec_module
File "urls.py", line 27, in
File "PyInstaller\loader\pyimod03_importers.py", line 531, in exec_module
File "views.py", line 10, in
File "PyInstaller\loader\pyimod03_importers.py", line 531, in exec_module
File "addScenes.py", line 8, in
File "PyInstaller\loader\pyimod03_importers.py", line 531, in exec_module
File "videosheet.py", line 49, in
File "PyInstaller\loader\pyimod03_importers.py", line 531, in exec_module
File "numpy_init.py", line 145, in
File "PyInstaller\loader\pyimod03_importers.py", line 531, in exec_module
File "numpy\core_init.py", line 48, in
ImportError:

There should be people who have already organized their collection using directories so it would make sense to tag all videos with the same tag inside a directory. This feature would require browsing videos by directories.

Is there any way to stop the automatic opening of a web browser when using docker?

When run, the following error appears and the container exits-

yapo | Traceback (most recent call last):
yapo | File "./yapo.py", line 74, in
yapo | startup.startup_sequence()
yapo | File "/YAPO/videos/startup.py", line 284, in startup_sequence
yapo | webbrowser.get().open_new_tab(site)
yapo | File "/usr/local/lib/python3.8/webbrowser.py", line 65, in get
yapo | raise Error("could not locate runnable browser")
yapo | webbrowser.Error: could not locate runnable browser

Describe the bug
Windows paths are case insensitive. If one was to add "e:\my_videos" and then "E:\MY_VIDEOS". YAPO-e will add a duplicate entry for each one that is already added from the previous path. The more critical problem is that 'cleaning the database' removes the duplicates, but it also physically deletes the files from the disk.

To Reproduce
Steps to reproduce the behavior:
On windows, add two identical paths with a different case. Then clean the library.

Expected behavior
Files from the same path should not be added twice.

Python version:
3.7

Additional context
Probably should check on insertion whether the OS is windows. And if it is should check in a case insensitive manner if the path already in the database.