spillerrec / boorusurfer2 Goto Github PK

We want to cache several things in the database, create the infrastructure needed.

Recent commit (29dae02) completely broke building on Linux.

Task list:

• Switch to CMake that actually supports multiple platforms
• Add #ifdefs to skip Windows specific code in src/Server.cpp
• Don't compile src/system/Thumbnailer-windows.cpp
• Add default implementation for src/system/Thumbnailer.hpp

When restarting the browser, having a lot of tabs open, we could easily send too many requests. Limit this to a sensible number, to avoid getting banned. It would also be nice to have an interface to monitor how much traffic is happening too.

see: /post/san/1155901/
博麗 became åšéº—

Infinity scrolling loads a new page when you scroll to the end. However if it doesn't fill up the entire page so that it is possible to scroll (if you have a very large monitor, or small thumbnails are used), then it is not possible to load the next one. Currently this is circumvented by always loading an extra page when you open the page, however this obviously fails if the monitor is large enough to have two pages without overflowing.

Some solution needs to be found. Perhaps doing the check for the end of page each time you load a new page?

If a post have not been accessed and it appears on an index page on San, everything gets overridden if as it does not know the post already exists in the system

Some notes might have formatting, like bold text. We need to convert from "dangerous" HTML to something more controlled, like a custom flavor of BBCode. This conversion needs to be done from both a string and DOM representation of the HTML.

Of some reason database will not unlock after loading several pages in parallel.

If the RSS reader sorts by time, the order becomes all messed up if it is missing. So we could use current time, and subtract a minor amount for each post.

We currently use the Yandere style colors, what about having Danbooru, Sankaku, etc. colors as alternative stylesheets?

You cannot currently compile BooruSurfer2 on Linux due to there being Windows specific linker flags in the configuration. Premake4 have shown not to be able to handle cross-platform specific options, so we have to change to something else. CMake should be able to do the job, and perhaps we can add SASS at the same time.

The Api can be slow, so prefetching could speed it up significantly. Figure out how to best do this.

It can be difficult to know the right tag name, especially since the naming conventions isn't completely consistent. For example character_name_(re:zero) does not use the full copyright name, but a shorthand.

We would like a suggestion list like BooruSurfer1. We need to add the SQL in the backend, and see if we can reuse the previous implementation in the frontend.

Scrollbar was set to show on index pages to avoid the floating thumbnails to rearrange themselves, but it is now always enabled for all pages. It should only be the index page, so the CSS needs to be fixed.

We should have some way of changing settings. (Basically I'm annoyed that the port number is hardcoded so I can't run the debug build at the same time,) Since we already have a DB, we might as well use that. SQLite is dynamically typed, which is perfect.

Things to configure:

• Port number
• Directory for saved files (Issue, file names end up being absolute, which might cause issue if you move it. Investigate and open a new issue)

Not yet added...

Even if the API does not know the post count, once we receive fewer post than we should, we know we have reaced the end, and thus shouldn't link to the next page.

The proxy page should not load the entire resource into memory before outputting it. Pass content to the browser as soon as possible, so it can incrementally load it. Large images, gifs and webm will benefit greatly.

It would be pretty nice. Try messing around with page break indicators

Large files, usually GIF, SWF, and WEBM files, only shows up when the whole file has been loaded on the Server. This is stupid in every way...

Entering the save page twice causes the full-resolution saved image not to work. Investigate

We can easily get notes with Sankaku, so we should add this immediately...

The entire page is sent, and while it shows up in the browser, it doesn't stop loading until flushing is done. flushing is a background task and should not affect page loading in the browser in any way, so fix this somehow.

The CSS already supports the use of .size_medium on the .post_list to increase the thumbnail size to match kona-chan/yandere-chan (currently standard), but the correct size should be chosen depending on the site currently viewed.

Several boorus do not have a proper Api, and RSS feeds are a good way to go here

/index/local-san/ fails to load. My first test caused an infinitely loop reading the same post from the DB. After clearing the DB, it loads, but fails to show any images.

When using the local DB, if the original thumbnail link was lost, the big image is linked on the index page. This often causes it to go out of page if hovering over it, and is likely to use a lot of memory on the client side.
So we should have some way of resizing it. Since we need to support a lot of formats, including video, I suggest using a stand-alone piece of software for this, or perhaps use a plug-in system to avoid having to include a big amount of libraries. Plugging into a existing thumbnail system seems to be a much more reasonable approach, but will likely require us to do platform specific code. We should investigate the existing solutions at least, and perhaps make a separate cross-platform library for this if needed, as I think it will be useful for others.

If a page load slowly and you hit the bottom of the page twice, it will do the request twice and add the page twice as well. We should only have one active request for the same page at a time.

The tag count is less important than the tag, yet it appears as important as the name. It should be less colorful and/or smaller.

It is not possible to read from the mem-cache while it is being flushed. It is locked because some operations, including reading, might increase the vector, invalidating it if capacity is breached.
This could be solved by keeping an insertion queue, and lock separately on reading vs. inserting. Insertion will simply be waiting until flushing is done.

Flushing could also be improved to not block other flushes, by letting the currently running flush re-flush when it is done. The otherwise blocked flush would just exit without doing anything.

Searches have stopped working on danbooru as it returns a 404. Perhaps they switched to the v2 API and just have some old basic functionality for compability with existing scripts

We could add a simple checker box pattern to the main image using CSS. This way, even if the image contains only the background color, you can still see that there is an image.

Sankaku-only feature for now. Would be nice with some video controls...

If Poco is not include first, linking fails of some reason. Investigate why, and see if it is solvable, as there is no way to do this without including it in all files which includes files that includes Poco

Depending if the scrollbar is present or not, a different amount of columns are used on the index page if x+1 can be shown without the scrollbar and only x with it. This is especially problematic if the height of the page is changed due to hover effects or infinity paging. Extra bad because it appears with a page width of 1920px.

Needs to be solved:

• It should be optimized for a page width of 1920px, and as many other common resolutions. Fiddle with the padding and some sizes to achieve this.
• Even with a custom sized window it should not be allowed to rearrange the thumbnails without the user actively trying to do so (e.g. by resizing the page, or clicking some imaginary button to change thumbnail size).

If a tag is for example 021, the link /site/index/021 will show page 21 for the root page, instead of page 1 for the search 021. Also check if tags starting with numbers incorrectly gets treated as numbers.

• The post time is now taking up two lines of text. Isn't just having the date good enough, instead of the exact timestamp?
• The tag list when hovering over a thumbnail can now go out of the left side of the monitor

Poco is big and clunky, and really not that nice. libnghttp2 is an implementation of the HTTP2 standard, and the C++ interface seems to be similar in complexity to Poco::HttpServer.
https://nghttp2.org/documentation/libnghttp2_asio.html

Investigate this further and see if it is a good idea to try out.
It uses Boost.asio, and since we use a few parts of Poco, like DateTime and Filesystem, we would probably want to switch these to the Boost variants as well. Getting rid of that huge framework would be wonderful.

Several errors can happen while loading, but right now we just end up with a broken page. Create an exception system to catch Api errors, and create error pages. Also do errors such as 404, invalid parameters, or 500