Code Monkey home page Code Monkey logo

fanfiction-manager's Introduction

Fanfiction-Manager

NOTE : This program is for personal use only. Please do not use it to share/plagiarize content.

Fanfiction manager/downloader/reader

Provides extreme flexibility with the help of Rule system to power-users* in downloading, tracking-status & reading novels from ANY site they want.

Novice-users** may only download Novels from site whose Rule is included.

Feel free to request for more website inclusions via issues

Current Status: BETA

  • Add new Novel
  • Download Novel
  • Update Novel
  • Rules included for sites
    • fanfiction.net
    • scribblehub.com
    • webnovel.com
    • novelupdates.com
      • need to add rules related to external linked sites
      • out of scope
    • comrademao.com
    • archiveofourown.org
    • royalroad.com
    • wattpad.com
    • mtlnovel.com
    • and more... check Rules tab in-app to get the full list
  • Rules Editor
  • Novel Reading Mode
  • Novel Details Editor
  • App auto-updater

Shortcuts

Global

Keys Function
Ctrl + F Fullscreen

Reading Mode

Keys Function
Ctrl + G Go to Chapter
Ctrl + H Toggle Sidebar

Known Issues

  • [WIP] No images support within chapters.
  • [NYI] Rules export option.
  • [NYI] Download next Chapter using nextURL when ChapterCount is <= 0
    • Although, the case is possible, but currently, no known website requires this.
      • Further testing required.
    • Possibility of infinite loop, in-case nextURL always comes up non-empty.
  • [WIP] More testing required for downloading from other websites.

Download

Rules may be outdated therefore update rules by clicking the Update Rules button on the top-right of Rules tab. Rules may be updated at random on the server, therefore do this regularly before adding/downloading/updating a novel.

For windows 64bit, download here

For others -

  • Download electron from this page
    • Example, for windows 64bit, you need to download electron-v19.0.5-win32-x64.zip
  • Extract the electron zip file to an empty directory
  • Delete default_app.asar file present in .../[electron-dir]/resources
  • Create new directory app in .../[electron-folder]/resources
  • Extract the contents of this zip file into .../[electron-dir]/resources/app directory
  • Now run electron executable
  • Done

Rule Generation

Script details -

  • PageTypeScript
    • checks if page is supported by current rule
    • if supported, return 0;
    • for captcha page
      • return -1 for Auto-Captcha
        • meaning page will auto-redirect, just need to wait for it
      • return -2 for Manual-Captcha
        • meaning there's a chance user needs to use the renderer to proceed from the page
        • otherwise similar to Auto-Captcha, if auto-redirects
    • returning anything else will mean
      • page not supported by rule
  • TOCScript
    • Table Of Contents
    • gets Novel info from page
    • return type -
      {
        "CoverURL": "",              // may be empty
        "Title": "novel name here",
        "Summary": "",               // may be empty
        "ChapterCount": 1,           // chapter count here
        "ChapterURLs": [             // list of chapter-URLs
            "",                      // must contain atleast the first chapter URL
            "",
        ],
      }
  • ChapterScript
    • gets the Chapter data from page
    • return type -
      [
        {
          "title": "",    // title of chapter
          "content": "",  // content
          "nextURL": ""   // when "ChapterCount" <= 0 & "nextURL" always comes non-empty
                          // will loop infinitely unless something else(javascript) breaks
        },
      ]
    • maybe one or more chapters at once, therefore an Array

Tips

  • All scripts support custom-redirection if they return JSON in the following format -

    {
      "retry": 1,
      "nextURL": "", // the URL to redirect to
    }
  • Make sure no infinite loops are present in any scripts.

    • Use Ctrl+Shift+R to reload application, in-case UI gets stuck or high CPU usage
  • Use Tester tab to load TestURL and execute particular scripts on them

  • Want to wait(for some seconds) in script, use -

    • await sleep(ms);
      • where ms is in milliseconds
      • simple Promise based in-built async/await function
  • Want to inject jQuery in script, use -

    • injectJquery();
  • Want to decode html-codes like &amp;, &larr;, ... to their actual character in script, use -

    • htmlDecode(input);
      • where input is a string
    • similarly htmlEncode(input) is also there
  • Make sure Title/title for Novel/Chapter is actually the innerText, not innerHTML

  • In Chapter's content field, HTML is to be used.

    • Make sure all anchor tags in HTML are marked with target="_blank"
      • This will open the link in external browser whenever clicked upon

* persons with javascript & DOM knowledge

** persons with no programming knowledge whatsoever

[WIP] Work in progress

[NYI] Not yet implemented

fanfiction-manager's People

Contributors

abanobnageh avatar gmastergreatee avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar

fanfiction-manager's Issues

Not able to run the code.

can you give me more descriptive way how to use this. because it is giving error while launching the app

Rule contribution (lightnovelworld | Slow update to avoid ban)

Just contributing a custom rule

{ "guid": "a804de22-86bb-4aac-a1bf-cae1addfc889", "rule_name": "lightnovelworld", "url_regex": "(?:https:\\/\\/www\\.lightnovelworld\\.com|.co\\/novel\\/([a-zA-Z0-9\\-\\/]+))", "pagetype_code": "return 0;", "toc_code": "injectJquery();\n\n// Extract the URL from the \"Novel Chapters\" link\nlet novelChaptersURL = $('a.grdbtn.chapter-latest-container').attr('href');\n\n// Fetch the content of the \"Novel Chapters\" page\nlet response = await fetch(novelChaptersURL);\n\n// Parse the HTML content\nlet htmlContent = await response.text();\n\n// Create a virtual DOM for parsing\nlet virtualDOM = $(htmlContent);\n\n// Extract base URL\nlet baseURL = window.location.origin;\n\n// Extract cover URL\nlet coverURL = virtualDOM.find('.novel-item .novel-cover img').attr('src');\n\n// Extract title\nlet title = virtualDOM.find('.novel-item h1 a.text2row').text().trim();\n\n//Summary\nlet summary = $('meta[itemprop=\"description\"]').attr('content');\n\n\n// Extract all chapter URLs\nlet chapterURLs = [];\n\n// Loop through each chapter list item\nvirtualDOM.find('.chapter-list li a').each(function () {\n let chapterLink = $(this).attr('href');\n // Prepend the base URL to the relative chapter URL\n chapterURLs.push(new URL(chapterLink, baseURL).href);\n});\n\n// Check if there's a next page for chapters\nlet nextPageLink = virtualDOM.find('.pagination-container a[rel=\"next\"]').attr('href');\n\n// If there's a next page, fetch its content and extract chapter URLs\nwhile (nextPageLink) {\n response = await fetch(nextPageLink);\n htmlContent = await response.text();\n virtualDOM = $(htmlContent);\n\n virtualDOM.find('.chapter-list li a').each(function () {\n let chapterLink = $(this).attr('href');\n // Prepend the base URL to the relative chapter URL\n chapterURLs.push(new URL(chapterLink, baseURL).href);\n });\n\n // Check if there's another next page\n nextPageLink = virtualDOM.find('.pagination-container a[rel=\"next\"]').attr('href');\n}\n\nlet retMe = {\n 'CoverURL': coverURL,\n 'Title': title,\n 'Summary': summary,\n 'ChapterCount': chapterURLs.length,\n 'ChapterURLs': chapterURLs,\n};\n\nreturn retMe;\n", "chapter_code": "// Ensure jQuery is injected\ninjectJquery();\n\n// Set to keep track of visited chapter URLs\nlet visitedChapterURLs = new Set();\n\nlet retMe;\n\n// Check if the current page is the Table of Contents (TOC) page\nif (isTOCPage()) {\n console.error(\"Error: Script is on the Table of Contents page. Not a chapter page.\");\n} else {\n let currentChapterURL = window.location.href;\n\n if (visitedChapterURLs.has(currentChapterURL)) {\n console.error(\"Error: Same chapter URL loaded twice. Breaking the script.\");\n }\n\n visitedChapterURLs.add(currentChapterURL);\n\n // Extract chapter title\n let chapterTitle = $('.chapter-title').text().trim();\n console.log('Chapter Title:', chapterTitle);\n\n // Scroll down to the bottom of the page\n await scrollDown();\n\n // Add an increased delay (15 seconds) to ensure content is loaded\n await sleep(8000);\n\n // Extract chapter content after scrolling\n let chapterContent = $('#chapter-container').html();\n console.log('Chapter Content:', chapterContent);\n\n // Get the URL of the next chapter\n let nextChapterPartialURL = getNextChapterURL();\n console.log('Next Chapter Partial URL:', nextChapterPartialURL);\n\n // Check if the next chapter URL is complete\n let nextChapterURL = null;\n if (nextChapterPartialURL) {\n // Get the base URL of the website\n let baseURL = window.location.origin;\n \n // Create the complete next chapter URL\n nextChapterURL = new URL(nextChapterPartialURL, baseURL).href;\n console.log('Next Chapter Complete URL:', nextChapterURL);\n } else {\n console.log('No Next Chapter Button found, or the next chapter has been visited before. It might be the last chapter.');\n }\n\n // Add a delay (8 seconds) after logging the next chapter URL\n await sleep(3000);\n console.log('Delay complete.');\n\n // Create the response object\n retMe = [\n {\n \"title\": chapterTitle,\n \"content\": chapterContent,\n \"nextURL\": nextChapterURL\n },\n ];\n}\n\nfunction isTOCPage() {\n return $('.toc-container').length > 0;\n}\n\n// Function to scroll down to the bottom of the page\nasync function scrollDown() {\n return new Promise(resolve => {\n let totalHeight = 0;\n let distance = 100;\n let scrollHeight = document.body.scrollHeight;\n\n let timer = setInterval(() => {\n window.scrollBy(0, distance);\n totalHeight += distance;\n\n if (totalHeight >= scrollHeight) {\n clearInterval(timer);\n resolve();\n }\n }, 50); // Reduced delay to 50 milliseconds\n });\n}\n\n// Function to get the URL of the next chapter\nfunction getNextChapterURL() {\n // Extract the URL of the next chapter link\n let nextChapterLink = $('a.button.nextchap');\n if (nextChapterLink.length > 0) {\n return nextChapterLink.attr('href');\n } else {\n // If no next chapter link is found, it means it's the last chapter\n // You can handle this case accordingly, for example, returning null or a special value\n return null;\n }\n}\n\n// Function for sleep/delay\nfunction sleep(ms) {\n return new Promise(resolve => setTimeout(resolve, ms));\n}\n\nreturn retMe;\n", "url_blocks": "" },

Status

Features to complete before first alpha

  • Web Renderer
    • Rule Test Renderer
      • Controls
    • Renderer
      • Controls
  • Site Rules Manager
    • Add Rule
    • Edit Rule
    • Delete Rule
  • Data Storage
    • Rules
    • Novels
    • Chapters
  • Downloader
  • Library
  • Updater
  • Reader

Allow Renderer interaction while loading

Currently all renderer actions are blocked.

Need to give the choice to the user if one wants to or not.

  • Option for activating/deactivating user interaction in the renderer

Display Story Status - Ongoing/Completed

Can you please add it to the main page so it shows if a story is completed or if it's in progress? It would help a lot of people so they know if a story in their library is completed or not.

Display More Novel-Metadata

Currently only novel-name, chapter-count, cover-art and summary are displayed in the Library tab

Need to display more information

  • Tags
  • Author
  • Completion Status
  • Last Updated - by novel author
  • Last Checked - by app, for new chapters
  • Read Till - in app

Feature to download in-chapter images

Currently, the only image being downloaded is Cover-Image. Allow rule to specify images to be downloaded.

  • Images will be downloaded in ./data/novels/{NovelGUID}/images folder

  • Add new parameter in ChapterScript return object.

    {
        "title": "",    // title of chapter
        "content": "",  // content
        "nextURL": ""
        "images": [
            {
                "name": "{unique name here}",
                "url": "{url of the image to download}",
            },
        ],
    }
  • content will have html containing images, whose src value will be relative.

    • will be in the format ./data/novels/{NovelGUID}/images/{name}
    • NovelGUID must be supplied to the ChapterScript.

EPUB Export

Add an EPUB export option for novels

  • EBUB Export

Low priority : Coz I don't need this & almost no demand

Webnovel.com not working??

this is what i get when trying to download something from webnovel.com is there a soulation?
Error -> [SCRIPT] Cannot read properties of null (reading 'value')

Retain Website Specific Styles

Display of chapter-text should be the same as was intended by the author.

Currently, the chapter-script has to do the entire styling. Should rather throw the styles in rule-data.
Append a copy of current-styles to the novel-data, when added.

Does not work on linux/crostini/archlinux.

Describe the bug
Tried with the default electron package from archlinux (electron 30), electron 23, electron 19 from aur and finally with the electron 19 from the readme, in each of the attempts it crashed with

A JavaScript error occurred in the main process
Uncaught Exception:
Error: Cannot find module 'follow-redirects'

To Reproduce
Steps to reproduce the behavior:
follow the readme instructions for others on linux

Expected behavior
the application should open

Screenshots
If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

  • OS: chromeos
  • Browser chrome
  • Version 123

Additional context

Royalroad.com rule issue.

Describe the bug
Hello, first I wanted to thank you for this manager, I like using it to keep all stories I read in one place.
Next, I wanted to report that there seems to be an issue with the rule for royalroad.com. When I try to update or download specific stories they download up to a certain chapter before the downloading stops. The story still shows as downloading but nothing actually happens.

I can reproduce this with 2 stories, they are:

At the time of writing this issue the first stops at 61/62 chapters and the second stops at 116/118 chapters.

To Reproduce
Steps to reproduce the behavior:

  1. Add either of the above stories (or directly move to the second step if either already exists).
  2. Once they are added click on the download button.
  3. It should stop downloading and clicking stop causes all the buttons for the novel to be disabled.

Expected behavior
The story should download normally.

Screenshots

Before clicking the download button for an already existing story:
image

After clicking download, this is how the story will look like until I close and reopen the manager:
image
image

If I click stop:
image

Desktop (please complete the following information):

  • OS: Windows 10
  • Browser: Chrome
  • Version: 120.0.6099.217 (Official Build) (64-bit)

Additional context
N/A

The "novelmax | binnovel" TOC script doesn't work.

Describe the bug
The "novelmax | binnovel" TOC script currently doesn't work. It gives the following error:

Error -> [SCRIPT] ajaxChapterArchiveUrl is not defined

To Reproduce
Steps to reproduce the behavior:

  1. Open the "novelmax | binnovel" rule.
  2. Add the following test URL: https://novelmax.net/n/getting-a-technology-system-in-modern-day
  3. Load the URL and then try to run the TOC script.
  4. See the error in the console.

Expected behavior
The rule should work correctly like it did before.

Screenshots
Here is how it looks like If I try to test it in the app:
image

Desktop (please complete the following information):

  • OS: Windows 10.

Additional context
N/A

Hangs up on costume rule for novelsite

Describe the bug
Created a custom rule to work on a unlisted novel site and the results showed fine in the tester, and in the rules when you edit and run it live while work.

To Reproduce
try a novel from the site provided in the rule given below

{
"guid": "03f193cd-ab75-43f7-a2b7-6c74e133f6ba",
"rule_name": "test",
"url_regex": "?:https://)*(?:www\\.)novelmax\\.net/n/.",
"pagetype_code": " const urlPattern = /^https:\/\/novelmax\.net\/n\/florida-mans-general-store-in-cultivation-world/i;\n\n // Check if the current page is supported by the rule\n if (window.location.href.match(urlPattern)) {\n // Check for captcha pages\n if ($('.captcha-page').length > 0) {\n // Check if it's an Auto-Captcha page\n if ($('.auto-redirect-captcha').length > 0) {\n // Auto-Redirect, no user intervention needed\n return -1;\n } else {\n // Manual-Captcha, user might need to use the renderer\n return -2;\n }\n } else {\n // Page is supported, return 0\n return 0;\n }\n } else {\n // Page not supported by the rule\n return -3;\n }\n}\n",
"toc_code": "let retMe = {\n 'CoverURL': $('img[src="https://novelmax.net/media/novel/florida-mans-general-store-in-cultivation-world.jpg"]').attr('src'),\n 'Title': $('h3.title').text(),\n 'Summary': $('.desc-text').text().replace(/\n/g, '
'),\n 'ChapterCount': 0,\n 'ChapterURLs': [],\n};\n\nawait sleep(100);\n\n// Extract all chapter URLs\nlet chapterURLs = [];\n$(".list-chapter li a").each(function () {\n let chapterUrl = $(this).attr("href");\n chapterURLs.push(chapterUrl);\n});\n\nretMe.ChapterURLs = chapterURLs.reverse();\nretMe.ChapterCount = retMe.ChapterURLs.length;\n\nreturn retMe;",
"chapter_code": "injectJquery();\n\nlet visitedChapterURLs = new Set();\n\nlet retMe;\n\nif (isTOCPage()) {\n throw new Error("Error: Script is on the Table of Contents page. Not a chapter page.");\n} else {\n let currentChapterURL = window.location.href;\n\n if (visitedChapterURLs.has(currentChapterURL)) {\n throw new Error("Error: Same chapter URL loaded twice. Breaking the script.");\n }\n\n visitedChapterURLs.add(currentChapterURL);\n\n // Check if the next chapter button is disabled\n let nextChapterButton = $('#next_chap');\n if (nextChapterButton.length &gt; 0 &amp;&amp; nextChapterButton.attr('disabled') !== undefined) {\n throw new Error("Error: Next chapter button is disabled. Breaking the script.");\n }\n\n retMe = [\n {\n "title": $('p:contains("Chapter")').text(),\n "content": $('#chr-content').html(),\n "nextURL": getNextChapterURL()\n },\n ];\n}\n\nfunction isTOCPage() {\n // Add logic to identify if it's the Table of Contents page\n // You can use specific elements or patterns that are unique to TOC\n // For example, if the TOC has an element with class "toc-container"\n return $('.toc-container').length > 0;\n}\n\nfunction getNextChapterURL() {\n // Extract the URL of the next chapter link\n let nextChapterLink = $('#next_chap');\n if (nextChapterLink.length > 0) {\n return nextChapterLink.attr('href');\n } else {\n throw new Error("Error: Last chapter reached. No next chapter found.");\n }\n}\n\nreturn retMe;\n",
"url_blocks": "pubfuture-ad.com"
},

Expected behavior
to scrape with no issues.

Screenshots
Non provided

Desktop (please complete the following information):
Windows 10, build 19045
Latest chrome

Additional context
Add any other context about the problem here.

No Cover Art - Fanfiction.net

Describe the bug
When am downloading fanfic from fanficiton.net there is no cover art. Is this a bug or how do I fix it.

[UpdateDownloadAll] Race Condition

Describe the bug
After clicking on UpdateDownloadAll, in case of novel-entry in the middle of the library, if currently selected novel has some updates, then the code-execution splits into 2 -

  1. First goes into the next novel-entry and starts loading that
  2. Second keeps downloading the chapters of the currently selected novel-entry

This bug happens mostly when the computer isn't switched off for a long time, for example my system hasn't been turned off for 3 weeks. Bug still appears very randomly. In some PCs, it can't be reproduced at all.

To Reproduce
Do as mentioned above

Expected behavior
Point 2 should be the one happening and not point 1.

Novel update related buttons stuck

Describe the bug

While adding a new novel from a supported rule, if "No PAGETYPE found" error pops up, then all downloading related functionality is blocked.

UpdateDownloadAll, and sometimes normal update on single novel has issues with getting chapter list

Describe the bug
A clear and concise description of what the bug is.

UpdateDownloadAll, and sometimes normal update on single novel has issues with getting chapter list

Not sure of the cause, works well when you test the effected novel in the rule tester, but on main page the functions sometimes do not perform the same (ex: updated novel on main list shows 500/500, go to the rule tester throw the novel in you can get 512 ch found

Desktop (please complete the following information):

  • OS: [e.g. iOS] Windows 10
  • Browser [e.g. chrome, safari] Chrome
  • Version [e.g. 22] beta 3.1.9

Additional context

Would label this really low prio, if it really needs to be addressed cause it could be a issue of a rule slamming a site, or enforced cache (Cloudflare)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.