Code Monkey home page Code Monkey logo

comicstreamer's Introduction


Introduction

ComicStreamer is a media server app for sharing a library of comic files via a simple REST API to client applications. It allows for searching for comics based on a rich set of metadata including fields like series name, title, publisher, story arcs, characters, and creator credits. Client applications may access comics by entire archive file, or by fetching pag e images, one at a time.

A web interface is available for searching and viewing comics files, and also for configuration, log viewing, and some control operations.

It's best used on libraries that have been tagged internally with tools like ComicTagger or ComicRack. However, even without tags, it will try to parse out some information from the filename (usually series, issue number, and publication year).

ComicStreamer is very early ALPHA stages, and may be very flakey, eating up memory and CPU cycles. In particular, with very large datasets, filters on the sub-lists (characters, credits, etc. ) can be slow.

If you have web development or graphic design skills, and would like to help out, please contact me at [email protected]. In particular, ComicStreamer needs a new logo!

Chunky Comic Reader for iPad has added experimental ComicStreamer support. Pro upgrade required, but it's well worth it for the other features you get. Check it out! If you are comic reader developer (any platform), and would like to add CS support, please contact me if you need any special support or features.


Compiled Package Downloads

Windows and Mac OS X


Requirements (for running from source)

  • python 2.7

(via pip):

  • tornado
  • sqlalchemy >= 0.9
  • watchdog
  • python-dateutil
  • pillow (PIL fork)
  • configobj >= 5.0.5
  • natsort

Optional:

  • pybonjour (for automatic server discovery)

Installation

For source, just unzip somewhere. For the binary packages, it's the usual drill for that platform. (No setup.py yet, sorry)

Settings, database, and logs are kept in the user folder:

  • On Linux: "~/.ComicStreamer"
  • On Mac OS: "~/Library/Application Support/ComicStreamer"
  • On Windows: "%APPDATA%\ComicStreamer"

Running

From the source, just run "comicstreamer" in the base folder (on windows you may want to rename it comicstreamer.py).

For the binary builds, run from the installed app icon. There should be no taskbar/dock presence, but an icon should appear in the system tray (windows), or status menu (mac).

A web browser should automatically open to "http://localhost:32500". On your first run, use the "config" page to set the comic folders, and the "control" page to restart the server. It will start scanning, and all comics in the given folders and sub folders will be added to database.

Some tips:

  • Use "--help" option to list command-line options
  • Use the "--reset" option (CLI) or control page "Rebuild Database" to wipe the database if you're having problems.

comicstreamer's People

Contributors

mike-ferenduros avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

comicstreamer's Issues

Duplicate folder matching needs some work

Firstly, nice bit of work with this. It's going to be handy in the future. I'm looking forward to when it's polished up and had cosmetic improvements.

I've found a case where two different folders can't be configured, presumably because they start with the same string and/or the fact that one has a space in it. The folders in question are:

/Volumes/Storage/Media/Manga
/Volumes/Storage/Media/Manga (Ongoing)

I've tried using \ to escape the space and quoting the folder with a space but I always end up with the "one folder can't contain another" error message.

SUGGESTION: Import ComicRack "Rating" Metadata

Issue:
Rating allows us to see our favorite comics and pull them from our library when we feel the need to pull that great series, issue or arc.

Suggestion:
Pull the ComicRack Rating metadata when importing a folder. Null ratings will remain null.

New Design

I would like to propose a new fresh design for this app's web interface.
The focus would be on usability, and simplified navigation.
This is just the start but I'll be committing a lot of front-end enhancements to my fork very soon.
prototype

home sketch

navigation

@beville Would love to pick your brain on the future of this project. Thanks

[FEATURE REQUEST] Story arc support for Mylar-organized library?

Hi,
First of all, great work on this nifty piece of code, it was the cog that made all my comic-enjoying system work ^.^

I use Mylar to manage and organize my comic collection, and they have recently adde the option to manage story arcs (Convergence, Identity Crisis, etc...), would it be possible to add a new categorization in the database sorting to recognize them?

That way, when accessing the comic collection from Chunky, i can dowload the whole story arc without the need to fetch individual comics.

Thanks in advance and keep being great!

SUGGESTION: Synology support

I've had a couple of requests for this, so passing them on: ComicStreamer seems like a natural fit to run on NASes, so it'd be nice to have a pre-built package for Synology devices (or whatever other platforms make sense).

SUGGESTION: Create Library Update/Write API

Issue:
In services like Chunky, it would be wonderful to be able to update the metadata on-the-fly. This would allow users to tag data as they access their library with things like "rating", "arc title" and/or the "have read" flag. If extensible enough, in the long run it would allow people could edit their metadata to adjust for mistakes.

Suggestion:
Create API that services can use to update the library. It shouldn't necessarily start robust - and should really be a minimum viable feature to begin with (ie. only update rating). Then it can grow as a feature-set.

BUG: ComicRack Metadata Not Importing Properly

Issue:
When importing a library, the metadata is not being pulled into ComicStreamer. All the books appear and are readable, but most of the metadata are not coming across.

Details:
OS: OSX (10.9.2) w/ Bootcamp Win8
Library: Library exists on the Win8 Bootcamp drive
Library Size: 1935
Metadata: Metadata created by ComicRack in Win8
ComicStreamer: Streaming library in OSX Mavericks

Steps to reproduce:

  1. Boot to Win8 on Bootcamp
  2. Import all comics into ComicRack (Win8)
  3. Add metadata to comics in ComicRack (Win8)
  4. Boot to OSX
  5. Run ComicStreamer
  6. Import library from Win8 Bootcamp drive
  7. Examine library

PDF Encrypted not handled

The one with PDF support, does crashes with encrypted pdf.

https://github.com/kounch/ComicStreamer

2016-05-13 08:13:02,986 - INFO - NEW -- to_remove size [0]
2016-05-13 08:13:03,040 - INFO - Monitor: Removing missing or modified files from db (0 files)
2016-05-13 08:13:03,040 - INFO - Monitor: 76764 new files to scan...
2016-05-13 08:14:01,191 - ERROR - Could not open static file '/Users/Tristan/Desktop/ComicStreamer.app/Contents/MacOS/static/comicbook/js'
PdfReadWarning: Xref table not zero-indexed. ID numbers for objects will not be corrected. [pdf.py:1707]
2016-05-13 08:24:17,937 - ERROR - Could not open static file '/Users/Tristan/Desktop/ComicStreamer.app/Contents/MacOS/static/comicbook/js'
Exception in thread Thread-1:
Traceback (most recent call last):
File "threading.py", line 801, in __bootstrap_inner
File "threading.py", line 754, in run
File "comicstreamerlib/monitor.py", line 82, in mainLoop
File "comicstreamerlib/monitor.py", line 264, in dofullScan
File "comicstreamerlib/monitor.py", line 166, in getComicMetadata
File "comicapi/comicarchive.py", line 710, in seemsToBeAComicArchive
File "comicapi/comicarchive.py", line 867, in getNumberOfPages
File "comicapi/comicarchive.py", line 843, in getPageNameList
File "comicapi/comicarchive.py", line 584, in getArchiveFilenameList
File "PyPDF2/pdf.py", line 1131, in getNumPages
PdfReadError: File has not been decrypted

Export library to excel or file containing library?

Is there a way (not counting copy of every page and pasting to Excel) to export the ComicStreamer library to Excel? Alternatively, is there a specific file in the existing folder structure that contains the library itself that can be opened in Notepad and moved into Excel?

Thanks.

#None number issues

I have a complete comic set, for this example Mighty Morphin' Power Rangers. Every book is titled correctly by Mylar (Mighty Morphin Power Rangers 9 (2016).cbr) so on and so forth. Everything is fine, except book 7 and 8 are showing in Chunky as Mighty Morphin' Power Rangers v2016. I checked the ComicStreamer library for this series, and these two are the only ones with an (') on Morphin'. Every file on my system is named the same, with no ' so I'm not sure why they're imported as such. See the photos below for a correct, and incorrect.

EDIT:
I noticed when viewing the .json from ComicStreamer, that the "Correct" issues do not have a Volume, but do have a year and issue #. The "Incorrect" has a Volume, but no issue # or year.

Correct
Image and video hosting by TinyPic

Incorrect
Image and video hosting by TinyPic

Stuck Removing missing or modified files from DB..

I recently deleted a series of about 27 issues. I noticed that Chunky still showed the issues, as well as didn't show my new series'. Looking at Comic Streamer, it's scanning. The logs say it's removing the series' issues, but this is after multiple restarts and shutdowns. I even cleared out the CS folder, and did a new git clone. I don't know where the DB is stored, or I'd wipe it to and start fresh. Any ideas?

Error starting davide-romanini's fork

Beville's works normally.


E:\ComicStreamer2>python comicstreamer
Traceback (most recent call last):
  File "comicstreamer", line 6, in <module>
    from comicstreamerlib.main import main
  File "E:\ComicStreamer2\comicstreamerlib\main.py", line 35, in <module>
    from server import APIServer
  File "E:\ComicStreamer2\comicstreamerlib\server.py", line 57, in <module>
    from comicapi.comicarchive import *
  File "E:\ComicStreamer2\comicapi\comicarchive.py", line 32, in <module>
    from unrar import rarfile
  File "C:\Python27\lib\site-packages\unrar\rarfile.py", line 26, in <module>
    from unrar import unrarlib
  File "C:\Python27\lib\site-packages\unrar\unrarlib.py", line 45, in <module>
    unrarlib = ctypes.WinDLL(lib_path)
  File "C:\Python27\lib\ctypes\__init__.py", line 365, in __init__
    self._handle = _dlopen(self._name, mode)
WindowsError: [Error 126] The specified module could not be found

E:\ComicStreamer2>

Compiled links are 404. Is this project dead? It has been 3 years.

I didn't know about this project until just in the last hour. I was trying out some new comic reading apps on Android and in network and cloud sync areas of a 3 of the apps I tried I saw this ComicStreamer as an option. Sounding very interesting in comparison to Dropbox, Google Drive, etc. I came to check it out and possibly set it up but it seems dead to me.

I'm a Windows user so perhaps it's been abandoned on Windows. I don't know. It's interesting that comic book apps have implemented it's use in their apps.

First and foremost, I hope this creator is healthy and okay and Beville just got tired of working on it. It does happen. Or, Beville is doing time in San Quinton (just kidding). I would like to see this come back though or a statement from Beville as to the current situation.

I'd love to find a compiled Windows version to play with but all roads lead to dead ends. If anyone has a compiled version to share I'd be appreciative.

Best of luck.

Parse strip time error

Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 810, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 763, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/opt/comicstreamer/comicstreamerlib/monitor.py", line 81, in mainLoop
    self.dofullScan(self.paths)
  File "/opt/comicstreamer/comicstreamerlib/monitor.py", line 524, in dofullScan
    md = self.getComicMetadata( filename )
  File "/opt/comicstreamer/comicstreamerlib/monitor.py", line 169, in getComicMetadata
    if ca.seemsToBeAComicArchive():
  File "/opt/comicstreamer/libs/comictaggerlib/comicarchive.py", line 629, in seemsToBeAComicArchive
    ( self.getNumberOfPages() > 0)
  File "/opt/comicstreamer/libs/comictaggerlib/comicarchive.py", line 786, in getNumberOfPages
    self.page_count = len( self.getPageNameList( ) )
  File "/opt/comicstreamer/libs/comictaggerlib/comicarchive.py", line 762, in getPageNameList
    files = self.archiver.getArchiveFilenameList()
  File "/opt/comicstreamer/libs/comictaggerlib/comicarchive.py", line 396, in getArchiveFilenameList
    for item in rarc.infolist():
  File "/opt/comicstreamer/libs/comictaggerlib/UnRAR2/__init__.py", line 127, in infolist
    return list(self.infoiter())
  File "/opt/comicstreamer/libs/comictaggerlib/UnRAR2/__init__.py", line 122, in infoiter
    for params in RarFileImplementation.infoiter(self):
  File "/opt/comicstreamer/libs/comictaggerlib/UnRAR2/unix.py", line 171, in infoiter
    data['datetime'] = time.strptime(fields[2]+" "+fields[3], '%d-%m-%y %H:%M')
  File "/usr/lib/python2.7/_strptime.py", line 467, in _strptime_time
    return _strptime(data_string, format)[0]
  File "/usr/lib/python2.7/_strptime.py", line 325, in _strptime
    (data_string, format))
ValueError: time data '2011-07-13 18:06' does not match format '%d-%m-%y %H:%M'

Library limited to 5600 comic books?

Hi,
I have tried your comicStreamer App under Windows seven.
Ios Chunky comic reader developper gave me this address.
English is not my native language, so i'm sorry in advance if my english is not fluent.

I have configuered A very big library (more than 100000 comics), and the scan freezed at 2500 comics scanned.
Then I have deleted within the configuration the folder of 100000 comics and added just 4 folders (total of 5800 digital comics), and the scan stopped at 5600 comics.
I did same thing for a folder of 700 comics, it worked fine.

This app is promising, keep up the good work, and maybe library size will expand :)

Thank you and best regards

SUGGESTION: pdf support

I've a lot of comics in pdf format. They should be considered just as the other cbr/cbz/.. formats.
I wrote a super simple implementation using PyPDF2 and the command line tool mudraw (mupdf-tools) here [https://github.com/davide-romanini/ComicStreamer/commit/14eac20f26f3ecaccdd697a707e90500a0fb9b50].
I just made a small test with comictagger, don't know if it works on ComicStream (can't setup it on my machine, PIL related) but it could be a proof-of-concept for further improvements.

SUGGESTION: add reading direction for comics in database

It can be infered from comicRack's 'Manga' field as 'Yes (Right to Left)', and would be easy to add to CBI too.
This is very useful info when reading Mangas and Comics.
Chunky comics reader can already interpret it BTW :)

cannot launch on windows

Hi All,

I had ComicStreamer running for a few days on my Windows 7 Pro box but after I lost power yesterday, I haven't been able to get it back up. I deleted c:\ComicStreamer and re-cloned the repo but am receiving the following error. Any ideas what's up? I didn't get this error the first time around.

C:\Windows\system32>python c:\ComicStreamer\comicstreamer
Traceback (most recent call last):
File "c:\ComicStreamer\comicstreamer", line 2, in
from comicstreamerlib.main import main
File "c:\ComicStreamer\comicstreamerlib\main.py", line 35, in
from server import APIServer
File "c:\ComicStreamer\comicstreamerlib\server.py", line 57, in
from database import *
File "c:\ComicStreamer\comicstreamerlib\database.py", line 17, in
from sqlalchemy.ext.declarative import declarative_base
ImportError: No module named ext.declarative

Scannig library for the first time problem.

Scan status: SCANNING
Monitor: sorting recursive file list

I only entered my 'general' comics folder, which includes many subdirectories. Would this be responsible for the error?

Btw, total comics are no more than 5000

Compile OSX

My Question was how to compile it under osx but I almost already answered that myself.

Stuck with the init patch and the other one...

Hi! how do I compile this under OSX (and I mean fork 0.8 https://github.com/kounch/ComicStreamer)

Update It runs!

I followed the two hints there where resulting in:
osx 10.11
xcode 7.3 & brew

brew install git
brew install webp
brew install pyqt
brew install unrar
brew install homebrew/python/pillow
brew install libyaml
pip install PyPDF2
pip install configobj
pip install configparser
pip install python-dateutil
pip install watchdog
pip install tornado
pip install sqlalchemy
pip install pybonjour --allow-external pybonjour --allow-unverified pybonjour
pip install natsort
pip install pyinstaller
pip install pillow
pip install unrar
pip install pyobjc-core
pip install pyobjc

-- PDF stuff (should probably be included in the OSX version)
brew install mupdf-tools
brew unlink mupdf-tools
brew install homebrew/x11/mupdf

Download 3.2 and developer version from http://www.pyinstaller.org
Unzip in same parent folder as the ComicsStreamer folder (not "in" the ComicStreamer folder)
Rename developer folder to pyinstaller-develop
Patch mac/Makefile replace pyinstaller-2.0 with pyinstaller-3.2
Patch mac/Makefile replace 'which rar' with '-a ../libunrar' thus making/reading cp -a ../libunrar $(APP_BUNDLE)/Contents/MacOS
Download http://www.rarlab.com/rar_add.htm unrar source http://www.rarlab.com/rar/unrarsrc-5.3.11.tar.gz (or newer)
Compile libunrar.so with make lib -f makefile
Move libunrar.so to libunrar folder in the ComicStreamer source

Goto: mac/Makefile folder run make...

I did nothing with the init or rumps whatever... runs when I start it from within the Contents/MacOSX folder :-)
Ow I get it... the icon is not hidden and the package will not start but.. unless you run it from the terminal... and it does not contain the mudraw tool...

So now what? how do i fix that...

Maybe somebody add this to a readme!

Unable to determine what file/folder is hanging ComicStreamer

I'm attempting to add my sorted and unsorted comic folders to ComicStreamer. There is a persistent, repeatable hang when I add my sorted folder at approximately 1500 files in. However, when I check the logs, all I see is

DEBUG - 200 GET /scanstatus?api_key=XXXXXXXXXXXXXXXXXXXXX (::1) 0.00ms

over and over again. The number of milliseconds changes, but there's no indication of what file is causing the scan to choke. Help?

Comic importing fails when images inside a comic book have non US characters in their names

If any image file inside a comic book archive has non US ASCII characters (e.g. "Cómic 001.jpg"), comic streamer importer fails with an exception at ComicStreamer/libs/comictaggerlib/comicarchive.py:

def getPageNameList( self , sort_list=True):
(...)
                files = natsorted(files, key=keyfunc,signed=False)

This is the exception that is raised when comic importing stops:

(...)in natsorted
    UnicodeDecodeError: 'ascii' codec can't decode byte (...

Library not updating automatically

Hey, running this on a mac mini with osx 10.8.

To refresh the library i have to restart the server. Was there a setting somewhere to have the library automatically refresh which i'm missing? Currently it's not updating by itself.

Thanks.

Port Forwarding - IPv6 address

I can't seem to figure out how to get Astonishing Comic reader to find the server when I port forwarded. My Xfinity router/modem has assigned me an IPv6 address publically, which I think could be the reason. I have tested and the local network IP works just fine, but want to have access to this outside of my network. Any ideas?

REQUEST: Multiple Users

I don't know how possible this would be, but it would be amazing to have multiple users for ComicStreamer and to be able to track their read comics status separately.

Thanks for all your hard work on such an awesome project!

OSX Web Interface not respondig

When connected to a local network the reader hangs for a couple of minutes before displaying the webpage. Remove the network connection and it is instant. None of the computers has internet access and i tried it on 3 different macs fresly installed macs with osx 10.11.2... network connection it hangs, no connection it works (well i used to could click on the restart, rebuild en shutdown button and it used to display that it was scanning... all that is also not working, but it is scanning :-)

Mac menu bar icon

A couple things I noticed on my Macbook Pro (retina):

  1. When I hover over the menu bar icon, I get a spinner and cannot click on the icon.
  2. More of a quibble, but the icon is not retina.

Optimize thumbnail generation

Actually comicstreamer generates them "on the fly". For better resources usage they must at least cached, for example saving them on db. This is especially valuable on NASes where image resize is not always cheap.
Furthermore, a smarter approach could try to find an embedded thumbnail in the original image (in exif or xmp metadata).

Comic importer fails when credit role is not set

After auto-tagging some comics with ComicTagger, some of them appear to have bad data in Credits, with some names added, but with no role associated.

This breaks comic streamer importer, making it fail at ComicStreamer/comicstreamerlib/monitor.py:

def saveChildInfoToDB(self, md_list):
(...)
            if md.credits is not None:
                for credit in md.credits:
                    person_names.add(credit['person'].strip())
                    role_names.add(credit['role'].lower().strip())

Since there is no role (it is empty), an exception is raised and comic importing stops:

KeyError: 'role'

Can't access collection over internet. Only works on LAN.

I use Astonishing Comic Reader with ComicStreamer and after some tweaking I got it working, but only over LAN. If I put the IP address as 192.168.1.11 I can download and browse all my comics just fine. If I put my normal IP in, I get nothing.
I am sure I'm just missing something. I'm not a computer genius. I was only able to set up the service using a rather detailed guide. Do I need to format the IP a certain way, or add other info to get it working?
Would be amazingly helpful for travel, since my .cbr collection is much larger than my tablet's storage.

Unable to access the server from mobile Safari (iOS) or Chunky...

My brain hurts trying to figure out this one. I can access the server from Any browser (tried Edge and Chrome on Windows 10, and Chrome on iOS), but I'm unable to open/browse the server using Safari, which also means that Chunky can't open it either. It times out or gives you an NSURLErrorDomain error. Is it some sort of certificate issue? Maybe? This happen both over LAN and web. Ports are correctly forwarded, etc etc.

ComicStreamer should ignore folders the system considers hidden

Please make Comic Streamer ignore folders that the system considers hidden. This should be the default I think. The inclusion of these folders in the database scan seems silly.

Reason:
I've just started using Bit Torrent Sync (in OS X) on the folder I have ComicStreamer pointed at. Because Bit Torrent Sync creates a '.sync' folder in the root of each folder it syncs, the end result is I now have a lot of duplicated entries in Comic Streamer because it doesn't ignore the '.sync' folder.

SUGGESTION: Import ComicRack "Main Character" Metadata

Issue:
Main Character is a metadata piece that stores the main character or group of the comic. A lot of people use this to organize their comics one level above "Series" (as suggested in the official ComicRack Organizer).

Suggestion:
Import the "Main Character" metadata field when adding the files to the library and make that data available in the API. This will allow Chunky to import the data and provide a new way to access the comic library.

Small js fix

templates/cbreader.html line 40 should be:

            libPath: "{{ static_url('comicbook/js/') }}",

(needs the trailing / character or the path isn't created properly in comicbook.js.

I don't know if beville is still around, but I'm getting errors when I search by character.

I'm running from source. I start the server and immediately put "Spider-Man" into the character search bar. I get stuck with a blank search list, permanently "processing." And here's what the logs do:

2017-02-21 22:56:26,569 - INFO - Stream server running on port 7614...
2017-02-21 22:56:26,634 - INFO - Monitor: Beginning file scan...
2017-02-21 22:56:26,673 - WARNING - Cannot run bonjour server! Maybe some packages need to be installed?
2017-02-21 22:56:58,099 - INFO - Monitor: 8238 new files to scan...
Exception in thread Thread-1:
Traceback (most recent call last):
File "C:\Python27\lib\threading.py", line 801, in __bootstrap_inner
self.run()
File "C:\Python27\lib\threading.py", line 754, in run
self.__target(*self.__args, **self.__kwargs)
File "C:\Users\Zack\Desktop\Tools\ComicStreamer-0.0.7\comicstreamerlib\monitor.py", line 81, in mainLoop
self.dofullScan(self.paths)
File "C:\Users\Zack\Desktop\Tools\ComicStreamer-0.0.7\comicstreamerlib\monitor.py", line 535, in dofullScan
self.commitMetadataList(md_list)
File "C:\Users\Zack\Desktop\Tools\ComicStreamer-0.0.7\comicstreamerlib\monitor.py", line 470, in commitMetadataList
self.session.commit()
File "C:\Python27\lib\site-packages\sqlalchemy\orm\session.py", line 790, in commit
self.transaction.commit()
File "C:\Python27\lib\site-packages\sqlalchemy\orm\session.py", line 396, in commit
t[1].commit()
File "C:\Python27\lib\site-packages\sqlalchemy\engine\base.py", line 1574, in commit
self._do_commit()
File "C:\Python27\lib\site-packages\sqlalchemy\engine\base.py", line 1605, in _do_commit
self.connection._commit_impl()
File "C:\Python27\lib\site-packages\sqlalchemy\engine\base.py", line 690, in _commit_impl
self._handle_dbapi_exception(e, None, None, None, None)
File "C:\Python27\lib\site-packages\sqlalchemy\engine\base.py", line 1341, in _handle_dbapi_exception
exc_info
File "C:\Python27\lib\site-packages\sqlalchemy\util\compat.py", line 199, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb)
File "C:\Python27\lib\site-packages\sqlalchemy\engine\base.py", line 688, in _commit_impl
self.engine.dialect.do_commit(self.connection)
File "C:\Python27\lib\site-packages\sqlalchemy\engine\default.py", line 423, in do_commit
dbapi_connection.commit()
OperationalError: (sqlite3.OperationalError) database is locked

If I hit back, I'm able to see the home screen, but nothing else will load, the server needs to be restarted. I know my whole library isn't scanned yet, you'll see that it was scanning while I ran the search. But I've run into this same issue with a fully scanned library, I don't think that has anything to do with it. I wish I'd saved the logs. Searching by series, publisher, start date, end date, all definitely work fine. Team definitely makes the server freeze.

Trouble scanning files - No such file or directory

I just got comicstreamer running and it started its first scan. I saw this on the console:

2016-06-01 18:10:58,204 - INFO - Monitor: Removing missing or modified files from db (0 files)
2016-06-01 18:10:58,204 - INFO - Monitor: 442 new files to scan...
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/opt/boxen/homebrew/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/opt/boxen/homebrew/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/Users/jpollak/src/ComicStreamer/comicstreamerlib/monitor.py", line 82, in mainLoop
    self.dofullScan(self.paths)
  File "/Users/jpollak/src/ComicStreamer/comicstreamerlib/monitor.py", line 264, in dofullScan
    md = self.getComicMetadata(filename)
  File "/Users/jpollak/src/ComicStreamer/comicstreamerlib/monitor.py", line 191, in getComicMetadata
    image_data = ca.getPage(0)
  File "/Users/jpollak/src/ComicStreamer/comicapi/comicarchive.py", line 769, in getPage
    image_data = self.archiver.readArchiveFile( filename )
  File "/Users/jpollak/src/ComicStreamer/comicapi/comicarchive.py", line 576, in readArchiveFile
    return subprocess.check_output(['mudraw', '-o','-', self.path, str(int(os.path.basename(page_num)[:-4]))])
  File "/opt/boxen/homebrew/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 566, in check_output
    process = Popen(stdout=PIPE, *popenargs, **kwargs)
  File "/opt/boxen/homebrew/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 710, in __init__
    errread, errwrite)
  File "/opt/boxen/homebrew/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1335, in _execute_child
    raise child_exception
OSError: [Errno 2] No such file or directory

In the logs I see:

2016-06-01 18:11:02,968 - DEBUG - 200 GET /scanstatus?api_key= (::1) 0.43ms
2016-06-01 18:11:01,962 - DEBUG - 200 GET /scanstatus?api_key= (::1) 0.42ms
2016-06-01 18:11:00,956 - DEBUG - 200 GET /scanstatus?api_key= (::1) 0.46ms
2016-06-01 18:10:59,949 - DEBUG - 200 GET /scanstatus?api_key= (::1) 0.82ms
2016-06-01 18:10:59,121 - DEBUG - Reading in 1 /Volumes/media/comics/Unknown/Pathfinder 07 (88)/Pathfinder 07 - Unknown.pdf
2016-06-01 18:10:58,941 - DEBUG - 200 GET /scanstatus?api_key= (::1) 0.58ms
2016-06-01 18:10:58,327 - DEBUG - Reading in 0 /Volumes/media/comics/Shannon Watters/Lumberjanes_ Issue One (2)/Lumberjanes_ Issue One - Shannon Watters.epub
2016-06-01 18:10:58,204 - INFO - Monitor: 442 new files to scan...
2016-06-01 18:10:58,204 - INFO - Monitor: Removing missing or modified files from db (0 files)

I did verify that last file does exist:

❯❯❯ ls /Volumes/media/comics/Unknown/Pathfinder\ 07\ \(88\)/Pathfinder\ 07\ -\ Unknown.pdf   ⏎
/Volumes/media/comics/Unknown/Pathfinder 07 (88)/Pathfinder 07 - Unknown.pdf

Comic identifier

I am looking for a way to find doubles regardless of the archive type.

I am thinking about

unique hash of the filesizes of all the pages

such that moved comics can be enabled again in the library instead of deleted and as bonus i could create a list of doubles. which would probably save me a lot of gig's.

Anybody any thought on that?

SUGGESTION: json version of pulled data

It would be interesting for future app development if there was an option to force all data retrieved to be formatted as json information exchange format.
Something like
/publishers (gets me the web page)
/publishers.json (gets me the json formatted info for publishers)

Is there any option like that available or it's just me and my rumblings? ^.^

Config file

Due to not being able to scan the local network i unplugged it while in the config page... eventhough it loades it cannot save because the network shares are not available but if i enable the network it cannot save because it hangs... see other issue. so when the paths of the comics is not found it could be nice to save... maybe give a warning and present the user to save anyway instead of the error also when the server needs restarting why does it not ask the user if it want to restart?

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.