Code Monkey home page Code Monkey logo

calibre-annotations's Introduction

Support

Support for this plugin is in the MobileRead Forums. There is a thread dedicated to the plugin, https://www.mobileread.com/forums/showthread.php?t=241206. Please post any problems and requests there.

Developer notes

Table of Contents

  1. OVERVIEW
  2. DEVELOPING SUPPORT FOR A NEW DEVICE OR READER APP
  3. ADDING A NEW DEVICE OR READER APP TO THE RELEASED PLUGIN
  4. MODIFYING AN EXISTING DEVICE OR READER APP
  5. DEVELOPER MODE
  6. PROGRAM FLOW

The Annotations plugin is designed for extensibility. Classes supporting reader apps and devices are contained in individual files loaded at runtime.

The plugin architecture provides two methods of adding annotations to calibre:

  • Fetching annotations from a connected USB device.
  • Importing annotations from an app which exports annotations, typically by email or through iTunes.

The file reader_app_support.py contains classes which your code subclasses:

  • A USB reader device (Kindle, SONY, etc) subclasses USBReader().
  • An iOS reader application (iBooks, Marvin, Kindle for iOS) subclasses iOSReaderApp().
  • A reader application which exports annotations (Bluefire Reader, GoodReader) subclasses ExportingReader().

Note that some reader apps may support both methods of annotations (e.g., Marvin). Apps supporting both methods may be declared as subclasses of USBReader or iOSReaderApp.

Within your class, two class variables declare which methods are supported:

  • SUPPORTS_EXPORTING: This class declares a method, parse_exported_highlights(), which parses text or files supplied by the user.

  • SUPPORTS_FETCHING: This class declares two methods, get_installed_books() and get_active_annotations(), which probe the connected device for the information.

Your class may declare both variables to be true if the reader is capable of both fetching and exporting.


  • Save a copy of the appropriate sample class included in the plugin, SampleExportingApp.py or SampleFetchingClass.py to your machine. Rename the copy appropriately, e.g. MyAnnotationsClass.py.
  • After installing the Annotations plugin to your installation of calibre, go to the calibre configuration directory. From within calibre, you can open this directory from Preferences|Advanced|Miscellaneous|Open calibre configuration directory.
  • Exit calibre.
  • Within the calibre configuration directory, open plugins/annotations.json.
  • Locate the property additional_readers.
  • Change path/to/your/reader_class.py to point to the copy of the sample class you created on your machine.
  • Save annotations.json.

Your class will be dynamically imported when calibre launches.

Run calibre in debug mode to see diagnostic messages.

calibre-debug -g

If your code cannot be imported, calibre will exit, with the problem reported to the console. When your class loads successfully, you see its name listed during calibre initialization.

To begin developing your code, locate SUPPORTS_EXPORTING or SUPPORTS_FETCHING in your class. Change the applicable variables to True. Your class will now be enabled.

After editing your class, save the changes and restart calibre.

Refer to readers/Marvin.py or readers/GoodReader.py for functional working code.


After developing and debugging a new reader app class, generate a pull request and I will review it for inclusion with the plugin.


To make modifications to an existing reader app class:

  • Exit calibre.
  • Copy the reader app class source file from the readers/ folder to your desktop.
  • Within the calibre configuration directory, open plugins/annotations.json.
  • Locate the property additional_readers.
  • Change path/to/your/reader_class.py to point to your local copy.

When the plugin loads, it will now use the reader app class definition in your local copy instead of the built-in copy. After developing and testing your modifications, generate a pull request and I will review it for inclusion with the plugin.


Within annotations.json is an entry developer_mode. Setting this variable to true enables an additional menu item in the plugin dropdown menu, Remove all annotations. This command can be useful while developing your class.


action.py contains the InterfaceAction subclass implementation.

action:genesis() is called when calibre launches. genesis() sets up logging, creates an opts object which is used to access global properties throughout the plugin, instantiates the annotatations sqlite database, initializes the .json prefs file, loads any external reader classes specified in the prefs file, then inflates the help file.

InterfaceAction overrides:

  • initialization_complete() listens for device_changed signals at action:_on_device_connection_changed(), so that we are notified when a device recognized by calibre is connected.
  • library_changed() is called whenever the user changes libraries.
  • shutting_down() is called when calibre is about to exit, giving the plugin a chance to unmount the iDevice and save the log.
  • accept_enter_event(), accept_drag_move_event() and drop_event() support drag & drop methods.

After genesis() executes, the plugin is ready to respond to user input. There are three user-initiated paths to importing annotations:

  • User drags a .MRV, .MRVI or .TXT file to the plugin icon. If the dropped file has the proper mime content, drop_event() fires do_drop_event(), which passes the dropped file to each of the installed reader classes until it is successfully parsed, or informs the user that no installed class could parse the file.

  • User invokes the dropdown menu item Fetch annotations from [connected device] (when a supported device is connected).

  • User invokes the dropdown menu item Import annotations from [reader class].

There are two approaches to processing annotations, Fetching and Importing:

  • Fetching means probing a connected device to discover annotation content, usually stored in one or more sqlite databases, or some proprietary format.
  • Importing means parsing a file containing a proprietary annotations description.

Fetching and importing both result in storing the annotations to annotations.db, a sqlite database created and managed by the plugin.

  • Fetching classes instantiate AnnotatedBooksDialog() from action:fetch_device_annotations().

  • Importing classes instantiate AnnotatedBooksDialog() from action:present_annotated_books().

Each reader class stores annotations to annotations.db using a standardized schema. After the annotations are processed, the user is shown a summary by instantiating annotated_books:AnnotatedBooksDialog(). This class displays all available annotations, color-coded by metadata quality. The user can preview individual book's annotations, and enable or disable individual books for importation.

annotated_books:fetch_selected_annotations() builds a list of selected books.

Once the user approves the import list, action:process_selected_books() automatically adds annotations from the selected books to the associated library book if the metadata matches completely. For incomplete matches, a confirmation dialog is shown with the plugin's best guess for the receiving book in the library.

Additional plugin functionality

Find annotations (dropdown menu): find_annotations:FindAnnotationsDialog()

Modify appearance (configuration dialog): appearance:AnnotationsAppearance()


Last update June 10, 2013 7:56:25 AM MDT

calibre-annotations's People

Contributors

aik099 avatar davidfor avatar griker avatar stbenjam avatar wold5 avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar

calibre-annotations's Issues

Amazon "Send to Kindle" bug causing mismatches: include file names in matches as well?

Hi! I love this plugin, thank you so much for making it!

Unfortunately, for the last year, Amazon's "Send to Kindle" functionality has been incorrectly naming files it sends to the device. Specifically, the book name becomes the file name, so:

  • The book name becomes {title} - {author}
  • Colons get turned to underscores
  • Probably some other things get turned into underscores

As far as I can tell, Amazon doesn't seem ready to fix this. On its own this issue isn't a big deal: it means that book titles look a bit messed up in the library.

But this also means that the Annotations plugin considers everything imported since then a mismatch.

Calibre does let you manually match books in the library, but Annotations does not seem to include that info.

Would it be possible to make it so that the book name matching accepts {title} - {author}-style book titles (with stuff replaced with underscores)? Perhaps as a toggleable preference, perhaps only for Kindle.

Thanks!

Reporter metadata:

  • Calibre version: 6.10.0
  • Language: English
  • OS: Pop!_OS 20.04 LTS (Ubuntu)
  • Plugin version: 1.17.13

message to restart keeps coming op even after restarting (obviously)

For some reason calibre keeps asking me to restart calibre before I can customize Annotations plug in. I reverted back to the latest 4.xx version and back to 5.xx a few days ago, but after a reinstall of the plugin there was no change.

I am using:

  • Calibre 5.7.2
  • macOS 11.1 Big Sur
  • English (United Kingdom)
  • Latest 1.13 plug in

Pressing "Clear All" button on "Import Annotations" dialog does not remove ticks from checkboxes

After doing a fetch annotations, the "Import Annotations" list is displayed. This has a list of the books that annotations have been found for. At the beginning of each line, there is a checkbox with a tick in it. There is also a "Clear All" button under the list. Pressing the "Clear All" button does not appear to do anything except changing the text of the button to "Set All". But, when anything is done to change the list, the other checkboxes are cleared. The anything includes clicking on a line or resizing the window.

The reverse happens when the "Set All" button is pressed. The checkboxes are not ticked until something else causes a repaint.

This is with:
calibre 1.17.0
Windows 7 32 bit
English language
Annotations plugin 1.3.0.

Changing an annotation gets added as a new annotation

While testing for the Kobo devices, fetched the annotations, then I changed one on the device and did another fetch. The changed annotation was added as a new annotation in the column. Doing a third fetch, removed the original version and left only the second version.

After the first fetch, I had:

  <div class="annotation" genre=""
  hash="fe72eed43f4dedfcd9c003e1eafb0afd" location_sort="000200"
  reader="KoboTouch" style="margin:0 0 0.5em 0">
    <table cellpadding="0" width="100%"
    style="background-color:LightGray;color:black;font-size:80%;font-weight:bold;margin:0;"
    color="Gray">
      <tbody>
        <tr>
          <td class="location" style="text-align:left">Extra
          Chapters</td>
          <td class="timestamp" uts="1387924396.0"
          style="text-align:right">25/12/2013 09:33:16</td>
        </tr>
      </tbody>
    </table>
    <p class="highlight" style="margin:0;text-indent:0.5em;">Extra
    Chapters</p>
    <p class="note"
    style="font-size:90%;font-style:italic;margin:0;">0</p>
  </div>

After adding text to the annotation and fetching again, I have:

  <div class="annotation" genre=""
  hash="fe72eed43f4dedfcd9c003e1eafb0afd" location_sort="000200"
  reader="KoboTouch" style="margin:0 0 0.5em 0">
    <table cellpadding="0" width="100%"
    style="background-color:LightGray;color:black;font-size:80%;font-weight:bold;margin:0;"
    color="Gray">
      <tbody>
        <tr>
          <td class="location" style="text-align:left">Extra
          Chapters</td>
          <td class="timestamp" uts="1387924396.0"
          style="text-align:right">25/12/2013 09:33:16</td>
        </tr>
      </tbody>
    </table>
    <p class="highlight" style="margin:0;text-indent:0.5em;">Extra
    Chapters</p>
    <p class="highlight" style="margin:0;text-indent:0.5em;"></p>
    <p class="note"
    style="font-size:90%;font-style:italic;margin:0;">0</p>
    <p class="note"
    style="font-size:90%;font-style:italic;margin:0;"></p>
  </div>
  <div class="annotation" genre=""
  hash="8f8b5c8c247f7470afbd0ecc58ac74a1" location_sort="000200"
  reader="KoboTouch" style="margin:0 0 0.5em 0">
    <table cellpadding="0" width="100%"
    style="background-color:LightGray;color:black;font-size:80%;font-weight:bold;margin:0;"
    color="Gray">
      <tbody>
        <tr>
          <td class="location" style="text-align:left">Extra
          Chapters</td>
          <td class="timestamp" uts="1388271595.0"
          style="text-align:right">29/12/2013 09:59:55</td>
        </tr>
      </tbody>
    </table>
    <p class="highlight" style="margin:0;text-indent:0.5em;">Extra
    Chapters</p>
    <p class="note"
    style="font-size:90%;font-style:italic;margin:0;">0 - update
    1</p>
  </div>

On the third fetch, after disconnecting the device and restarting calibre, I get:

  <div class="annotation" genre=""
  hash="8f8b5c8c247f7470afbd0ecc58ac74a1" location_sort="000200"
  reader="KoboTouch" style="margin:0 0 0.5em 0">
    <table cellpadding="0" width="100%"
    style="background-color:LightGray;color:black;font-size:80%;font-weight:bold;margin:0;"
    color="Gray">
      <tbody>
        <tr>
          <td class="location" style="text-align:left">Extra
          Chapters</td>
          <td class="timestamp" uts="1388271595.0"
          style="text-align:right">29/12/2013 09:59:55</td>
        </tr>
      </tbody>
    </table>
    <p class="highlight" style="margin:0;text-indent:0.5em;">Extra
    Chapters</p>
    <p class="note"
    style="font-size:90%;font-style:italic;margin:0;">0 - update
    1</p>
  </div>

From looking at the code, the issue is the hash.

The method "merge_annotations", gets the old version of the annotations and the new version. It then gets a list of the hashes from both versions. Then it works out what hashes are in the new annotations but not the old. The annotations for these hashes are then added to old annotations.

The problem with this is the way the hashes are calculated. They are produced from the selected text and the annotation text. Changing one of these on the device will produce a new hash value. Hence they won't match and the changed annotation will appear as a new annotation.

I would suggest using the annotation_id. That shouldn't change when the annotation is modified (it definitely won't on the Kobo devices). Also, the hash can then be used to determine if the annotations have actually changed. If they haven't, the update of the metadata for the book can be skipped. This will mean the modification timestamp of the book will only change when there are actual changes to the metadata.

Add support for multiple libraries with different custom column assignments

(Generated from calibre plugins forum post)

The last thing is that I use several libraries. In my test library, I added a column called "Annotations" and selected it. Later, while using another library which doesn't have that column, I opened the Annotations configuration. And had the following error:

calibre, version 1.13.0
ERROR: Unhandled exception: TypeError:'NoneType' object has no attribute 'getitem'
calibre 1.13 isfrozen: True is64bit: False
Windows-7-6.1.7601-SP1 Windows ('32bit', 'WindowsPE')
('Windows', '7', '6.1.7601')
Python 2.7.5
Windows: ('7', '6.1.7601', 'SP1', 'Multiprocessor Free')
Traceback (most recent call last):
File "calibre_plugins.annotations.config", line 273, in annotations_destination_changed
File "calibre_plugins.annotations.common_utils", line 737, in existing_annotations
TypeError: 'NoneType' object has no attribute 'getitem'

After that, the configuration dialog is shown with no fields and calibre crashes. Windows shows the "calibre.exe has stopped working" error with the usual check online or close options.

Crash and hang when opening Find Annotations

I have had an occasional hang and crash when opening the "Find Annotations" dialog. It happens after the dialog is opened and while it is looking for the annotations. The following traceback was produced while running from source:

'''
calibre 1.17* isfrozen: True is64bit: False
Windows-7-6.1.7601-SP1 Windows ('32bit', 'WindowsPE')
('Windows', '7', '6.1.7601')
Python 2.7.5
Windows: ('7', '6.1.7601', 'SP1', 'Multiprocessor Free')
Traceback (most recent call last):
File "calibre_plugins.annotations.config", line 495, in run
File "calibre_plugins.annotations.config", line 539, in get_annotations_date_range
TypeError: 'NoneType' object has no attribute 'getitem'
'''
When I hunted around, I found that the problem was that I had an annotation that didn't have a timestamp. Because of what was there, I think that at some point I had edited the annotation and messed up the formatting. The error above happens because the timestamp attribute could not found and hence the value couldn't be retrieved. Adding a try-except block around this would solve the problem. It would also prevent issues if someone changed the timestamp value to something that wouldn't convert to a float.

This is also an issue in the new timestamp checking code when merging annotations. In that case, I don't know whether the edited annotations should be preserved, or automatically removed as they can't be matched with annotations coming from the device.

local variable 'ts_style' referenced before assignment

When trying to import annotations from my Kindle or when I attempt to modify the appearance, I get this error:

calibre, version 1.14.0
FEHLER: Nicht abgefangener Fehler: UnboundLocalError:local variable 'ts_style' referenced before assignment

calibre 1.14 [64bit] isfrozen: True is64bit: True
Windows-8-6.2.9200 Windows ('64bit', 'WindowsPE')
('Windows', '8', '6.2.9200')
Python 2.7.5
Windows: ('8', '6.2.9200', '', 'Multiprocessor Free')
Traceback (most recent call last):
File "calibre_plugins.annotations.config", line 304, in configure_appearance
File "calibre_plugins.annotations.appearance", line 452, in init
File "calibre_plugins.annotations.appearance", line 264, in initialize
File "calibre_plugins.annotations.appearance", line 243, in css_edited
File "calibre_plugins.annotations.appearance", line 330, in preview_css
File "calibre_plugins.annotations.annotations", line 137, in to_HTML
UnboundLocalError: local variable 'ts_style' referenced before assignment

German Language Version
Plugin-Version: 1.3.0

Add support for reading AlReaderXE-Ink annotations from connected Android device

Hi,

When using AlReaderXE-Ink app (installed on the Android-based Onyx Boox e-readers) the annotations are stored in 2 files:

  • citations - AlReaderXE-Ink/Sync/cites.txt (example: cites.txt)
  • bookmarks - AlReaderXE-Ink/Sync/bookmarks.txt (example: bookmarks.txt)

Even though Android devices connected via USB don't show up as USB drives, but Calibre is still able to see them via OpenMTP protocol.

File formats, that reader app can open: TXT, HTML, RTF, FB2, FB2.zip, DOC, DOCX, PRC, MOBI, CHM, EPUB, CBR, CBZ

The particular device I'm using is called Onyx Boox Nova 3 and is detected as Nova3 in Calibre. The same manufacturer is also producing other devices with the same file architecture, like Nova2, Nova, Poke3, Note3, but I'm not sure how each of them is named inside the Calibre. See https://downloads.boox.com/ for the full list.

In contrast to Kindle, the AlReaderXE-Ink folder is located at the top level of the device storage (not inside the folder, where books are located). The actual books are located at the top level Books folder, but Calibre already knows that by itself.

P.S.
Installing AlReader on any Android device (e.g. phone) can be done via Google Play (see https://play.google.com/store/apps/details?id=com.neverland.alreader). Likely when installed this way top-level folder might be named a bit otherwise, than AlReaderXE-Ink, but bookmarks/citations would work the same way.

Thank you.

Please read this before creating a new issue

When creating a new issue, please include:

  • The version of calibre you are using
  • Calibre's language setting (available in Preferences | Interface | Look and Feel | Choose language)
  • Your operating system, and 32- or 64-bit, if applicable. For example:
    • Windows 7, 64-bit
    • OS X 10.8.4
    • Ubuntu 12.04 32-bit
  • The version of the plugin installed
  • The specific steps to recreate the problem
  • A description of the problem

File "annotated_books - Copy.py" should not be in release

If you view the contents of the zipfile of the last couple releases, you will noticed a file named "annotated_books - Copy.py". Presumably this file was created by a developer on a Windows machine during the development process. The fact that it is there indicates that the person doing the release is not creating a release from a clean repo.

support Moon+ Reader

Hi,
thanks for your excellent work.
Do you have a try about Moon+ Reader (Moon+ Reader for Android )?
This app can export the annotations to a .mrexpt file, which is a plain text, and can be import by this app.
Can you add the function that import .mrexpt annotations to calibre?
thank your again.
look forward the new function.

PS:
I'm sorry,I made a mistake that report your post in MobileRead Forums ([GUI Plugin] Annotations - MobileRead Forums ).

Annotation fetching breaks with unhashable type: 'SortableTableWidgetItem'

Plugin version: 1.15
Calibre version: 5.16.1
OS: 64 bit Windows and Linux
Language: English

If I fetch annotations from my Kindle, it is able to collect the relevant annotations just fine (and shows them in a list), but if I click "Import Annotations", it starts importing and at around 90% it shows the following error:

calibre, version 5.16.1
Error fetching annotations: <p>Unable to fetch annotations from Amazon Kindle.</p>

Traceback (most recent call last):
  File "calibre_plugins.annotations.action", line 324, in fetch_device_annotations
  File "calibre_plugins.annotations.action", line 1080, in process_selected_books
  File "calibre_plugins.annotations.action", line 503, in generate_confidence
TypeError: unhashable type: 'SortableTableWidgetItem'

send annotations back to device

Hello, I've got an old kindle (compatible with the copy notes funcion of calibre), and I'd like to copy the notes to the new paperwhite 4, is it possibile to to this using calibre or your addon? Are there other methods? thanks

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.