d1vanov / libquentier Goto Github PK

There is the following message in logs:
ibquentier/src/note_editor/SpellChecker_p.cpp @ 114 [Info]: Can't disable dictionary: no dictionary was found for language en_&GB

No description provided

Otherwise such a test won't launch under Windows.

No description provided

Need to do this properly though: the image should be written into a file, in the same file IO thread as all other images to ensure no races. Also, the undo/redo commands should be implemented.

No description provided

No description provided

Page 57 of Nixnote's User Documentation states:

The tricky part about sending changes to Evernote are tags. Before a new child tag can be created, its parent tag needs to exist at Evernote.

For instance, if I were to create TAG1 and TAG2 and set TAG2 as a child to TAG1, neither tag has a valid Evernote GUID so, when I synchronize, I need to be sure to synchronize TAG1 first, get the new GUID, update the parent GUID for TAG2, then finally synchronize TAG2. If there is an error with TAG1 and it can't be synchronized, then TAG2 will not be synchronized either. Most of this is accomplished with lists to keep track of what tags are ready to be synchronized.

Need to deal with this problem:

1. Sort the dirty tags need to be sent to Evernote by parent-child relations
2. Each time the parent tag is sent i.e. receives guid, update its child tags, set parent tag guid to them

Unfortunately, QDateTime is a huge mess which caused random crashes, see e.g. https://bugreports.qt.io/browse/QTBUG-49473. Need to migrate away from using it wherever possible.

Currently all the authentication stuff lives inside SynchronizationManager class' internals. It should be factored out to a separate class - preferably using some IAuthenticationManager interface that would actually be used by SynchronizationManager. The rationale is to avoid the hard dependency on QEverCloud's desktop OAuth - if there's no such dependency, porting the library to mobile platforms (which might happen some day, who knows) would be much simpler.

As I mentioned in hunspell/hunspell#549, although libhunspell itself doesn't offer compile-time check for availability of the new API, I could test for its availability using CMake's try_compile command: should add some tiny code piece using the new libhunspell API first appeared in 1.4.1 version. Then right after finding libhunspell and its development headers (in the end of cmake/modules/LibquentierFindHunspell.cmake) should try to compile this fragment of code and in case of success do something like add_definition(-DHUNSPELL_NEW_API_AVAILABLE). Then within the source code actually using libhunspell API (SpellChecker_p.cpp) split the code fragments using new and old libhunspell APIs using #ifdef HUNSPELL_NEW_API_AVAILABLE.

Testing: should build with both recent enough libhunspell and with some old one (1.3.x, for example) and ensure the correct branch of code works in each case - either with the help of debugger or via adding some trace-level log entry and watching for it.

No description provided

Ideally should have a string composed at CMake step containing git branch and last commit hash. If that information is not available (git not installed or build is not done from a git repo), can just default this information to "unknown".

TagModel in Quentier requests note count for each tag individually during the initial load of tag items from the local storage + on some events like note update. With many tags within the account it leads to many slot invokations which makes the UI thread unresponsive for a couple of seconds.

The suggestion is to implement the method to request note counts for all tags at once so that there's the only one slot invokation which is then just parsed by the TagModel in Quentier.

• Should remove as much logics of log entry composing as possible from the public header. If that means file name and line number should become separate arguments to QuentierLogger::addLogEntry, so be it.
• Should write the log to file within a separate thread, otherwise it's too damn slow

By default the conflicting copies notes would be put in the same notebooks as the source notes but it should be possible to specify the name of a dedicated notebook to hold the conflicting notes.

All other signals & slots referring to counts have "get" prefix. Need to make the notes' ones conform.

It is of little to no use outside note editor. And for the note editor it is merely an implementation detail.

That should make the process of multiple notes downloading much faster.

The time has finally come to start doing this.

There are (at least) two places where the words' highlighting is used: Find and replace and Spell checker. Perhaps, the newly added findAndReplaceDOMText script could help in both places.

findAndReplaceDOMText(node, {
 find: 'something',
 wrap: 'span',
 wrapClass: 'hilitorHelper'
});

1. QT_NO_CAST_FROM_ASCII
2. QT_NO_CAST_TO_ASCII
3. QT_NO_CAST_FROM_BYTEARRAY
4. QT_NO_NARROWING_CONVERSIONS_IN_CONNECT (actual since Qt 5.8)

It's hard to predict how much would need fixing after the introduction of either of these macros. If it turns out to be overwhelming, can as well just cancel that. But if it's manageable, it would be good to introduce these.

Turned out, Evernote sends incomplete resource objects within the conventional sync chunks: they lack the data body so it needs to be downloaded separately. Without that the sync fails if any resource is present within the sync chunk without its note.

Currently libquentier includes API for providing stack traces: SysInfo::stackTrace (in headers/quentier/utility/SysInfo.h). Currently no Windows backend is actually implemented for it so the method just returns the following string: Stack trace obtaining is not implemented on Windows, patches are welcome.

Boost stracktrace library is available since boost 1.63 and includes more or less cross-platform support for getting stack traces. Would be nice to introduce optional use of this boost library by libquentier and see how it goes. By optional I mean either use this library or use the "old way" i.e. the current implementation doing nothing on Windows. Probably will also need to expose the use of this library in VersionInfo.h (auto-configured file by CMake from template at `headers/quentier/utility/VersionInfo.h.in).

The current name is knda silly, I just didn't think enough about what it should be when introducing it.

Need to figure out how to generate .ts files and how to update .qm files as necessary using CMake. Also need to verify that lupdate runs successfully and generates something meaningful/workable.

Also need to find out where the translations for the library are to be installed to be visible for the application.

For each request there should be one corresponding signal indicating the failure of the request - the generic one is difficult to use properly as it's unclear in response to what it's being sent and what should one do upon receiving it.

Need to write .appveyor.yml file describing the build process along with installing all available pre-built dependencies and building those not available.

Most probably it has something to do with Unicode i.e. with missing support for it in JavaScript. Not sure I'd be able to handle this but need to try at least.

Per Evernote's spec notebooks, tags and saved searches need to have unique names in case-insensitive manner. Need to have a smart conflict resolution logic when syncing the changes from the remote storage to the local one.

Example of a non-trivial situation:

1. In an Evernote client, do the following: suppose there was a tag called "bash"; rename it to "bash2" and then create a new tag called "bash3" but don't sync yet.
2. Go to Evernote web, rename "bash" tag to "bash3" there.
3. Return back to Evernote client and run the sync. What would happen?

Mac's official Evernote client renames "bash2" tag to "bash3" (i.e. the remote changes wins over the local one) and renames the local "bash3" tag to "bash3 1" i.e. changes the name of the different tag conflicting with the downloaded one by name in order to avoid the conflict.

Need to use this logic, only maybe to add some meaningful prefix to the name, like " - conflicted" or " - conflicted (n)" where would be the name of the conflicting local item and n would be some number >= 2 purely for the resolution of potential multiple conflicting items.

Currently the source code of ENMLConverter simply hardcodes the transformations between ENML and HTML to fulfill the needs of the note editor. It should be more flexible.

By default all the found dictionaries can be used but it would also be nice to allow user to filter them by the dictionary's name (or language code if the name matches one).
It should be made easy to switch this setting - for example, via the note editor's context menu.

Not quite sure but it looks like QImage doesn't give away the memory for once loaded image. Need to run an experiment: don't store the thumbnail data as QImage within Note, instead just store it as QByteArray and see if it helps to reduce the memory consumption.

For now without syntax highlighting but as simple as the following:

#!xml

</div>This is a second command I used on Linux:<br clear="none" /></div><div><pre style="font-family: Monaco, Menlo, Consolas, 'Courier New', monospace; font-size: 0.9em; border-radius: 4px; letter-spacing: 0.015em; padding: 1em; border: 1px solid #cccccc; background-color: #f8f8f8; overflow-x: auto;">$ sudo second command</pre></div><div><br /></div>

The error text says: "Internal error: empty note store url for the linked notebook's note store".

Some surrounding content from the log:

libquentier/src/synchronization/RemoteToLocalSynchronizationManager.cpp @ 4636 [Debug]: "Got authentication tokens for all linked notebooks, can proceed with their synchronization"
libquentier/src/synchronization/RemoteToLocalSynchronizationManager.cpp @ 4811 [Debug]: "RemoteToLocalSynchronizationManager::downloadLinkedNotebooksSyncChunks"
libquentier/src/synchronization/RemoteToLocalSynchronizationManager.cpp @ 4727 [Debug]: "RemoteToLocalSynchronizationManager::getLinkedNotebookSyncState"
libquentier/src/synchronization/SynchronizationManager_p.cpp @ 1680 [Debug]: "SynchronizationManagerPrivate::noteStoreForLinkedNotebookGuid: guid = ""a93f6851-5107-4575-adab-cf43aa54fd0d"
libquentier/src/synchronization/SynchronizationManager_p.cpp @ 1693 [Debug]: "Found no existing note store corresponding to linked notebook guid ""a93f6851-5107-4575-adab-cf43aa54fd0d"
libquentier/src/synchronization/SynchronizationManager_p.cpp @ 569 [Debug]: "SynchronizationManagerPrivate::onRemoteToLocalSyncFailure: ""Internal error: empty note store url for the linked notebook's note store"

Linked notebooks are a funny feature of Evernote. When a linked notebook is added to your account, you get a notebook, its notes and all the tags associated with those notes. But no other tags from the linked notebook owner's account. As it was revealed empirically, that means that sometimes you get tags with parent tag guids which are inaccessible. The official Evernote client just shows such tags as if they had no parent and it appears I need to do the same thing.

But this logic needs to be implemented on sync algorithm side, not on local storage side because the check for parent tag's accessibility within local storage is useful as it prevents mistakes in code from messing up the database.

As a first step, need to finally migrate from using QList of qevercloud::Tag to some indexable data structure - indexable by guid, by name, by parent tag guid and by linked notebook guid. Lookup complexity is important because tags can easily hit the number of a thousand (at least during the full sync).

https://dev.evernote.com/doc/articles/note_links.php

I.e. need to support the links starting with evernote:///: the note editor should emit a special signal like "note link clicked" with the guid extracted from the link's URL.

Also need to have a dedicated API for insertion of a note link given note guid, user id and shard id.

No description provided

Right now there is no special processing i.e. the HTML just gets pasted as is. It doesn't hurt the conversion of the note to ENML since all forbidden/unsupported tags and attributes are eliminated but

1. They are only eliminated at the point of conversion to ENML so would be visible only after the next conversion from ENML back into HTML
2. The pasted HTML might contain img tags pointing to some external images. These need to be downloaded (or at least attempted to be downloaded) and attached to the note as new resources.

Hopefully it should be possible to convert the HTML about to be pasted into valid XML via libtidy5 and then parse it via QXmlStreamReader

Steps to reproduce: input a misspelled word and try to replace it using context menu.

Currently SpellChecker class (from headers/quentier/note_editor/SpellChecker.h) provides quite a limited support for dealing with dictionaries it is using:

1. The dictionary files (.dic and .aff files used by hunspell backend) are detected automatically, without any user's intervention, their locations are persisted with the help of QSettings. That's mostly good only this way doesn't provide much flexibility for cases when new dictionaries are installed, some existing ones are uninstalled, their location changes etc.
2. Currently it is only possible to list the language codes of dictionaries used by SpellChecker + methods to enable/disable the certain dictionary.

Although it sort of leaks the implementation detail, I believe it'd be worth it to add new methods allowing to associate language codes with particular .dic and .aff files and thus provide the management of these, for example:

class QUENTIER_EXPORT SpellChecker: public QObject
{
    Q_OBJECT
public:
<...>
    class QUENTIER_EXPORT Dictionary: public Printable
    {
    public:
        Dictionary();
        Dictionary(const QString & languageCode, const QString & dicFileLocation,
                         const QString & affFileLocation, const bool enabled);

        const QString & languageCode() const;
        void setLanguageCode(const QString & languageCode);
 
        const QString & dicFileLocation() const;
        void setDicFileLocation(const QString & dicFileLocation);

        const QString & affFileLocation() const;
        void setAffFileLocation(const QString & affFileLocation);
        
        bool isEnabled() const;
        void setEnabled(const bool enabled);
        
        /**
         * @return True if dictionary object is valid i.e. its language code is valid and its paths to dic and aff files are valid i.e. they are path to existing readable files which base names correspond to the language code and which extensions correspond to dic and aff correspondingly, false otherwise
         */
        bool isValid() const;
        
        virtual QTextStream & print(QTextStream & strm) const Q_DECL_OVERRIDE;
        
    private:
        QString     m_languageCode;
        QString     m_dicFileLocation;
        QString     m_affFileLocation;
        bool          m_enabled;
    };

    QVector<Dictionary> listAvailableDictionaries() const;

    /**
     * @return existing valid dictionary for the provided language code, invalid dictionary otherwise
     */
    Dictionary findDictionary(const QString & languageCode) const;

    /**
     * @return True if dictionary corresponding to the language code was successfully enabled, false otherwise
     */
    bool enableDictionary(const QString & languageCode);

    /**
     * @return True if dictionary corresponding to the language code was successfully disabled, false otherwise
     */
    bool disableDictionary(const QString & languageCode);
    
    /**
     * @return True if valid dictionary was successfully added to or updated within the list of available dictionaries, false if the dictionary is invalid
     */
    bool addOrUpdateDictionary(const Dictionary & dictionary);
    
    /**
     * @return True if the dictionary corresponding to the language code was successfully removed from the list of available dictionaries, false if no dictionary corresponding to the language code was found
     */
    bool removeDictionary(const QString & languageCode);
};

    // TODO: should replace this with some more proper way                                                              
    this.clearHighlightTags = function(html) {                                                                          
        var highlightOpenTagStart = 0;                                                                                  
        while(true) {                                                                                                   
            highlightOpenTagStart = html.indexOf("<span", highlightOpenTagStart);                                       
            if (highlightOpenTagStart < 0) {                                                                            
                break;                                                                                                  
            }                                                                                                           
                                                                                                                        
            var highlightOpenTagEnd = html.indexOf(">", highlightOpenTagStart);                                         
            if (highlightOpenTagEnd < 0) {                                                                              
                break;                                                                                                  
            }                                                                                                           
                                                                                                                        
            var highlightCloseTagStart = html.indexOf("</span>", highlightOpenTagEnd);                                  
            if (highlightCloseTagStart < 0) {                                                                           
                break;                                                                                                  
            }                                                                                                           
                                                                                                                        
            var classAttributeValueStart = html.indexOf("hilitorHelper");                                               
            if ((classAttributeValueStart <= highlightOpenTagStart) && (classAttributeValueStart >= highlightOpenTagEnd)) {
                continue;                                                                                               
            }                                                                                                           
                                                                                                                        
            var highlightedText = html.substring(highlightOpenTagEnd + 1, highlightCloseTagStart);                      
            html = html.substring(0, highlightOpenTagStart) + highlightedText + html.substring(highlightCloseTagStart + 5, html.length);
        }                                                                                                               
                                                                                                                        
        return html;                                                                                                    
    }

highlightOpenTagStart should be less than highlightOpenTagEnd, therefore the condition (classAttributeValueStart <= highlightOpenTagStart) && (classAttributeValueStart >= highlightOpenTagEnd) doesn't seem to make much sense, it should be || instead of && it seems.

Easily reproduces by input a same word into the find and replace fields. Then press Replace all button - Note editor hangs and app crashes on exit.
Actually, the words might be not same but similar, like 'wake' and 'awake'.

minidump.tar.gz

Synchronization logic is quite complex and definitely needs some regression tests. However, before we can have the tests, we need the infrastructure for them i.e. we need some way to fake the Evernote servers communicating with the local app. I see several possible solutions:

1. Add INoteStore interface which would be implemented by NoteStore (for production synchronization) and FakeNoteStore (for tests). The injection of a particular note store would then need to happen in runtime. It means some switch would be needed to tell libquentier it needs to inject the fake note store instead of the real one for tests. It can be a dedicated API method or it can be something better hidden, for example, examining some environment variable like LIBQUENTIER_USE_FAKE_NOTE_STORE_FOR_SYNCHRONIZATION_TESTING. Or it might examine the value within some settings file instead of the environment variable.
2. Use some trick with preloading some shared library "catching" the calls to QEverCloud's methods. On Linux it could work with the help of LD_PRELOAD but not sure if there are equivalents on Windows and Mac.
3. Implement fake server on the local host with which QEverCloud would communicate. Most complex of all variants as it would require decoding and encoding of data from/to Thrift binary format. QEverCloud currently doesn't export the stuff related to that.

It inserts a broken HTML:

<div>
<table collapse;="" margin-left:="" 0px;="" table-layout:="" fixed;="" width:="" 400px;\x22=""><tbody>
<tr><td 1px="" solid="" rgb(219,="" 219,="" 219);="" padding:="" 10="" px;="" margin:="" 0px;="" width:="" 400px;\x22=""><div><br></div></td></tr>
</tbody></table>
</div>

Fatal log level in several logging frameworks terminates the application in addition to printing the "fatal" log entry. Currently libquentier's fatal log level doesn't terminate the application and thus can be the source of confusion. On the other hand, I don't really want to encourage anyone to terminate the application in this way as well. So it's better to just get rid of this log level and replace its occurrences within the source code by Error log entries or even Warning log entries.

Also need to change QNCRITICAL macro to QNERROR since that would actually correspond to Error log level.

That's what Evernote's native client does and that's what's convenient and kinda even expected. It's rather hard though because it involves

1. Capturing the newline insertion with Enter
2. Trying to figure out the contents of the current line - i.e. does it start with a TODO checkbox
3. If so, we should insert the newline and then insert a TODO checkbox

Need to write .travis.yml file describing the build process along with installing all available pre-built dependencies and building those not available.