entry_dedupe flip-flops between two entries on every update; this seems to be because

Related: <a class="issue-link js-issue-link" data-error-text="Failed to load title" da

The issue fixed by <a class="commit-link" data-hovercard-type="commit" data-hovercard-

entry_dedupe flip-flops between entries about reader HOT 3 CLOSED

lemon24 commented on July 18, 2024

entry_dedupe flip-flops between entries

from reader.

Comments (3)

lemon24 commented on July 18, 2024

Related: #292 / c7e516f.

from reader.

lemon24 commented on July 18, 2024

Hmm... it seems recent sort was not set correctly (new entry shows way up in the feed); need to look more into this.

Update: also, it doesn't stop the flip flopping.

Update #2: it did stop the flip flopping, but update still adds one issue every time (the one the plugin deletes right away); this is expected / unavoidable without some kind of tombstone (#96 (comment)).

from reader.

lemon24 commented on July 18, 2024

The issue fixed by fc80a49 (and in part, this whole issue), happened because dedupe happens at different points between "on-line" dedupe (after an entry is added/updated), and backfill (on demand, for groups of entries that are all the same). This could arguably be fixed by making "on-line" be more like backfill; related: #246 (comment).

Also related, in case we unify the pipelines:

reader/src/reader/plugins/entry_dedupe.py

Lines 274 to 283 in fc80a49

    
           # unlike _get_entry_groups, we cannot rely on e.last_updated, 
        
           # because for duplicates in the feed, we'd end up flip-flopping 
        
           # (on the first update, entry 1 is deleted and entry 2 remains; 
        
           # on the second update, entry 1 remains because it's new, 
        
           # and entry 2 is deleted because it's not modified, 
        
           # has lower last_updated, and no update hook runs for it; repeat). 
        
           # 
        
           # it would be more correct to sort by (is in new feed, last_retrieved), 
        
           # but as of 3.14, we don't know about existing but not modified entries 
        
           # (the hook isn't called), and entries don't have last_retrieved.

from reader.

Recommend Projects

entry_dedupe flip-flops between entries about reader HOT 3 CLOSED

Comments (3)

Related Issues (20)

Recommend Projects

React

Vue.js

Typescript

TensorFlow

Django

Laravel

D3

Recommend Topics

javascript

web

server

Machine learning

Visualization

Game

Recommend Org

Facebook

Microsoft

Google

Alibaba

D3

Tencent

	# unlike _get_entry_groups, we cannot rely on e.last_updated,
	# because for duplicates in the feed, we'd end up flip-flopping
	# (on the first update, entry 1 is deleted and entry 2 remains;
	# on the second update, entry 1 remains because it's new,
	# and entry 2 is deleted because it's not modified,
	# has lower last_updated, and no update hook runs for it; repeat).
	#
	# it would be more correct to sort by (is in new feed, last_retrieved),
	# but as of 3.14, we don't know about existing but not modified entries
	# (the hook isn't called), and entries don't have last_retrieved.