sman591 / discourse-nntp-bridge Goto Github PK

View Code? Open in Web Editor NEW

19.0 19.0 1.0 99 KB

Discourse plugin to keep NNTP & Discourse in sync

License: MIT License

Ruby 100.00%

discourse nntp

discourse-nntp-bridge's People

Contributors

Stargazers

Watchers

Forkers

cloudxtreme

discourse-nntp-bridge's Issues

NNTP References Mess Up Importer's Post Organizer

When an NNTP post is made in a newsgroup (csh.b) that references a post in (csh.a) the bridge will place the post inline to the thread in csh.a, instead of in a new thread in csh.b (as it is located in NNTP proper).

Create tests for post importing & exporting

Currently only small parts have tests written for them. A good base to kick off from would be https://github.com/grantovich/CSH-WebNews/tree/api-v1/spec

As a user, I want the ability to make a discourse-only post

Maybe. Use case I can think of is our big move to Discourse, and wanting to make discourse-only pinned post that is only displayed to Discourse users.

If a newsgroup importer consistently throws an exception, excess Sidekiq retries build up

This could get messy.

Send NNTP control.cancel when a post is deleted

Undefined behaviour: If a post's reply is imported before it's original post

Discourse Schema Change

Job exception: PG::UndefinedColumn: ERROR:  column users.email does not exist
LINE 1: SELECT  "users".* FROM "users" WHERE "users"."email" = '[redacted]'...
                                             ^
: SELECT  "users".* FROM "users" WHERE "users"."email" = '[redacted]'  ORDER BY "users"."id" ASC LIMIT 1

The Discourse schema has changed from storing the email address of the user in the users table to having a users_emails table with foreign keys relating email addresses back to the user.

Separate NNTP config from plugin.rb

Either via environment variables or a config/nntp.yml file

Bad date/time on Cyrus

I am not able to get this plugin to work with Cyrus.
First, I had an issue where the 'rake discourse_nntp_bridge:assign_newsgroups' would not recognize newsgroups (cyrus shared folders). I figured that it's because Cyrus is case sensitive so I had to remove the .downcase off of this line

The next error I get is "Job exception: Bad date/time". I found no discrepancy between the date and time of the Cyrus and Discourse servers.
Here's the full backtrace from Discourses /logs

/var/www/discourse/plugins/discourse-nntp-bridge/gems/2.5.1/gems/thoughtafter-nntp-1.0.0.3/lib/nntp.rb:867:in `check_response'
/var/www/discourse/plugins/discourse-nntp-bridge/gems/2.5.1/gems/thoughtafter-nntp-1.0.0.3/lib/nntp.rb:836:in `shortcmd'
/var/www/discourse/plugins/discourse-nntp-bridge/gems/2.5.1/gems/thoughtafter-nntp-1.0.0.3/lib/nntp.rb:819:in `io_longcmd'
/var/www/discourse/plugins/discourse-nntp-bridge/gems/2.5.1/gems/thoughtafter-nntp-1.0.0.3/lib/nntp.rb:814:in `longcmd'
/var/www/discourse/plugins/discourse-nntp-bridge/gems/2.5.1/gems/thoughtafter-nntp-1.0.0.3/lib/nntp.rb:616:in `newnews'
/var/www/discourse/plugins/discourse-nntp-bridge/lib/discourse_nntp_bridge/server.rb:15:in `message_ids'
/var/www/discourse/plugins/discourse-nntp-bridge/lib/discourse_nntp_bridge/newsgroup_importer.rb:24:in `sync!'
/var/www/discourse/plugins/discourse-nntp-bridge/app/jobs/regular/nntp_bridge_importer.rb:10:in `execute'
/var/www/discourse/app/jobs/base.rb:137:in `block (2 levels) in perform'
/usr/local/lib/ruby/gems/2.5.0/gems/rails_multisite-2.0.4/lib/rails_multisite/connection_management.rb:63:in `with_connection'
/var/www/discourse/app/jobs/base.rb:127:in `block in perform'
/var/www/discourse/app/jobs/base.rb:123:in `each'
/var/www/discourse/app/jobs/base.rb:123:in `perform'
/usr/local/lib/ruby/gems/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:187:in `execute_job'
/usr/local/lib/ruby/gems/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:169:in `block (2 levels) in process'
/usr/local/lib/ruby/gems/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/middleware/chain.rb:128:in `block in invoke'
/var/www/discourse/lib/sidekiq/pausable.rb:81:in `call'
/usr/local/lib/ruby/gems/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/middleware/chain.rb:130:in `block in invoke'
/usr/local/lib/ruby/gems/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/middleware/chain.rb:133:in `invoke'
/usr/local/lib/ruby/gems/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:168:in `block in process'
/usr/local/lib/ruby/gems/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:139:in `block (6 levels) in dispatch'
/usr/local/lib/ruby/gems/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/job_retry.rb:98:in `local'
/usr/local/lib/ruby/gems/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:138:in `block (5 levels) in dispatch'
/usr/local/lib/ruby/gems/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq.rb:36:in `block in <module:Sidekiq>'
/usr/local/lib/ruby/gems/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:134:in `block (4 levels) in dispatch'
/usr/local/lib/ruby/gems/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:199:in `stats'
/usr/local/lib/ruby/gems/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:129:in `block (3 levels) in dispatch'
/usr/local/lib/ruby/gems/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/job_logger.rb:8:in `call'
/usr/local/lib/ruby/gems/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:128:in `block (2 levels) in dispatch'
/usr/local/lib/ruby/gems/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/job_retry.rb:73:in `global'
/usr/local/lib/ruby/gems/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:127:in `block in dispatch'
/usr/local/lib/ruby/gems/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/logging.rb:48:in `with_context'
/usr/local/lib/ruby/gems/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/logging.rb:42:in `with_job_hash_context'
/usr/local/lib/ruby/gems/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:126:in `dispatch'
/usr/local/lib/ruby/gems/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:167:in `process'
/usr/local/lib/ruby/gems/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:85:in `process_one'
/usr/local/lib/ruby/gems/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/processor.rb:73:in `run'
/usr/local/lib/ruby/gems/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/util.rb:16:in `watchdog'
/usr/local/lib/ruby/gems/2.5.0/gems/sidekiq-5.1.3/lib/sidekiq/util.rb:25:in `block in safe_thread'

Did anyone get this plugin working with Cyrus?

Handle NNTP control.cancel message imports

That's a feature. We should support it.

Move Discourse -> NNTP syncing to background worker

Discourse uses Sidekiq, so it should be pretty easy to create a custom worker for this.

If the worker fails (for a reason such as NNTP temorarily not connecting), it should retry.

NNTP quotes with comments directly below the quote get marked as quotes

Something such as the below would be interpreted as a single quote:

> Hello world!
Oh, hello there

New replies to a topic from NNTP don't bump the topic date

Create something like a 1 month old topic
(time goes by)
A reply is made on NNTP, which is then imported into discourse

On import, the topic's bump date isn't updated, which it should.

Stuart Olivera wrote:
(extra new line)
> ...