foxxmd / context-mod Goto Github PK

TODO before public announcement:

• Setup informational post on /u/ContextModBot
• Setup subreddit for satellite support/feedback
• Create issue template on github

Places to post announcement:

• r/botwatch
• r/bots (may be dead)
• r/bot
• r/ModSupport
  • precedent 1
  • precedent 2
• r/RequestABot
• r/modhelp
  • plenty of threads where people are asking for bot suggestions/help so seems appropriate
• Per this suggestion
  • r/thesefuckingaccounts
  • u/blogspammr
• Share on Reddit Mods discord server in #automation

Right now would need to pass all prefix content down from config builder all the way to action/rule if I want to see the context for some log statement on write. Need to refactor this so a logger can be created from a pool or something with prefix content already built. Basically make it more stateless to create a logger.

When clicking "Actioned Events" I get this error.

"Depleted in ~2 hours".

Does 2 hours signify an issue?
Is 2 hours a good thing?

Based on the description of karma farming from this thread.

Comments

Check the content of a comment activity against a list of "top" text comments based (and retrieved from) on the submission source:

• if submission is external attempt to get comments using some scraping method:
  • implement youtube api call to get top comments
    • https://www.googleapis.com/youtube/v3/commentThreads?key=${API_KEY}&textFormat=plainText&order=relevance&part=snippet&filter=snippet&videoId={VIDEO_ID}&maxResults=100
  • implement twitter api to get top replies
• if submission is reddit-based
  • check for cross-posted submissions and get top comments
  • search for other submissions with the same title and get top comments
  • allow user-defined white/blacklist for subreddits to include submissions from

For detecting a match:

• use fuzzy searching with user-defined threshold for sameness
• allow user-defined minimum character count

Submissions

• Do a reddit search for submission title and use fuzzy searching with user-defined threshold for sameness
  • Allow user-defined white/blacklist for subreddits search

• cache retrieved comments using a unique id based on submission source

• Restricting subreddit search

  • Define subreddits to search with this syntax: https://reddit.com/r/mealtimevideos+videos/search?q=${QUERY}&restrict_sr=1
  • For blacklist just filter subreddits out of returned results

• Search by submission url

  • Remove query string from submission url IE remove ?someQueryParam=... since reddit seems to only search by base url
  • Use url token in query to search by url IE .../search?q=url:${BASE_URL}

Originally suggested as being able to check for a hiatus/gap here by u/SillyStranger5009

Some examples of things that could be checked (to build a rule against):

• Within a date range if there are been a gap in author activity
• Within a date range find the baseline for frequency of activity and...
  • if there is a std deviation or percentage increase below or above the baseline
  • if the baseline frequency matches a user-defined value

How to apply?

• Allow specifying these sets of behaviors as criteria with AND/OR operands
• Allow specifying if these events must occur 1 or N times (per each or in total)
• Allow white/blacklisting subreddits to count events from

Not really context bot specific but it would be super helpful if the viewer also used/displayed the examples I've annotated. Will probably need to run the viewer locally and debug.

• Provide more variables to contextual data in RuleResult that can be used for templating
• On each rule document what data is available (using annotations so its available in schema)

Used the heroku link and then provided it the needed envs.

The site shows a heroku error and when trying to connect via heroku run bash it never connects.
It's just forever showing Running bash on ⬢ context-mod... / connecting, run.4890 (Hobby).

Any ideas if this is a heroku issue or something with the deploy?

• Determine memory usage for individual image (both raw and loaded into resemble)
• Determine what impact converting image to smaller size has on cumulative memory usage, comparison speed, and general cpu usage
• Potentially refactor image comparison object usage (repeat/recent reference objects) based on results from above

When

• Using attribution, repeat, or recent activity
• AND using submission as reference
• AND submission is an image

Implement a way to compare submission image to images from submissions in history.

Notes:

• What library?
  • Resemble.JS
    • Requires node-canvas binary
  • Rembrant.JS
    • Requires node-canvas binary
  • pixelmatch
    • Can't compare images with different dimensions
  • matches-subimage
    • Based on pixelmatch but can handle different sizes
• Need to determine if submission URL is an image
  • MIME type of downloaded resource?
  • By extension ending?

• Approve
• Ban user

Current dockerfile dependencies stem from when CM was using node-canvas for image comparison. It has been replaced by sharp, which only depends on libvips and may already be built into alpine...need to test what can be removed.

Create some well-documented common configurations that be used as a teaching tool as well as a valid jumping off point for new users.

• Add basic rule and action examples
• Add advanced concepts guide (caching, ordering, etc.)
• Add at least one yaml example
• Add full markdown templates to show off advanced mustache features
• Add footer example
• Add partials example once its implemented
• Add at least one complete subreddit example (config, templates, really world config values...)
• Add a "Getting Started" document to run through all of the above plus deployment

It'd be nice to not only be able to check a permalink but a whole user. This would be great for new spam as we could add the new spam rule and then run it against that whole account removing all their posts.

A mod may want any other bots processing activities to run before RCB. This could be achieved by:

• delaying activity processing after initial retrieval
• refreshing activity state after delay before continuing processing

Implement alternate property to allow pulling content from a wiki page.

• Add missing property annotations/comments
• Add default values
• Refactor interfaces to consolidate repeated properties (will make documentation easier going forward)

Once api is stable document endpoints and provide samples for authentication/usage

Would be cool to get Discord webhooks or something so that user can notified, outside of reddit, when a critical check runs

Reddit returns good information about Submissions with media links (youtube, vimeo, etc.)

Taking a page out of toolbox's playbook:

• aggregate an author's submission history on secure_media.oembed.author_url
• use large default window (100 or 200 submissions)
• allow trigger on threshold of author_url count (variant with useSubmissionAsReference)

Implement cloud templates so new users can deploy a working bot without having to figure out infrastructure.

• Heroku
• EC2
• Digital Ocean

There needs to be a page/section in the web ui where you can see the rules parsed into nice html elements. Similar to how the actioned events page maybe?

Implement run() in abstract class to handle run/skip based on include/exclude criteria

There are some good pointers in the ui but would be good to document in a readme along with hierarchy/tuning/etc.

Need to make action classes actually do something

• Lock
• Remove
• Report
• Flair
• Comment

I'd like to see a page with the results for every check that's run as well as a button to rerun the check on that post/comment.

Is there an env to use redis? I had assumed PROVIDER_STORE=redis would work.

So i can run it on the server

Need to encapsulate rule/action running in try-catch so they can fail but application can continue

I'd be great to allow both filtering by a string/regex and enabling the types of messages being output for example.

It'd be nice to enable/disable Reddit API Stats: Initial 582 | Current 582 | Used ~0 | Events ~0.11/s in the web logs. Same for this line Run Stats: Checks 1 | Rules => Total: 1 Unique: 1 Cached: 0 Rolling Avg: ~null/s | Actions 0.

If a page doesn't specify a lang attribute, a screen reader assumes that the page is in the default language that the user chose when setting up the screen reader. If the page isn't actually in the default language, then the screen reader might not announce the page's text correctly. Learn more.

Seems that actioned events max kept is based on total events not actioned ones meaning after x events the "actions events" page is cleared.

May be worthwhile to replicate repost sleuth bot on a (much) smaller level IE subreddit/bot level.

Things that would need to be or need to be considered...

• Support more than just redis for database?
• Definable (but optional) collections EX known spam, retired memes, all subreddit posts, etc...
• UI requirements
  • Upload image by file or URL
  • Batch upload from local directory?
  • Progress indicator for processing
  • Stats (number of fingerprints, search peformance)
• Rule refactoring to allow searching all, by collection, and/or user history
• user-configurable if database should be shared across bot/instance

So that users can pre-define mustache rendering fragments for a rule

• Add a partials property to rule json that is parsed on startup
• Allow partials property to be a string (one partial, name of rule) or object so many partials can be defined per rule
• Partial can be string to render or wiki: discriminator that retrieves wiki page contents that should then render
• Enforce partial name uniqueness

Enabling redis caching while in monolith mode I had assumed without a sub/op config the app would take this env and also cache reddit posts/comments but checking redis I'm not seeing anything but web sessions.

Currently cdnjs.cloudflare.com is used for some CSS.

Instead of showing the full link at the end of the log maybe [COM ID] could link to it?

If a submission or comment already has been replied to (at top level) from a moderator (or automod) then we don't want to also perform actions on it as we can assume it's already been manually actioned.

This shouldn't be an issue if the bot is using a fast poll time but want to cover all the bases.

Noticed this in my logs.

Checks 1 | Rules => Total: 1 Unique: 1 Cached: 0 Rolling Avg: ~null/s | Actions 0

Either a modal or a window popup should open with the results. Currently there isn't really any user feedback for submitting a link apart from the link disappearing.

These should be buttons as it doesn't take you to another page which is what a href aka link is for.

If the bot performs an action that submits content (comment, report) it should contain a footer saying:

• It's a bot
• Link to the bot subreddit
• Do not reply/DM directly to the bot for issues -- contact subreddit mods if sub issue, bot subreddit if technical issue

Right now all rule result summary data is uncensored IE thresholds, exact windows/totals are revealed in the text.

Need to implement user-facing summaries that redact specific, critical values so that spammers/bad-actors can't determine how to fly under the rader on triggering rules OR make sure there is enough raw data in the rule results for mods to build it themselves.

• How bot works
• Minimum reqs
• Link to json schema validation example
• Some basic examples
• ENV table and arguments
• Usage/logging

Currently you can load but not save the config from the editor.

Would like to integrate with toolbox usernotes to enable mod interaction tracking:

Integration requirements

• Opt-in using subreddit configuration
• Read (decode. uncompress) notes and store in cache
• Write (encode, compress) notes and invalidate cache

Implementation

• Add additional criteria to author filter/rule. Usernote criteria should be based on warning type, text, and count of notes (and/or some combination of all of the above)
• Add new Action to add usernote to Author