steem-fossbot / steem-fossbot-voter Goto Github PK

Add support to add multiple accounts to the voter.

Maybe a section where you can add/remove/modify all the keys that are stored in the script.

This would allow people with multiple accounts to use the bot more efficiently.

We only have positive languages checking, and only for four languages: English, German, Spanish and French.

Using the library already used, we can check for many more.

For Asian languages, we would need to do something else, but probably a character set match would work well enough.

Additionally, we could compliment the positive language checking with negative checking, such as post_has_no_english_language_use

It is difficult to use the keyword white / blacklist metrics without knowing what kind of keywords are being extracted from posts.

Keywords extracted should be shown on the stats page, and appear in the email / log summaries.

Note that this would require this information being saved in the postsMetadata internal structure.

Currently the steem.js API method getFollowing returns the followers and ignored / muted users together.

These are differentiated by the what field which is blog for followed users and ignore for muted / ignored users.

On first run the option should be given to run on more than the last 100 newest posts.

Not sure how best to present this option to the user though?

Last Log currently will show the last log of the Bot Run if digest is not set, or the digest email if digest is set.

Last Log should show the last Bot Run. This was it will be better for "debugging" an algorithm.

So I had an issue where using the "author_is_followed, weight = 1000" algorithm was skipping quite a few of my followed authors posts in my feed. Adjusting MIN_SCORE_THRESHOLD to be lower than the default value seemed to clear this issue up. I will have to play with it more to tune it in better, but figured I'd mention this since it is not included yet in the documentation for "author_is_followed, weight = 1000".

Mentioning carefully tweaking MIN_SCORE_THRESHOLD slowly until feed posts are voted on as desired in the docs might come in handy for some folks using "author_is_followed, weight = 1000". Great bot, thanks a ton for it. Done initial testing, that's all I found so far. Will definitely be firing up my clean machine for a donation very soon. :)

Be best way to implement both #4 Golos support and #38 try out steemdata would be to abstract blockchain access away from usage code and provide a standard interface to fetch data and request changes (e.g. cast votes) to the blockchain.

Note that these different implementation can be switched at run time so there will be no need to fork the project for Golos support. Similarly, we don't need to permanently choose between steem.js and steemdata, or any other blockchain access method for that matter.

A separate ticket should be raised to add blockchain access method changing in the UI.

The Heroku Scheduler add-on is the suggested method for triggering bot runs (iterations where votes are cast) as described in the installation instructions.

It should be very clearly noted that it is a "best effort" service and may not run every time it is supposed to.

From their manual page:

Scheduler is a “best effort” service, meaning that execution is expected but not guaranteed. Scheduler is known to occasionally (but rarely) miss the execution of scheduled jobs. If scheduled jobs are a critical component of your application, it is recommended to run a custom clock process instead for more reliability, control, and visibility.

This is important information and do avoid confusion and disappointment it should be clear in the instructions.

Currently, a user is shown a static message after a successful bot run if triggered manually by them from the dashboard.

There's no technical reason why we should not direct them immediately to the voting stats for that run. This should be done for ease of use and better UX

Note, this a bug because users could be very confused by some of these

Some features have not been fully documented and there exist some quirks of the software which should be clearly mentioned in the docs.

• If only one metric with weight >= MIN_SCORE_THRESHOLD, percentage increase is not applied, and voter regulation is only added when voting limit is reached, as a hard value. This is to support algorithms with few metrics to work correctly. Should be clear in the UI that this condition is activated

For some use cases, it would be good to throw a random score into the mix for every post scoring.

This would be particularly useful for the use case where all the bot does it vote for a large list of whitelisted or followed users.

The random score could be between 0.0 and 1.0 (as is usual for computer library random number generators), and normally distributed. Then the algorithm weight becomes the maximum score to be added (or subtracted, if negative).

Suggested key name: random_normal

Could use other flavours for other probability distributions, e.g. Poisson, exponential, log-normal, etc.

To support the Golos version (Russian), and potentially any other language, we should take out all English language string literals and replace with a translations database.

I suggest we use POEditor for this, it works well, exports directly to GitHub and is free to use, with certain restrictions.

Should add separate ticket to create language selection method, and to translate for each desired language, certainly Russian at least

how about a metric for all posts which appear in my feed ?

Regards

When issue #31 is complete, we can add metrics related to muted / ignored users, which are the opposite of followed user metrics:

• post_up_voted_num_muted: Number of muted voters who up voted on post
• post_down_voted_num_muted: Number of muted voters who down voted on post
• post_up_voted_any_muted: At least one muted voter up voted on post
• post_down_voted_any_muted: At least one muted voter down voted on post
• author_is_muted: Author is muted

Some of the metrics are based on live data which can change, such as who has voted for it, current estimated value, etc.

Should posts which have already been scored be scored again?

Not sure what the best way to do this would be? Maybe the entire last run of posts can get looked at again? Really want to avoid things getting messy though.

Current the default MIN_POST_AGE_TO_CONSIDER is set to 30 minutes. This is because in this post it is confirmed that the curation reward is

curation_reward_weight = standard_weight * [(now - post_create_time) / 30_minutes]^2

Voting for a post after 30 minutes or more gives 100% of the curation reward to the curator, as

(30 minutes / 30)^2) = 1 which is 100%

It's anyone's choice how this wish to vote, but as a default we could set the MIN_POST_AGE_TO_CONSIDER so that the curation reward is at 50%, which gives back to authors and could be argued is better for the community.

To find this use

(21.2132034 minutes / 30)^2 = 0.4999999983 which is approx. 50%

Note also that this is just the minimum post age. Depending on when posts are posted, and how often the bot is run, it is highly likely that older posts will be voted for. So what we are really saying is that we want at least 50% of the reward, and will probably take more most of the time.

Steemdata is reported to have massively increased access speeds and modern query building using a standard MongoDB interface to the Steem blockchain.

It could greatly improve existing queries and also make practical the kind of large data queries needed to implement metrics requiring, for example, summary statistics of the last 100 posts of a certain author, etc.

Needs further investigation but looks promising

I am seeing errors in my apps logs similar to the following when trying to access my app on heroku.

2017-02-17T13:05:49.741203+00:00 heroku[router]: at=error code=H10 desc="App crashed" method=GET path="/" host=f5.herokuapp.com request_id=e61139e8-9ae8-43dd-97fa-a991e4c4e2bb fwd="" dyno= connect= service= status=503 bytes=

Additionally, when the scheduler runs, it seems to hit an exception when making the api call, as seen by the following log entries;

2017-02-17T13:06:02.540589+00:00 app[scheduler.4785]: events.js:160
2017-02-17T13:06:02.540591+00:00 app[scheduler.4785]: throw er; // Unhandled 'error' event
2017-02-17T13:06:02.540591+00:00 app[scheduler.4785]: ^
2017-02-17T13:06:02.540592+00:00 app[scheduler.4785]:
2017-02-17T13:06:02.540593+00:00 app[scheduler.4785]: Error: unexpected server response (301)
2017-02-17T13:06:02.540593+00:00 app[scheduler.4785]: at ClientRequest.response (/app/node_modules/ws/lib/WebSocket.js:719:15)
2017-02-17T13:06:02.540594+00:00 app[scheduler.4785]: at ClientRequest.g (events.js:291:16)
2017-02-17T13:06:02.540595+00:00 app[scheduler.4785]: at emitOne (events.js:96:13)

This seems to have started happening after recent updates, as per the following link;
https://steemit.com/tag/@ausbitbank/release-notes-for-steemit-version-0-1-170216

Thanks :)

It is good to have the documentation in the repo but it would be better organised for GitHub usage in the Wiki

The calculation of the score threshold is multi-variable and somewhat complex.

To help understanding, and do aid in picking a good minimum threshold, percentage increase, max votes per day and scoring metrics in general, it would be good to show this graphically in the Stats section.

It could be its own chart or added to the overview chart, perhaps toggle-able on / off (this is very possible with C3js)

It would be nice to add users who follow and mute you to this list of users we can check things against, like if the post is theirs or if they've voted on the post.

TODO: the metric key names

Golos is the Russian language version of Steem. It seems very possible to swap in golosjs-lib instead of using Steem.js as in the current implementation.

The voter bot does not exclude voting on the user's own posts.

Own-votes should either be excluded, or an option to exclude them be added

Add the simple modification to hyperlink anchors to request open in new window / tab

Settings for glossary library were accidentally removed, they should be restored from

glossary = require("glossary")

to

glossary = require("glossary")({minFreq: 3, collapse: true, blacklist: glossaryBlacklist})

Additionally, the minimum frequency should be settable by a configuration variable.

Some metrics were originally proposed for authors but were not implemented due to time constraints:

NOTE: these do not include the post being considered at present in their calculations

• author_last_post_payout: Last post payout, or est payout if payout has not happened yet.
• author_total_post_payout_avg: Average post payout for all posts by author (sampled on last Nposts)
• author_total_post_payout_median: Average post payout for all posts by author (sampled on last Nposts)
• author_last_post_time: Time since last post in minutes (or should it be days?)

To this, we can also add a few more:

• author_age: Number of days since account registration
• author_age_since_first_post: Number of days since authors first post

Question

Maybe it would be better to use months instead of days for these?

Loading animations should be used where content is to be loaded or long processes to be completed.

• Run bot
• Test algo
• Stats, when loading charts

This looks like a great collection of animations which work in pure CSS and under MIT license: https://github.com/ConnorAtherton/loaders.css

It should be possible to run Voter as a local installation.

The main areas of difference seem to be:

• Environment variables (process.env.SOMETHING)
• Redis database
• Assumption of greater volume of potential data storage

Currently the number of votes today increment on the score threshold is square scaled. See the graph from the algorithm docs:

From my own further testing and observations since release, I feel that the effect is much to little to begin with, and jumps too much at the end. While this was the intention, the effect is too great. What we have is a sudden cut off when the vote limit is almost reached, and then met. The desired effect is actually to gently reduce voting and still allow votes to trickle through regularly.

The current formula is the blue line, which is:

increase amount = (max score in window - (average + percentage increase)) * ((number of votes today / MAX_VOTES_IN_24_HOURS) ^ 2)

Simply remove raising the number of votes today to MAX_VOTES_IN_24_HOURS ratio to the power of 2, leave as is. This results in the orange line, which is:

increase amount = (max score in window - (average + percentage increase)) * (number of votes today / MAX_VOTES_IN_24_HOURS)

Note that this may be made irrelevant by issue #7

The email digest will only send the daily report when the next vote is cast on the next day.

It should send when the next bot run occurs, whether or not there is a vote to be cast.

Add a section in the voter to manually add links for the bot to vote on.

Manual curation would be useful for people who want to add links to vote on.

To add efficency users should be able to queue up several posts.

Could add metrics for the various payout settings posts can be set as?

• post_payout_is_powerup: Post payout is set to Power Up 100%
• post_payout_is_default: Post payout is set to Default (50% / 50%)
• post_payout_is_decline: Post payout is set to Decline payout

Add more NLP (Natural Language Processing) metrics for post content:

• number of profanities (with retext-profanities)
• number of readability warnings (with retext-readability)
• number of spelling mistakes (with retext-spell)
• number of incorrect English usages (with retext-usage)

Some of these may be useless but it is worth checking out what is available through retext since we're already using it.

We should also expand the use of sentiment anaylsis by using boolean values for negative, neutral and positive sentiment.

Only the keyword white / black lists are case insensitive.

All lists should be case insensitive.

Hi personz,

I have just discovered your project trough github tags, and I would like to tell you how awesome it is.

I love the beautiful d3 charts, amazing documentation and the licensing choice. I see and feel the love you've put into the project, and I'm looking forward to its success.

Please let me know when the new release is ready, and I will re-steem the announcement as well as share it with the whales.

Cheers,

@Furion

In the Edit Algo webpage, if you click on a metric in your algorithm it gets auto-filled into the edit fields at the top of the page, for ease of updating.

However if there is a lower range limit, it does not filled in if it is zero.

No action is taken if a previously non-empty white / black list is cleared and updated, the previous contents persist.

Reproduction steps:

1. Open Edit Algo page in web app
2. Clear any previously non-empty whitelist or blacklist
3. Click the Update button

Expected behavior:

White / blacklist which was cleared is now cleared after update

Actual behavior:

White / blacklist contents do not change

Because of the way that Heroku server idling works, the server needs to be completely created from a compiled binary every time it idles. It will idle within a short period of disuse.

The start up time for the server is very long and could probably be improved a lot with some simple adjustments. This would make the web app seem a lot more responsive and improve UX.

Add all nouns in post title to keywords as these are highly likely to be useful determinants of content.

Will need to use a better language parser.

The post fetch limit per request is 100 for steem.js but the user is able to set the limit higher than this. Because fetching is not paged / segmented, this causes the post fetch to fail.

Found by (at)beeskee

Could use the user's Steemit name more prominently. Such as "personz voter bot" instead of "Voter", etc.

Note, should only show this when there is an active session, to protected the privacy of the user.

I would feedback on the best way to do this one.

Currently, when counting the number of votes cast in the last 24 hours, the bot counts the total, which includes up votes and down votes.

I'm not sure what the best way of handling this would be, but I think an adjustment should be made.

Since the bot is concerned only with up voting, perhaps only the up votes should be counted.

However, since down voting reduces voting power, perhaps they should be counted. But maybe separately?

From a voting power conservation / wise usage perspective, maybe instead restricting voting (by threshold adjustment) using number of votes per day, it would be better to use the change in voting power directly?

This would enable a customisable "support" badge, or text phrase, etc., to be recognised for special attention by the bot. It would allow focused support clubs, guilds, or other organisations.

Because an image, or any other web resources, is just a string, setting it to an image, video or phrase is essentially the same thing. Note though that the image (or other resource) used would have to not only be the same image file but reside at the same hosting service.

This proposal requires the addition of a field to the internal algorithm object, something like support_string, and the ability to set it via the front end UI on the Edit Algo page.

The actual metric would be something like:

post_has_supporter: Post contains supporter string (badge URL or phrase)

NOTE: anyone would be able to add this to a post, so it may not be a sought-after feature.

Currently it's not possible to view what your current voting power is, or how many votes you've cast in the last 24 hours. It would be good to have this on the overview.

It would also be good to see how voting power changes as votes are cast, so in the bot run detailed stats.

In the Steemit.com UI front end there is the possibility to "resteem" a post by another user. This is like retweeting a tweet on Twitter, and their post appears in the feed of anyone who follows the resteem-er.

Can we get this information from steem.js? If so it could be another feed source.

In fact, we're only using the new posts feed in the current version. Maybe this could be expanded in general to allow users to pick a feed or multiple feeds?

Needs some careful consideration, we also do not want to overcomplicate things.

Due to the (at)cheetah bot account having such a high standing in the community, it would be an interest metric to see if cheetah has left a comment on the post.

cheetah can indicate:

• it found similar content on another website
• that the author is on the cheetah blacklist

Only calculate metrics which are relevant to the current algorithm.

Great idea, there's no need to look at areas which will never be processed.

Suggested by (at)beeskee

There is a certain use case for the bot in which is it only used to look through posts when the user is not able to engage with the social network, such as going on holiday, weekend or even at night.

To facilitate this slightly more positive usage, we should make a big obvious pause button on the dashboard to pause the bot. This will prevent it from casting votes but it will still process votes.

The algorithm testing is not very developed. Several features should be improved and added:

• Variable number of new posts to fetch
• Improve UI with stats as well as post list
• Improve test email / log