ftb-gamepedia / mediawiki-butt-ruby Goto Github PK

When editing, this is outputted a lot.

Cookie#domain returns dot-less domain name now. Use Cookie#dot_domain if you need "." at the beginning.

Right now, we don't have tests for any methods that do any HTTP stuff. This is kinda blargh. I would really enjoy having all of our methods have associated tests, so we don't have stupid errors like #34.

My only issue is I don't know how we should actually make these tests. We could use mocks, but I don't particularly enjoy using mocks for this kind of thing because you end up testing that you set up your mocks correctly, and not that you interact with the API correctly. Some other libraries, like the MediaWiki-Bot Perl library uses one of the many test Wikipedias to perform unit testing. Obviously, internet connection issues can result in errored builds, which is not so great.

what_links_here produces a NoMethodError for nilClass always. This is due to the list never being set.

When calling #get_other_langs_of_page. Tested with Witchery as the first arg.

  "query": {
    "pages": {
      "17975": {
        "pageid": 17975,
        "ns": 6,
        "title": "File:2014-08-06 22.15.48.png"
      },
      "18029": {
        "pageid": 18029,
        "ns": 6,
        "title": "File:2014-08-07 09.56.30.png"
      },
      "18030": {
        "pageid": 18030,
        "ns": 6,
        "title": "File:2014-08-07 09.15.23.png"
      },
      "18031": {
        "pageid": 18031,
        "ns": 6,
        "title": "File:2014-08-07 09.26.20.png"
      },
      "18032": {
        "pageid": 18032,
        "ns": 6,
        "title": "File:2014-08-07 09.16.07.png"
      },
      "113877": {
        "pageid": 113877,
        "ns": 6,
        "title": "File:2016-03-12 00.39.16.png"
      },
      "113879": {
        "pageid": 113879,
        "ns": 6,
        "title": "File:2016-03-12 00.47.17.png"
      },
      "113880": {
        "pageid": 113880,
        "ns": 6,
        "title": "File:2016-03-12 00.55.27.png"
      },
      "113883": {
        "pageid": 113883,
        "ns": 6,
        "title": "File:2016-03-12 01.06.24.png"
      },
      "113886": {
        "pageid": 113886,
        "ns": 6,
        "title": "File:2016-03-12 01.10.42.png"
      }
    }
  }
}

is an example response, but query_ary assumes the query result is simply an array of objects, not a hash of objects.

This problem is present for all duplicate file property query methods.

A simple method in Butt getting the wiki's url, like http://habbo.gamepedia.com or http://minecraft-br.gamepedia.com or whatever.

• #get_language for the default language of a wiki (would return en on our wiki, would return es on the Spanish Minecraft Wiki, etc).
• Namespace (and maybe other things that are needed?) language support, like butt.get_localized_namespace("Category", "pt-br") => "Categoria".

Errors should be raised in query when appropriate.

app/vendor/bundle/ruby/2.3.0/gems/mediawiki-butt-0.10.2/lib/mediawiki/query/lists/users.rb:135:in 'strptime': no implicit conversion of nil into String (TypeError)

When the user does not exist.

A bunch of the query methods (probably most of them actually) enter an each loop, and then return inside it. These loops don't actually loop, so there's no reason to use a loop.

???

The file repo methods return some strange things. Particularly, they all return Hashes instead of arrays. For example

{{"name"=>"hydra-commons",
  "url"=>"https://hydra-media.cursecdn.com/ftb.gamepedia.com"}=>nil,
 {"name"=>"local",
  "url"=>"https://hydra-media.cursecdn.com/ftb.gamepedia.com"}=>nil}

User agent headers should be customization in the initialization.

The continue parameter needs to be utilized, somehow. It should check if the continue parameter exists, and if it does, continue to perform queries.

The user agent needs to be set to "username/MediaWiki::Butt" on login. Perhaps set it during initialization as well, as "NotLoggedIn/MediaWiki::Butt" or something.

This is described in the [MediaWiki API documentation](https://www.mediawiki.org/wiki/API:Client code)

When the username passed does not exist:

NoMethodError: undefined method `[]' for nil:NilClass
from /usr/local/lib/ruby/gems/2.4.0/gems/mediawiki-butt-1.3.0/lib/mediawiki/query/lists/users.rb:79:in `get_contrib_count'

true/false is not adequate for the number of errors that can happen. We could return symbols, or just raise errors. I dunno.

We should not use global variables. They make code unpredictable and bad.

• Allcategories
  • get_all_categories, returns an array of all categories
• Allimages
  • get_all_images, returns an array of all images
• Allpages
  • get_all_pages_in_namespace, returns an array of all pages in a namespace
• Allusers
  • get_all_users, returns an array of all users
• Blocks
  • get_all_blocks, returns an array of all block IDs
• Embeddedin
  • get_all_transcluders, returns an array of all page titles that transclude a certain page
• Filearchive
  • get_all_deleted_files, returns an array of all deleted files
• Protectedtitles
  • get_protected_titles, returns an array of page titles that only people in the given group can create
• Usercontribs
  • get_all_contribs, returns a hash of arrays, formatted as:

{ 
  'revid' => {
    'title' => pagetitle,
    'summary' => comment,
    'size' => 69,
    'flags' => [
      'bot', 
      'minor', 
      'new'
    ], 
    'patrolled' => true
  }, 
}

• Watchlist
  • get_full_watchlist, returns an array of all the page titles in the user's watchlist

require_relative 'lib/mediawiki-butt'

butt = MediaWiki::Butt.new("http://ftb.gamepedia.com")
butt.login("ESAEBSAD", "-REDACTED-")

puts butt.get_contrib_count("xbony2", true)

results in...

/Library/Ruby/Gems/2.0.0/gems/string-utility-2.0.0/lib/utils.rb:3: warning: Refinements are experimental, and the behavior may change in future versions of Ruby!
/Users/xbony2/git/MediaWiki-Butt-Ruby/lib/mediawiki/query.rb:124:in `<module:Lists>': undefined method `using' for MediaWiki::Query::Lists:Module (NoMethodError)
    from /Users/xbony2/git/MediaWiki-Butt-Ruby/lib/mediawiki/query.rb:123:in `<module:Query>'
    from /Users/xbony2/git/MediaWiki-Butt-Ruby/lib/mediawiki/query.rb:5:in `<module:MediaWiki>'
    from /Users/xbony2/git/MediaWiki-Butt-Ruby/lib/mediawiki/query.rb:4:in `<top (required)>'
    from /Users/xbony2/git/MediaWiki-Butt-Ruby/lib/mediawiki/butt.rb:2:in `require_relative'
    from /Users/xbony2/git/MediaWiki-Butt-Ruby/lib/mediawiki/butt.rb:2:in `<top (required)>'
    from /Users/xbony2/git/MediaWiki-Butt-Ruby/lib/mediawiki-butt.rb:1:in `require_relative'
    from /Users/xbony2/git/MediaWiki-Butt-Ruby/lib/mediawiki-butt.rb:1:in `<top (required)>'
    from test.rb:1:in `require_relative'
    from test.rb:1:in `<main>'

Add support for the Purge API.

API Docs
Manual

A Ruby-version of the JS mw.util.rawurlencode method.

From calling $butt.edit:

/Library/Ruby/Gems/2.0.0/gems/mediawiki-butt-0.3.0/lib/mediawiki/edit.rb:28:in `edit': undefined method `[]' for nil:NilClass (NoMethodError)

The edit works though, except for that error there.

The move parameter redirect should actually be suppress_redirect, and its documentation should be updated. It does the opposite of what it says it does.

• :query_limit_default to 'max'
• :use_continuation to true

Change #edit to take a title, the text, and an options hash containing a minor: key, a bot: key, and a summary: key. Defaults are the same.

Change #create_page to take a title, the text, and an options hash containing a summary: key and a bot: key. Defaults are the same.

Change #move to take from, to, and an options hash containing a reason:, talk: and redirect: key. Defaults are the same.

• Categories
  • get_categories_in_page, returns array of categories
• Contributors
  • get_total_contributors, returns array of usernames
  • get_logged_in_contributors, returns array of usernames
• Duplicatefiles
  • get_duplicated_files_of, returns an array of file titles
  • get_all_duplicated_files, returns an array of file titles
• Extlinks
  • get_external_links_in_page, returns array of URLs
• Imageinfo
  • get_image_size, returns an int of bytes or nil if missing == ""
  • get_image_dimensions, returns a hash formatted as { 'width' => 100, 'height' => 100 }, or nil if missing == "". iiprop size and dimensions get both size and dimensions, so there is no need for multiple params = {} definitions
• Images
  • get_images_in_page, returns array of file names
• Info
  • get_protection_level, returns a hash formatted as { 'type' => 'edit', 'level' => 'sysop', 'expiry' => 'infinity' }
  • do_i_watch?, returns a boolean whether or not the currently logged in user watches the given page
  • get_number_of_watchers, returns an int of how many people watch the page
  • can_i_read?, returns a boolean whether or not the currently logged in user can read the page
  • get_display_title, returns a string of the display title
  • get_page_size, returns an int of how large the page is
  • get_number_of_views, returns an int of the number of views the page has Not possible with current API
  • is_page_redirect?, returns a boolean of whether or not the page is a redirect
  • is_page_new?, returns a boolean of whether or not the page only has one revision
• Iwlinks
  • get_interwiki_links_in_page, returns an array of all the interwiki links on the page
• Langlinks
  • get_other_langs_of_page, returns a hash formatted as { 'de' => 'Skyrim Wiki' } (Note that this does not work with the Translate extension, sorry FTB Wiki)
• Links
  • get_all_links_in_page, returns an array of all links on the page.
• Templates
  • get_templates_in_page, returns an array of templates that are transcluded in the page.

"The following response properties from action=login, deprecated in 1.27, are
now removed: lgtoken, cookieprefix, sessionid. Clients should handle cookies
to properly manage session state." - Early Changelog. And logging into https://www.mediawiki.org/wiki/ via irb gives a response of "No such action".

How are we going to deal with the support of extensions? I'd like to be able to stuff like this.

Eli edit:

Here are all the extensions we should support by the first release including this feature. It was a combined list of all the extensions on FTB, Wikipedia, and MediaWiki. I have removed all extensions (and skins) with no API. Many of the CurseGamepedia extensions have no docs, so I'm not sure if they have APIs or not. I'll ask Alexia when we get to them

• Data Miner
• All Sites
• Sync Service
• GlobalBlocking
• HydraCore
• MobileFrontend
• Gadgets
• Hydralytics
• SEO
• Site Globals
• Wiki Points
• Translate
• Redis Cache
• Admin Minder
• BunnyEars
• Claim Wiki
• CheckUser
• HydraAuth
• Curse Profile
• Subscription
• PageImages
• WikiLink
• RSS Pie
• Social
• Curse Twitter
• Achievements
• MsUpload
• Cite
• SpamBlacklist
• TitleBlacklist
• Abuse Filter
• OreDict
• Tilesheets
• TimedMediaHandler
• Diff
• SecurePoll
• CentralNotice
• Liquid Threads
• CommonsMetadata
• Citoid
• WikiLove
• TextExtracts
• BounceHandler
• FeaturedFeeds
• Thanks
• Disambiguator
• OAuth
• ParsoidBatchAPI
• OATHAuth
• SiteMatrix
• UrlShortener
• CentralAuth
• ApiFeatureUsage
• CodeReview
• Global Usage
• MassMessage
• Renameuser for CentralAuth
• GlobalRenameRequest
• GlobalRenameQueue
• LabeledSectionTransclusion
• TemplateData
• AntiSpoof for CentralAuth
• Flagged Revisions
• PageTriage
• Education Program
• SandboxLink
• GeoData
• PageAssessments
• RelatedArticles
• GettingStarted
• CirrusSearch

Here are a bunch of more list queries we need to support:

• iwbacklinks
• logevents Support all events, and all actions.
• langbacklinks
• imageusage
• exturlusage
• deletedrevs
• recentchanges
• querypage
• prefixsearch
• tags

Right now, the limiting stuff only supports hard-coded integers. We should also support setting it to max.

Summaries don't seem to work. Testing this with "@@replaceallincategory Extra Utilities; mod=EXU; mod=EU" with sample result.

The limit and type parameters in get_category_members should be switched around in placement.

For example

def edit(title, text, opts = { bot: true } )
  params = {
    title: title,
    text: text
  }  
  params[:bot] = '1' if opts[:bot]
  params[:summary] ||= opts[:summary]
  params[:minor] = '1' if opts[:minor]
  params
end  
# => :edit
edit('title', 'text')
# => {:title=>"title", :text=>"text", :bot=>"1", :summary=>nil}
edit('title', 'text', summary: 'summary')
# => {:title=>"title", :text=>"text", :summary=>"summary"}
edit('title', 'text', summary: 'summary', minor: true)
# => {:title=>"title", :text=>"text", :summary=>"summary", :minor=>"1"}

It happens because the hash argument is overwritten by the provided hash ({ summary: 'summary', minor: true }). Defaults should be specified like

def edit(title, text, opts = { } )
  opts[:bot] = opts[:bot].nil? ? true : opts[:bot]
  params = {
    title: title,
    text: text
  }  
  params[:bot] = '1' if opts[:bot]
  params[:summary] ||= opts[:summary]
  params[:minor] = '1' if opts[:minor]
  params
end  

instead, or something along those lines (it depends on the type of value).

I really gotta stop leaving these in here

I think persistent HTTP connection may lower the server burden.

This is part of #63, as it always returns XML regardless of whether JSON is set or not. See here.

This is a list of needed meta queries, organized by module.

• UserInfo
  • hasmsg, returns true if ['query']['userinfo']['messages'] == "", else false
  • changeablegroups, returns a hash of all the groups that the user can add/remove people from, formatted as { 'add' => [], 'remove' => [], 'addself' => [], 'removeself' => [] }
  • options, returns a hash containing all options and their values, basically identical to what the wiki provides.
  • realname, returns a string containing their real name, or nil if it is ""
  • email, returns a string containing their email address, or nil if it is ""
• FileRepoInfo
  • name|displayname, returns a hash of the names and their displaynames, example: { 'hydra-commons' => 'a shared repository', 'local' => nil }
  • name|rootUrl, returns a hash of the names and their rootUrls, example: { 'hydra-commons' => http://hydra-media.cursecdn.com/ftb.gamepedia.com', 'local' => 'http://hydra-media.cursecdn.com/ftb.gamepedia.com' }
  • name|local, returns a hash of the names and whether they are local or not, example: { 'hydra-commons' => false, 'local' => true }
  • name|url, returns a hash of the names and their URLs, example: { 'hydra-commons => 'http://hydra-media.cursecdn.com/ftb.gamepedia.com', 'local' => 'http://hydra-media.cursecdn.com/ftb.gamepedia.com' }`
  • name|thumbUrl, returns a hash of the names and their thumbnail URLs, example: { 'hydra-commons => 'http://hydra-media.cursecdn.com/ftb.gamepedia.com/thumb', 'local' => 'http://hydra-media.cursecdn.com/ftb.gamepedia.com/thumb' }`
  • name|favicon, returns a hash of the names and their favicon URLs, example: {hydra-commons=> 'http://hydra-media.cursecdn.com/commons.cursetech.com/6/64/Favicon.ico', 'local' => 'http://hydra-media.cursecdn.com/ftb.gamepedia.com/6/64/Favicon.ico' }
• SiteInfo
  • namespaces, returns a hash of the IDs and their names ('*'), example: { 'User talk' => 3 }
  • namespacealiases, returns a hash of the IDs and their names ('*'), like above.
  • specialpagealiases, returns a hash of the realname and their aliases
  • magicwords, returns an array of all magic word 'names'
  • usergroups, returns a hash with arrays of all usergroups and their rights, example: { '*' => ['can shit into xbony's mouth'] }
  • fileextensions, returns an array of all file extensions allowed on the wiki. upload should be slightly updated to require a check in this array, so that it's more safe.
  • restrictions, returns a hash basically identical to the one that the API provides.
  • skins, returns an array of all the '*' values
  • extensiontags, returns an array of all extension tags
  • functionhooks, returns an array of all function hooks
  • variables, returns an array of all variable IDs

@xbony2 I don't see allmessages being very useful. What do you think?

For example (FTB), get_contrib_count('ImmortalPharaoh7') returns the expected data, get_contrib_count('ImmortalPharaoh7 ') throws an error. get_userlists('editcount', 'ImmortalPharaoh7 ') returns the expected data. However, because get_contrib_count, and likely others, check for username equality its find call does not return anything. I'm not sure how we could handle this without sacrificing data accuracy.

We use some sort of older version. See also.

Here's a list of all the property methods that are on-hold until Gamepedia updates.

• Transcludedin
  • get_transcluders, returns array of page titles
• Deletedrevisions
  • get_deleted_revisions, still need to decide on what to return. Maybe split it up into multiple methods
• Fileusage
  • get_file_usage, returns array of page titles
• Redirects
  • get_redirects_to_page, returns an array of page titles that redirect to the page.

Reference: #7

They call old methods that don't exist. See RubyMine's complaints.

NoMethodError (broken loghash):

• get_block_log (block.rb)
• get_overall_log (log.rb)
• get_move_log (move.rb)
• get_move_redirect_log (move.rb)
• get_patrol_log (patrol.rb)
• get_modify_protection_log (protect.rb)
• get_protect_log (protect.rb)
• get_autopromotion_log (rights.rb)
• get_rights_log (rights.rb)

NoMethodError (missing method):

• get_reblock_log (block.rb)
• get_unblock_log (block.rb)
• get_upload_import_log (import.rb)
• get_merge_log (merge.rb)
• get_overall_log (log.rb)

ArgumentError (wrong number of arguments/duplicate method name):

• get_delete_log (delete.rb)
• get_deletion_restore_log (delete.rb)

All but the first parameters for post can probably be safely removed.

• autoparse: Absolutely useless. Why would anyone care to get an HTTPMessage. This is never used.
• header: This is never used. However, it might be useful in some cases. If it is decided to keep it, its default should be changed to an empty hash {}.
• override_assertion: This one, interestingly, is used. However it's not used where it says it's used in the documentation. In the docs, it says it is used in login. It's actually used in logout and I'm not really sure if that's totally necessary. Will need to test. If it isn't necessary there, it can be removed.

Right now, if there's an error, we simply return the error code as a String. This makes it difficult or weird to check for errors. For example, we want to be able to do this, as is idiomatic in Ruby:

begin
  hash = butt.some_function
  return hash['important_value']
rescue SomeError
  p 'Oh no we errored'
end

but right now, we have to do this:

hash = butt.some_function
if hash.is_a?(String)
  p 'Oh no we errored'
else
  return hash['important_value']
end

or this, for methods that return a String either way

string = butt.other_function
if /Error:/ =~ string
  p 'Oh no we errored'
else
  return string.reverse!
end

This is awful.