molybdenum-99 / mediawiktory Goto Github PK
View Code? Open in Web Editor NEWFull-featured MediaWiki client
License: MIT License
Full-featured MediaWiki client
License: MIT License
Current example:
response = client.
query.
generator(categorymembers: {title: 'Category:Countries_in_South_America', limit: 30}).
prop(revisions: {prop: :content}).
perform
Imaginary example:
response = client.
query.
generator(categorymembers: 'Category:Countries_in_South_America'). # singular values -- sets first key of hash
limit(30). # limit is common and propagated to all sub-modules having limit parameters
prop(revisions: :content). # again -- singular value by default is value for first key
perform
This approach may one time become fragile, but worth trying? Or some another approach?
Older, and, unfortunately, still popular versions of MediaWiki had pretty different API description page, like http://harrypotter.wikia.com/api.php
Should also be parseable (though, probably, would not be that easy).
Docs: https://www.mediawiki.org/wiki/API:Upload
Approach to utilize: https://github.com/lostisland/faraday#advanced-middleware-usage
MediaWiktory.wikipedia.query...
MediaWiktory.wikitravel('ru').query....
MediaWiktory.wikia('tardis', 'fr').query...
/w/api.php
(not always there, so, can't do it uncoditionally), like MediaWiktory::Client.new('https://en.wikipedia.org')
โ no need to path specification.As per this docs
Most reasonable default behavior, I assume, is receive content (follow redirects), but there should be an opt-out (for writing bots and other study software, targeting EXACTLY redirects logic).
...for everything.
YARD allows "fake" docs for auto-generated methods, so, we just need to make API docs for all modules clearly but concisely resembling original.
For example: https://en.wikipedia.org/w/api.php?action=help&modules=query%2Bgeosearch
gscoord
param is
Format: Latitude and longitude separated by pipe ("|").
Current usage:
api.query.geosearch('50.004444|36.231389')....
Should be:
api.query.geosearch(50.004444, 36.231389)....
Hi!
I want to extract a specific section of a page. How can I do that? Thx for your help!
It's not hard to generate some more code, and allow to have things like:
client.query.params
# => [:titles, :list, :meta, :generator....]
puts client.query.help
# Prints something like this page: https://en.wikipedia.org/w/api.php?action=help&modules=query
With multitude of actions and modules and modules and submodules... Can be REALLY helpful.
Currently fails trying to parse this empty response with undefined method 'pages' for nil:NilClass
.
Should return response with empty pages list.
Something like
c = MediaWiktory::Wikipedia::Api.new do |b|
# b is Faraday::Builder, allowing to change underlying web client, add middleware and so on
end
Cookies are important.
Need to find cookies details in MediaWiki docs, then implement! Use faraday-cookie_jar
gem or something.
Many popular sites have MediaWiki version lower than latest. Some of them even as old as 1.19 (of 2012!)
Differences between versions can both in available modules and core logic (like token management)
Ideal support for multiple version should consist of:
lib/generated/<version>/*.rb
(currently, code is generated from online help at mediawiki.org/w/api.php?action=help; for multiple versions it should be generated from MediaWiki sources, using tags?)Sub-ideal solution should at least tag modules with minimal version they require, and warn user on attempt to use those modules on wikis with lower version.
NB: In older versions of MediaWiki (even in slightly older ones) query continuation logic is much more complex and inconsistent than now.
Only a draft thoughts currently, but I can imagine, for ex., some special QueryClient
, which is more "semantic" than thin API wrapper with its query
, revisions
, prop
and so on:
c = QueryClient.new('https://en.wikipedia.org')
c.page('Argentina').wikitext
c.pages('Argentina', 'Bolivia')
c.category('Soth America').count
c.category('Soth America').pages
c.search(prefix: 'List of').count
# ...and so on
Same way, specialized EditClient
or AdminWatchClient
could be implemented.
From this page many params, like "assert", "origin", "maxage", and so on, should be passed at client creation (and optionally can be rewritten on each request).
Like this:
client = MediaWiktory::Client.
new('https://en.wikipedia.org', assert: 'user') # default value for each request
client.query.
...
perform(assert: 'bot') # rewriting for concrete action
For example, wikipedia.query.list(prefixsearch: {search: 'List of cities in'})
-- will return key query.prefixsearch
, and this key's content should be merged on subsequent calls
Hi,
Not an issue but a question. I'm doing my final project for a coding school and making an app with Rails backend/React frontend. I would like to allow a user to search a song title and return matches based on that search from Mediawiki API. I've been playing with some of the parameters but can't quite hone in on that specific search query. Would also be great if it returned an image with the link. Can anyone point me in the right direction? I've also been messing with the Mediawiki sandbox but also having trouble honing in with query search.
Thanks,
D
w = MediaWiktory::Client.new('https://en.wikipedia.org/w/api.php')
res = w.query.generator(prefixsearch: {search: 'Guardians', limit: 100}).prop(revisions: {prop: :content}, info: {prop: :url}).redirects(true).perform
res.continue!
# ArgumentError: Unknown key in continued page: revisions
# from /media/storage/work/gems/mediawiktory/lib/mediawiktory/page.rb:32:in `block in merge!'
If you look at raw structure: page 1, page -- you'll notice some strange things, like one page is described as pageid+title, and its content is on continuation page. So, current merging algorithm is not sufficient.
Lots of clients does not require all 100+ actions, so generator for them could probably be "optimized" with command line argument which actions to choose. Less code, less classes required, less memory consumption and so on.
Follow-up of #8
{"error"=>{"code"=>"gcmmissingparam", "info"=>"One of the parameters cmtitle, cmpageid is required", "*"=>"See https://en.wikipedia.org/w/api.php for API usage"}}
Response::Error
currenty preserves only "info"
fieldCurrenlty, we have:
query.generator(:search).namespace(0)
{..., namespace: 0, ...}
It is not that humane. Potentially, Api should introspect what namespaces current wiki have and provide some constants or shortcuts for more obvious work with them, not their ids.
Introspection could be done on code generation stage.
Mininum:
Better:
Of course, values in "better" section badges should became reasonable before publishing!
See also here: https://github.com/badges/shields for list of useful badges.
All data-modifying actions require some dance with "tokens". It should be done transparently for user.
See https://www.mediawiki.org/wiki/API:Tokens
Note, that token management have different behavior in:
See also #9 about version compatibility
NB: Current version of "manual" token management is:
token = api.query.meta(:tokens).response.dig('tokens', 'csrftoken')
response = api.edit.title('Wikipedia:Sandbox').text("Test '''me''', MediaWiktory!").token(token).response
It is, in fact, not that bad, and probably not worth over-automatization at all.
Action
should have method for parameter validation, and call it automatically before perform. Possible validations:
For now -- just a little better:
No idea about correct API for now, just to remeber how it is done:
For example, if you want page content, currently you'll do something like this: pages.first.revisions.first.content['*']
, which is kinda awful.
Don't invented, still, what could be done here.
Things should became simpler, but magic should be avoided.
...otherwise just response string should be returned.
If response contains errors
key, an error should be raised
If response contains warnings
-- do nothing by default, but allow easy access to the warnings (and maybe have mode for printing the warnings?)
From here:
If you want to protect against edit conflicts (which is wise), you also need to get the timestamp of the last revision
MediaWiktory could automate it (check if you still editing last revision), with possible opt-out.
Comprehensive and "real" usage examples should be written for several use cases:
Page is one of main concepts in MediaWiki content, and having it as a Hashie is a bit of under-implementation!
Follow-up of #8
Response#warnings?
(or Response#has_warnings?
)There should be pretty inspect both for Action and for Response: informative yet concise.
Examples of uncovered cases:
*
.api.query.meta(:tokens)
"Also would nice to:
Currenlty MediaWiktory will fail on attempt to get something from http://en.wikipedia.org
because of redirect to httpS. It should be handled transparently.
prop: :categories
, currently, developer is required to write prop: [:categories]
, which is dumb)A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.