jodoglevy / gargl Goto Github PK
View Code? Open in Web Editor NEWRecord web requests as they happen and turn them into reusable code in any programming language.
License: MIT License
Record web requests as they happen and turn them into reusable code in any programming language.
License: MIT License
Hi,
you have make an awesome tool to grab data on the web :) thanks a lot for that.
I have try to use your tool and export from the generator to a node.js module.
I have some issue :
npm install jquery
not work I use jsdom and code that i found here : var jsdom = require('jsdom');
var window = jsdom.jsdom().createWindow();
var $ = require('jquery')(window);
But with your example node.js file return :
{ readyState: 0,
getResponseHeader: [Function],
getAllResponseHeaders: [Function],
setRequestHeader: [Function],
overrideMimeType: [Function],
statusCode: [Function],
abort: [Function],
state: [Function],
always: [Function],
then: [Function],
promise: [Function],
pipe: [Function],
done: [Function],
fail: [Function],
progress: [Function],
complete: [Function],
success: [Function],
error: [Function],
status: 0,
statusText: 'No Transport' }
There is a way to use this or is not the reponses i should see ?
Thanks.
Hi,
Thank you for your awesome tool !
I tried it with a search engine and added a response field like in your demonstration video. The response I have when I test it isn't filtered with the css selector, I have the entire HTML file.
Did I not understand something ?
Regards,
Guduche
joe,
Great work on this. I was wondering if you could post a tutorial on how to use node.js with the javascript output?
Hi,
I am trying to use Gargl and pygargl to download a xml file from website. After get into the website, I need to click some selections to confirm which items to be downloaded, then I click the button of "export data", it will download the xml file from the website to my computer. And there is some information "content-disposition:attachment; filename=Export090916.xml" from the response headers.But I cannot find anything in response body. Could you please give me some ideas about how to download the xml file using Gargl and pygargl?
Thanks!
Hi Joe,
when I edited the sample.gtf from the repository, then
A subsequent import into Gargl Chrome extension does not parse the file.
I am attaching the test data.
The JSONs are valid, syntactically correct.
Kind regards,
Karol
Hi,
i try to test gargl i get this error :
Exception in thread "main" java.lang.UnsupportedClassVersionError: Main/Gargl : Unsupported major.minor version 51.0
at java.lang.ClassLoader.defineClass1(Native Method)
at java.lang.ClassLoader.defineClassCond(ClassLoader.java:637)
at java.lang.ClassLoader.defineClass(ClassLoader.java:621)
at java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
can your please tell me what is the problem ?
i run this command java -jar GarglJavaGenerator.jar -outdir gen -input testyahoo.gtf -lang javascript
Hi,
I'm using Gargl and it's awesome !
However I have a question : is there any way to get attribute value of an element ?
For exemple get the "href" value ?
Thanks !
Hello,
I have been wanting to try out this framework but I write in C#. After dabbling around a bit I decided to add a C# generator. See Yahoo template output here The problem is, I am having trouble with setting request headers. The way I am doing it now works, but it requires a separate include file (straight from StackOverflow).
The problem is that the simple way to add headers would be req.Headers.Add("User-Agent", "TestAgent");
but User-Agent, among many other headers, are restricted and throw an exception when modified incorrectly. They must be set by req.UserAgent = "TestAgent";
. You can see how this does not match the traditional style of Gargl.
So my question is... does anyone have a better way to do this? I was thinking about adding the include to the generated file, but it may get bulky if using multiple templates. Is the best way to go just to add this logic into the Generator itself (and spit out valid values)?
When we use the Gargl chrome extension, we found that chrome does not record all of the information of certain responses.
For example, after solving Google's ReCaptcha correctly, chrome does not give the correct information of the response. It hides the content of response, and just gives us " )]}'["uvresp",,,,1] ". But we cannot get the entire content of the response but it does show properly in firefox.
And for the previous issue when there is "content-disposition:attachment; filename=Export090916.xml" in request headers, I can not find the response of xml file in Chrome it does show in Firefox.
These are not shortfalls of the Gargl extension but of the Chrome browser itself.
I'd like to be able to get a Response Field from an input element, rather than from the innerHTML of a tag.
Hi Joe,
I am a big fan of Gargl. Now I have a problem, when I click on some certain buttons, I can get post requests of XHR from network tab, but I cannot get these records from Gargl. The website uses a lot of ajax. Could you please release a new version or give me some idea about how to make a complete record on all these XHR?
Thank you so much!
Only generated JavaScript packages and the Gargl Python project currently support this.
For reference: #25 (comment)
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.