lhcardoso / crawwwler Goto Github PK
View Code? Open in Web Editor NEWAutomatically exported from code.google.com/p/crawwwler
Automatically exported from code.google.com/p/crawwwler
In order to run crawwwler in a clustered environment, we need network
wrapper classes associated with each and every plugin. This one for
CCrawlerPlugin can be the protoype.
To illustrate: If e.g we want to run the crawler plugin on one server and
the url extractor plugin on another, then we would need the following:
Server 1:
CUrlExtractor
CCrawlerPluginNetworkAdapter (verbose name for explanation)
Server 2:
CCrawlerPlugin
The CUrlExtractor would subscribe to the CCrawlerPluginNetworkAdapter which
would act as a dummy. It would pass data between the CUrlExtractor and the
CCrawlerPlugin on server 2. Nothing more.
It is possible that this functionality could be built in to each of the
plugins and toggled by means of a switch. I'm really not too sure how this
aspect will work yet, so I'm open for discussion.
Original issue reported on code.google.com by [email protected]
on 3 Feb 2008 at 1:30
A web front-end to search the database. Probably Ruby on Rails
Original issue reported on code.google.com by [email protected]
on 6 Feb 2008 at 11:52
Need to design and implement the main back end data store now that a basic
crawling prototype has been implemented
Original issue reported on code.google.com by [email protected]
on 3 Feb 2008 at 1:15
Need to implement a plugin that inherits from CPlugin that will connect to
the sql data store.
Initially this plugin needs only to subscribe to the CCrawlerPlugin and
save data about the server along with the raw html data to the data store
Original issue reported on code.google.com by [email protected]
on 3 Feb 2008 at 1:21
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.