I have had this idea for a while and figured I would act on it but still haven't due to barrier to entry of learning ruby. Anyway, here it goes:

The Clipper card is a multi-transit system paycard for the SF Bay Area. The system has an autoload feature but it only works when the balance goes below $10.

The basic idea is to create an agent which:

• Monitors balance for threshold
• When below threshold, auto-deposit pre-defined amount
• Bonus: Could track daily transit usage
• Bonus: There is a delay of up to 5 days when depositing online, Huginn could be smart and pre-determine when the threshold will be reached and deposit 5 days early

This is a little short sighted as it only applies to the Bay Area but it probably could serve a nice boilerplate for other transit systems.

As far as implementation goes, some options are scripted purely in Ruby (depends how much AJAX happens on page), scripted via PhantomJS, or we create an API wrapper for Clipper and hit that.

I think it would be super swell to abstract out payload manipulation.

The EmailAgent already does this manipulation in one direction: it allows a user to take a number of fields from a payload and combine them into a single field within the message field "template".

It would also be useful to go in the other direction: take a single field, and, given a template, extract a number of fields. The templates could look exactly the same as those given to the message field. Named captures in regexps would be more powerful, but then there'd be two problems.

I think I'm suggesting a FieldCombineAgent and a FieldExtractAgent. The FieldCombineAgent would do what the EmailAgent already does (that is, fill in a template given a payload), but the FieldExtractAgent would be new functionality. Basically, the FieldExtractAgent would allow an event with a low-complexity payload to satisfy an event with a high-complexity input (of which zero currently exist), and the FieldCombineAgent would exist to aid in conceptual symmetry and event composability.

For use cases, I'm thinking of things like sending a structured SMS to create an issue in an issue tracker or sending a structured tweet or email or IRC message to kick off a job/push/test suite.

It'd be nice to have a Gmail agent. The most useful event would be receiving an email, but conceivably, sending an email could also be useful. The payload would include the label hierarchy and star status along with all the other typical email fields (from, to, cc, bcc, subject, body, attachments[?]). These could then be passed along to the TriggerAgent to do all sorts of useful things.

Paired with Google Voice, this could also allow for hacky SMS integration (which I'd really like).

Following steps from Quick start, while copying .env to .env.example; socket is uncommented and set to /tmp/mysql.sock.

MySQL may not connect through that socket always.

It could alternately exist @ ["/var/run/mysqld/mysqld.sock", "/opt/local/var/run/mysql5/mysqld.sock", "/tmp/mysql.sock"]

Hence, comment out socket under .env.example

An agent that can take geolocations and trigger events when the location is in a region.

This would be useful to trigger things when you go to work, shops, etc.

Hi guys,

I been trying to get the DigestAgent working, but some reason ActionMailer is not sending any emails

I have the following in my development.rb

DOMAIN = "localhost:3000"

config.action_mailer.default_url_options = { :host => DOMAIN }
config.action_mailer.asset_host = DOMAIN
config.action_mailer.perform_deliveries = true # Enable when testing!
config.action_mailer.raise_delivery_errors = true
config.action_mailer.delivery_method = :smtp
config.action_mailer.smtp_settings = {
address: "smtp.allanmacgregor.com",
port: 465,
domain: "allanmacgregor.com",
authentication: "login",
enable_starttls_auto: true,
user_name: "[email protected]",
password: "xxxx"
}
end

I'm also not seeing any errors or the email message

Allow the Trigger Agent to re-emit the full original Event, and/or perform formatting on the output.

Trying to deploy a development version to a cloud server (Linux node_name 2.6.32-279.19.1.el6.x86_64 #1 SMP Wed Dec 19 07:05:20 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux), I got a strange error - after about five or six calls of propogate! from the scheduler, I start getting these errors:

16:04:06 schedule.1 | { 47046200 rufus-scheduler intercepted an error:
16:04:06 schedule.1 | 47046200 job:
16:04:06 schedule.1 | 47046200 Rufus::Scheduler::EveryJob "3s" {}
16:04:06 schedule.1 | 47046200 error:
16:04:06 schedule.1 | 47046200 47046200
16:04:06 schedule.1 | 47046200 ActiveRecord::ConnectionTimeoutError
16:04:06 schedule.1 | 47046200 could not obtain a database connection within 5 seconds (waited 5.000160871 seconds). The max pool size is currently 5; consider increasing it.

...and so on on every second tick. This was with my own modified version of Huginn, but just ticking away in the background without any browser access. I also had the dj: line in Procfile disabled to narrow down the problem. However, a vanilla install of the latest version from github (under the same account but with a slightly different set of Gems) worked without error.

After much hacking about, I discovered that this change worked for me:

scheduler.every '3s' do # 3s for testing - same problem occurs with 1m
  ActiveRecord::Base.connection_pool.with_connection do
    propogate!(mutex)
  end
end

I don't have any confidence that this is the proper way to fix the problem, and I may be the only person experiencing it, but I wanted to leave a trace for later Googling. The fresh vanilla version uses rufus-scheduler 2.0.22 but my modified version uses 3.0.0 - this seems to be the deciding factor, according to my tests. Furthermore, this change is also needed:

scheduler = Rufus::Scheduler.new

It seems backward-compatible with v2.0.22... I've also reproduced the fault and the fix on a completely different box.

Oops... I didn't realize a pull request would also create an issue. This is a no-op.

Some of us are going to start hanging out in #huginn on freenode. Please join us!

Been playing around with huginn and I seem to have encountered an issue filtering the twitter stream with a hashtag. If I filter #test , it shows up in the twitter_stream.rb Terminal window, but doesn't get noticed as an event. If I filter test , it shows up in the twitter_stream.rb Terminal and comes across as an event. Does anyone else see this behavior? (or I could just be doing something wrong)

Is there any centralized place to share agents? Even just a wiki page? It seems that with so many forks there are probably many useful or interesting agents out there that are just not findable yet. Or is the intention that once an agent is deemed ready/complete it will be merged into this origin repo?

Right now, huginn is dependent on kgio, which does not have a Windows version. I don't know what other dependency issues could exist on Windows, but this was the first one I ran into that I couldn't solve myself.

Let's start this architecture discussion.

I think Huginn Agents should be distributed as ruby gems, inheriting from a huginn-agent gem containing the base models and agent behaviors. Agent gems should also be allowed to distributed partials and controllers for any custom agent behavior; as the agent model grows it will make sense to distribute dashboards and custom UIs for some types of agents.

What is the best way to organize a Rails system like this? Engines?

I didn't expect to spend so much time trying to get SMTP working. Would someone be so kind as to share their .env that works?

I'd love to see a GMail example and a local/ISP example. If you don't want to share your personal info, just replace the specific address and password as if it is for [email protected] or [email protected] with a password of "abc123".

I may just have a specific issue with my local setup, but I've made some telnet tests that seem to indicate that I can talk to the SMTP servers I'm specifying in my .env.

I know the WebsiteAgent exists, but it'd be nice to have a real RSS agent as well. RSS/Atom provide data with better fields, and people expect feeds to be crawled (not always the case with straight-up web content [does the WebsiteAgent obey robots.txt?]). This would also remove the need for the CSS selector business (while it's powerful, it's not exactly convenient).

It would be great to have a RESTful agent capable of long polling.
This would, for example, allow me to have huginn watch lights turn on and off in my house using the openremote rest API
http://www.openremote.org/display/docs/Controller+2.0+HTTP-REST-JSONP
I'm sure there would be many more applications.
I'm new to huginn and to Ruby but if someone can point me to parts of the code I should look at, I'm willing to give it a try.

http://cytoscape.github.io/cytoscape.js/

This is a pretty neat project. I'd like to see easy deployment to a heroku dyno. Obviously 'contributions welcome' applies to this feature request, but I figured I'd log this here for tracking purposes.

Just wanted to say, this is very, very cool.

I've recently had my own idea of making something that would push notifications to OSX Notification Center depending on event triggers. But it seems like huginn takes care of all of the hard stuff!

I think this would be a very useful feature. I know this is OSX specific, but an application for Windows/Linux can easily be made to receive notifications that would work well with the respective platforms. I'm thinking simple pop-ups, etc.

Thoughts?

HI!

I've setup the agent with wunderground, put in my wunderground API key as option and my town id as another option, but when I hit "run" the following message appears:

Exception during check: undefined method []\' for nil:NilClass -- [\"/srv/huginn/app/models/agents/weather_agent.rb:69:incheck'", "/srv/huginn/app/models/agent.rb:268:in `async_check\

Am I missing something?

The options are configured as follows:

{
"location": "10640",
"api_key": "0xxxxxxxxxxxxxx6"
}

Location should be Frankfurt, Germany

I have noticed that there are duplicate tweets in twitter stream the payload saved with the same tweet tow times.

Thanks

I have a /device/:device_name(/:event_name) route that handles input from devices that aren't targeted at a specific agent id, unlike webhooks_controller.rb. I want to search through to find an Agent that can handle the device and event, so I have this in my controller:

    Agent.all.each do |agent|
      # Have an agent waiting for this device
      if agent.options[:device] == params[:device_name]
        if agent.options[:event].nil? || agent.options[:event] == params[:event_name]
          # Do my stuff

Obviously, with lots of agents this may soon get rather slow. As far as I understand, however, I can try this:

    Agent.where("options LIKE ':device: '#{params[:device_name]}'").where("options LIKE ':event: '#{params[:event_name]}'").each do |agent|
      # Do my stuff

However, that is rather ugly, and looking at the database sometimes the values are in quotes and sometimes they aren't. So, my questions are:

1. Is there a better way of doing my code?
2. Is there something that can be done to serialize_and_symbolize.rb to clean up the rather messy query?

Running on Ruby 1.9.3 and WEbrick and 32-bit Windows XP, I don't get messages being passed down the chain, the "pending messages" in the top-right corner fills up with jobs, and the "Run event propagation" button has no obvious effect. Furthmore, accessing /delayed_job results in this error page

Routing Error

No route matches [GET] "/delayed_job"

Try running rake routes for more information on available routes.

I'm at a loss as to what could be the problem. Everything else seems to work fine.

I got this error when trying to initialize the db (rake db:create). Researching for the error message showed that this is a very common error with many different solving approaches. I tried nearly all hints (forcing rake 0.8.7, adding differen require-command) but none of them changed something about the error message. As I don't know ruby enough I hope someone could help me out. The bundle-command finished with the message "Your bundle is complete!".
Here is the Backtrace of the error (has been cloned to /srv/www/huginn):

/srv/www/huginn # rake db:create --trace
(in /srv/www/huginn)
rake aborted!
uninitialized constant Rake::DSL
/var/lib/gems/1.8/gems/dotenv-0.6.0/lib/dotenv/railtie.rb:5
/var/lib/gems/1.8/gems/dotenv-rails-0.6.0/lib/dotenv-rails.rb:1
/var/lib/gems/1.8/gems/bundler-1.3.5/lib/bundler/runtime.rb:72:in require' /var/lib/gems/1.8/gems/bundler-1.3.5/lib/bundler/runtime.rb:72:inrequire'
/var/lib/gems/1.8/gems/bundler-1.3.5/lib/bundler/runtime.rb:70:in each' /var/lib/gems/1.8/gems/bundler-1.3.5/lib/bundler/runtime.rb:70:inrequire'
/var/lib/gems/1.8/gems/bundler-1.3.5/lib/bundler/runtime.rb:59:in each' /var/lib/gems/1.8/gems/bundler-1.3.5/lib/bundler/runtime.rb:59:inrequire'
/var/lib/gems/1.8/gems/bundler-1.3.5/lib/bundler.rb:132:in require' /srv/www/huginn/config/application.rb:7 /srv/www/huginn/Rakefile:7:inrequire'
/srv/www/huginn/Rakefile:7
/usr/lib/ruby/1.8/rake.rb:2383:in load' /usr/lib/ruby/1.8/rake.rb:2383:inraw_load_rakefile'
/usr/lib/ruby/1.8/rake.rb:2017:in load_rakefile' /usr/lib/ruby/1.8/rake.rb:2068:instandard_exception_handling'
/usr/lib/ruby/1.8/rake.rb:2016:in load_rakefile' /usr/lib/ruby/1.8/rake.rb:2000:inrun'
/usr/lib/ruby/1.8/rake.rb:2068:in standard_exception_handling' /usr/lib/ruby/1.8/rake.rb:1998:inrun'
/usr/bin/rake:28

It seems like it'd be worth discussing ways to allow for easy API key management. One thing to keep in mind is the distinction between app keys & user keys: correct me if I'm wrong, but I think a single Huginn deployment can be configured with a single Twitter consumer key & consumer secret, but oauth creds for each user.

My thoughts:

Quick & dirty: App keys go in .env, user keys get their own fields on the Account screen.

Better: Same as above, but separate API keys view and menu item to the Account dropdown.

Betterer: Same as above, but user keys can be accessed/modified from agent creation page and get cached for future instances of that agent. If the user is the admin, keys for the app will be available there, too.

Hi,
it seems in the dependency-configuration is fastercsv missing.

matze@pucky ~/test/huginn $ rake secret
The method 'YAML.enable_arbitrary_object_deserialization!' is deprecated and will be removed in the next release of SafeYAML -- set the SafeYAML::OPTIONS[:default_mode] to either :safe or :unsafe.
rake aborted!
no such file to load -- fastercsv
/home/matze/.gem/ruby/1.8/gems/activesupport-3.2.12/lib/active_support/dependencies.rb:251:in require' /home/matze/.gem/ruby/1.8/gems/activesupport-3.2.12/lib/active_support/dependencies.rb:251:inrequire'
/home/matze/.gem/ruby/1.8/gems/activesupport-3.2.12/lib/active_support/dependencies.rb:236:in load_dependency' /home/matze/.gem/ruby/1.8/gems/activesupport-3.2.12/lib/active_support/dependencies.rb:251:inrequire'
/home/matze/.gem/ruby/1.8/gems/rails_admin-0.4.5/lib/rails_admin/support/csv_converter.rb:2
/home/matze/.gem/ruby/1.8/gems/activesupport-3.2.12/lib/active_support/dependencies.rb:251:in require' /home/matze/.gem/ruby/1.8/gems/activesupport-3.2.12/lib/active_support/dependencies.rb:251:inrequire'
/home/matze/.gem/ruby/1.8/gems/activesupport-3.2.12/lib/active_support/dependencies.rb:236:in load_dependency' /home/matze/.gem/ruby/1.8/gems/activesupport-3.2.12/lib/active_support/dependencies.rb:251:inrequire'
/home/matze/.gem/ruby/1.8/gems/rails_admin-0.4.5/lib/rails_admin.rb:8
/home/matze/test/huginn/config/application.rb:7
/home/matze/test/huginn/Rakefile:7
(See full trace by running task with --trace)

I had to install it manually later.

I really love this but I worry that some of the design decisions aren't very flexible and I feel like I'm about to hack the shit out of it. The first thing is the hard coding of the configurations, and then the coupling to mysql via database.yml but more importantly the migrations. I'd like to use postgres. Then I'm probably going to work on trying to make it cheaper to run on heroku. Am I wasting my time if I start shooting pull-requests your way?

Is it currently possible to schedule an agent from one time until another, ie. "run daily from 12:30 to 12:45 every minute"?

If so, how?
If not, does anyone know of a gem that could provide that functionality?

This could be a nice-to-have or could be better handled by requiring tests for agents, but it seems like it'd be handy if the agents that get initially installed:

• included all current agents
• they were all setup in such a way that if they were all run immediately upon logging in, they'd produce immediate results that can be used for new users to familiarize themselves with Huginn

Seems the twitter stream is still using the basic authentication that is now officially deprecated. Oauth is now the only supported method

I'm a big fan of this idea, and I'd like to develop a few agents to help me move off IFTTT and Pipes. I'd also like to avoid duplicating the effort of others if there's already work being done on agents that interest me.

Is your intention to use this issue tracker as a place to coordinate agent development? i.e. can I create issues for the agents I'm interested in and express my interest in developing them?

shell$ foreman start

23:08:11 web.1 | started with pid 3090
23:08:11 schedule.1 | started with pid 3092
23:08:11 twitter.1 | started with pid 3094
23:08:11 dj.1 | started with pid 3096
23:08:16 dj.1 | /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/shared_helpers.rb:2:in require': no such file to load -- rubygems (LoadError) 23:08:16 dj.1 | from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/shared_helpers.rb:2 23:08:16 dj.1 | from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/setup.rb:1:inrequire'
23:08:16 dj.1 | exited with code 1
23:08:16 system | sending SIGTERM to all processes
23:08:16 | from /var/lib/gems/1.9.1/gems/bundler-1.3.5/lib/bundler/setup.rb:1
23:08:16 web.1 | terminated by SIGTERM
23:08:16 schedule.1 | terminated by SIGTERM
23:08:16 twitter.1 | terminated by SIGTERM

Any suggestions?

It would be helpful to get Huggin setup just by doing:

vagrant up

The procedure to install, https://github.com/cantino/huginn/wiki/Novice-setup-guide, can be done by provisioning script. Simple bash script should be sufficient.

In my usage of Huginn recently, it has seemed like Events may need a bit more structure than we're currently giving them. Specifically, right now it's difficult to

• know how to display an arbitrary event in the UI
• know how to format an arbitrary event for sending over email, SMS, Twitter, etc.
• know where an event came from

Do you guys have any insights into how events could be structured to provide a stronger schema while still keeping the flexibility that we have now?

The concept of "interfaces" (from Java, Go, etc.) seems like a useful starting point. Perhaps sending Agents should declare one or more interfaces that their events conform to, and then receiving Agents can declare what interfaces they understand. E.g., the summary interface will always have a human-readable summary field, the alert interface will always have cause, date, and source fields, etc. Microformats could also be used.

Thoughts?

Hi Andrew,
Where do you see this project in comparison to ScraperWiki? ScraperWiki is also open-source, but it also allow developers to write their agent equivalents in multiple programming languages and also share them between users. I am interested to hear your thoughts if they overlap or serving different purposes ( given that ScraperWiki is in 4 years of development, it might be good to hear your thoughts about your project's philosophy)

I added support for OpenShift on the "openshift" branch here:

http://github.com/ghelleks/huginn

It's kind of intrusive, so not ready for a formal pull request yet. In the meantime, using the branch is documented here:

https://github.com/cantino/huginn/wiki/Deployment-on-OpenShift

Next Steps

I want to explore the next steps for Huginn and to talk about something much more ambitious than the current implementation.

Huginn, the Protocol

Goals:

• Huginn Agents shouldn't have to be implemented in Ruby inside of a Rails app. Agents should be implementable in any language.
• You should be able to interact with both local and remote Agents in the same way, with the same API.
• The system should be fast enough to tackle a wide variety of problems.

Proposal:

• Agents can be written in any language on any platform that can open persistent connections.
• Agents create and consume JSON Event objects.
• To consume events from an Agent you connect to that Agent's IP address and port and subscribe to a standing query for matching events. The default query is something like subscribe to *, returning all generated events, but specific Agents can allow for more complex predicates, such as subscribe where zipcode = 12345 (syntax TBD).
  • This could easily be wrapped in WebSockets.
  • This predicate language should be standardized and, importantly, very simple to implement.
  • It's possible that Agents should create events in a rolling event log and allow for short-term event replay from a specific timestamp so that a re-connecting Agent can recover any lost events.
• Agent configuration is not part of the spec. Some Agents may be configured via the command line, others via a web GUI, still others can be paid remote services.
  • The reference implementation will make it easy to customize and deploy Agents on your own system with configuration via a web GUI similar to the current system.
  • The reference implementation may allow Agents to be sandboxed (a la docker) so that you can install semi-trusted 3rd-party code.
• Agents should be runnable locally or remotely. The goal is to allow you to maintain control of your data by running everything locally, or to subscribe to remote Agents and participate in a larger data ecosystem. Some questions:
  • How should local Agents be packaged and distributed?
  • What would a marketplace look like where you could easily subscribe to, or download and run, Agents for any purpose?

What do you think?

Get Involved!

Want to help create the future of Huginn? Get involved! You can comment and contribute here, send pull requests, follow @tectonic on Twitter, or contact me directly on my personal site to discuss further involvement.

When you setup a new agent to receive events from an existing one that already has events, the first time event propagation is run, ALL of those events are sent through receive. That obviously causes some trouble if the sender agent has accumulated a lot of events already.

That doesn't make a whole lot of sense to me as default behavior. It would make much more sense to only send future events through.

I could tackle changing this, and I'm looking through the code for where that's being done, but I'm not that familiar with the code base and am having trouble finding it. A pointer to where that's happening would be appreciated.

Right now contributing to Huginn while keeping your personal changes private is somewhat difficult.

You can fork Huginn and then make a private remote of the fork, but pushing changes from the private duplicate is difficult without leaking your personal configuration, unless you're very careful about how you structure your branches.

Should we move all configuration into a single .gitignored YAML file? Or would it be better if this was a Rails engine?

What do you think about removing the deployment-specific files and configuration from master?

I am not a RoR developer. I am a technically minded growth hacker and want to use Huginn. Let me say first, thanks for making this interesting program and to all the people helping.

I will say that the newbie guide and the quick start guide both have holes in them. However I have a Vagrant box with Ubuntu Lucid 32 up and running with Huginn working. However I get the below error when trying to click on the Admin page. Everything else seems to be working and Agents are generating events. I would be happy to contribute my Vagrant box once everything is working so it's easy for anyone to play with Huginn.

RuntimeError in RailsAdmin::MainController#dashboard

Task Argument Error

Rails.root: /vagrant
Application Trace | Framework Trace | Full Trace

lib/capistrano/sync.rb:35:in block (2 levels) in <top (required)>' lib/capistrano/sync.rb:28:inblock in <top (required)>'
lib/capistrano/sync.rb:27:in `<top (required)>'

I have some code from a similar project that can be adapted to create a Reddit Agent

It'd be nice if the contributor setup/workflow accommodated people who have free Github accounts, especially for hit noobs (ie. me).

Hello... First time posting on Github, pretty exciting. I'm not a programmer, just a hobbyist, so I'm not sure I can do this myself but I'd like to modify or create a secondary Weather Agent where the user can get weather info for non-USA locations.

This seems easily possible in the Wunderground API, but I can't quite find how Huginn creates the request URL. I'm looking at app/models/agents/weather_agent.rb and I see references to ZIP but I'm not sure how to go about changing it to another method of specifying location (I think the Country / City method would be good).

I'll keep looking around but if you can quickly give me some clues maybe I can get it done!

Thanks.