fedmsg needs a common logging system that is independent (for console scripts) but which can work well along side other logging mechanisms (like inside other FI apps).

There can be only one. We need to use php's memory allocation features...

http://stackoverflow.com/questions/7952209/zmq-getsocket-method

for @nirik

It makes more sense.

What happens when you open up telnet on it and just start barfing garbage on it.

What happens when you spam it? How many messages/s and bytes/s can it handle?

/etc/fedmsg.ini currently only allows a single sending host on any given machine.

Instead the config file should allow a list of sending services on the machine, each with a different publish_endpoint.

fedmsg.text has a _ function that isn't used or hooked up.

There's a lot of stuff going on there, which should be documented.

in fedmsg/crypto.py.

Don't forget community01.dev!

We should probably write a fedmsg script that will output iptables commands for us, so when we change the config we can change iptables seamlessly.

Important for posterity.

This will require editing extras/mediawiki/fedmsg-emit.php to get the revision information out in a reliable way.

When ctrl-cing fedmsg-relay with a FedoraBadgesConsumer running, it doesn't shutdown cleanly (or at all)

start it and stop it a few times and you'll see a traceback fly by.

The purpose it was meant to serve no longer makes sense since every endpoint isn't coupled with a fedmsg-hub. I once thought they would be 1-to-1.

fedmsg-status should either be deleted or the config convention should be changed to denote which endpoints should have heartbeats (run under fedmsg-hub) and which should not (standalone send_message).

/usr/lib/python2.7/site-packages/fedmsg/encoding.py

I got this traceback when developing supybot-fedmsg

Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/supybot/log.py", line 351, in m
    return f(self, *args, **kwargs)
  File "/usr/lib/python2.7/site-packages/supybot/irclib.py", line 132, in __call__
    method(irc, msg)
  File "/usr/lib/python2.7/site-packages/supybot/plugins/MeetBot/plugin.py", line 116, in doPrivmsg
    M.addline(nick, payload)
  File "/usr/lib/python2.7/site-packages/supybot/plugins/MeetBot/meeting.py", line 494, in addline
    linenum=linenum, time_=time_)
  File "/usr/lib/python2.7/site-packages/supybot/plugins/Fedmsg/plugin.py", line 83, in wrapper
    msg=dict(args=args, kw=kw),
  File "/usr/lib/python2.7/site-packages/fedmsg/__init__.py", line 60, in _wrapper
    return func(*args, **kw)
  File "/usr/lib/python2.7/site-packages/fedmsg/__init__.py", line 89, in publish
    return __local.__context.publish(topic, msg, **kw)
  File "/usr/lib/python2.7/site-packages/fedmsg/core.py", line 190, in publish
    self.publisher.send_multipart([topic, fedmsg.encoding.dumps(msg)])
  File "/usr/lib64/python2.7/json/encoder.py", line 203, in encode
    chunks = list(chunks)
  File "/usr/lib64/python2.7/json/encoder.py", line 428, in _iterencode
    for chunk in _iterencode_dict(o, _current_indent_level):
  File "/usr/lib64/python2.7/json/encoder.py", line 402, in _iterencode_dict
    for chunk in chunks:
  File "/usr/lib64/python2.7/json/encoder.py", line 402, in _iterencode_dict
    for chunk in chunks:
  File "/usr/lib64/python2.7/json/encoder.py", line 402, in _iterencode_dict
    for chunk in chunks:
  File "/usr/lib64/python2.7/json/encoder.py", line 436, in _iterencode
    o = _default(o)
  File "/usr/lib/python2.7/site-packages/fedmsg/encoding.py", line 42, in default
    return super(FedMsgEncoder, self).default(obj)
  File "/usr/lib64/python2.7/json/encoder.py", line 178, in default
    raise TypeError(repr(o) + " is not JSON serializable")
TypeError: time.struct_time(tm_year=2012, tm_mon=8, tm_mday=21, tm_hour=15, tm_min=38, tm_sec=20, tm_wday=1, tm_yday=234, tm_isdst=0) is not JSON serializable

When you hit ctrl-C during fedmsg-status you get an ugly traceback.

Traceback (most recent call last):
  File "/usr/bin/fedmsg-tail", line 9, in <module>
    load_entry_point('fedmsg==0.2.0', 'console_scripts', 'fedmsg-tail')()
  File "/usr/lib/python2.6/site-packages/fedmsg/commands/__init__.py", line 70, in wrapper
    return func(**config)
  File "/usr/lib/python2.6/site-packages/fedmsg/commands/tail.py", line 67, in tail
    for name, ep, topic, message in fedmsg.__context._tail_messages(**kw):
  File "/usr/lib/python2.6/site-packages/fedmsg/core.py", line 206, in _tail_messages
    subs[e].recv_multipart(zmq.NOBLOCK)
  File "socket.pyx", line 680, in zmq.core.socket.Socket.recv_multipart (zmq/core/socket.c:6095)
  File "socket.pyx", line 583, in zmq.core.socket.Socket.recv (zmq/core/socket.c:5725)
  File "socket.pyx", line 617, in zmq.core.socket.Socket.recv (zmq/core/socket.c:5587)
  File "socket.pyx", line 118, in zmq.core.socket._recv_copy (zmq/core/socket.c:1567)
  File "error.pyx", line 92, in zmq.core.error.ZMQError.__init__ (zmq/core/error.c:910)
KeyboardInterrupt

No idea why this is happening but when using mysql all the exists functions return false no matter what

The copy of /etc/fedmsg-config.py was still the copy from the RPM when I checked. I overwrote it by hand for now.

This should be a "from A to Z" document so that any joe-schmoe can make a new Fedora badge.

It should detail:

• How to make the image (and make it compliant with a standard style we decide on). Talk to mizmo about this.
• How to write the consumer that awards the badges.
• How to test the consumer.
• How/where to submit the badge-image+consumer to so it gets actually included in the infrastructure.
• Will there be a review process? Who will be in charge?

The fedmsg-status script is checking for it.

I wrote the mediawiki plugin back in fedmsg-0.1.0 but now we're on fedmsg-0.2.1. The config file has changed considerably, so the plugin will likely need to be updated first.

The entry-points model of moksha is getting in the way here.

I was going to test out fedmsg-irc on app02.stg, but fedoracommunity has an old moksha producer lurking there which gets picked up by the hub underlying fedmsg-irc. From the python API, the irc command needs to be able to say: "Don't load anything via extension points... only load what I tell you."

The following commands should be able to be daemonized:

• fedmsg-hub
• fedmsg-relay
• fedmsg-irc

There should be some sort of standardized way of daemonizing them and standard options to run them in the foreground.

for @nirik

First, there are problems on py2.6

13:54:36 < abadger1999> threebean: When I print schema.keys => ['log', 'update', 'agent', [...], 'tag', 'user', 'test']
13:54:43 < abadger1999> Is that [...]  meant to be there?
14:02:41 < abadger1999> huh.. I still get [...] when I import in python shell... /me tries pip installing the actual module
14:06:59 < abadger1999> threebean: The [...]   -- Try it on a RHEL/Centos6 box
14:07:14 < abadger1999> threebean: Looks like it's not happening on python-2.7 but it is on python-2.66

14:09:14 < abadger1999> threebean: I'd also make that a frozenset

Should we just get rid of the schema all together? It may be over-engineering.

@decause is drooling.

Make sure to provide links to it both from the Messaging_SIG wiki and from the README of this repo.

One for normal events, and the other for more verbose statistical events like httpd logs.

http://httpd.apache.org/docs/2.2/logs.html#piped can be used for httpd logging.

The following conversation went down in #fedora-admin

08:51:30 < misc> +config['endpoints']['relay_inbound'] = config['relay_inbound'] , seems weird, why not get the
configuration right at the first time ?
08:52:03 < threebean> long story.. this is one of the few places you need to do that :/
08:52:28 < threebean> ugh.. now that you mention it though, that confusing-ness should be hidden inside the fedmsg API.

This will affect the git-hook, the mediawiki hook, and fedmsg-logger.

For super large messages (like those coming from bodhi), fedmsg-irc truncates its output in a way that looks like a bug or a mistake.

Each process tries to bind its own zmq socket and they fail with ZMQError: Address already in use.

Don't emit any real names if they have private set.

There should be a master list somewhere so we don't lose track of all this.

http://ask.fedoraproject.org/badges/ has badges. We should incorporate them at some point.

12:05:00 < ricky> threebean: Random note on fedmsg's crypto.py: In the sign function, you sign the json encoding of the message dict, and then continue to pass the dict around

12:05:22 < ricky> However, it might not be guaranteed that the json encoding of a dict is the same on different machines

13:42:26 < threebean> ricky: oo, right. there could be a difference of the ordering of the pairs.

13:45:50 < threebean> hmm, I can trying monkeying around with an ordereddict that sorts its keys before encoding.

13:51:32 < threebean> a better idea might be to derive the signature from a different encoding. I'm thinking msgpack: http://pypi.python.org/pypi/msgpack-python/

I think we'll need to use @lmacken's pyrasite to figure it out, amirite?

The endpoint name needs to be rewritten for publishers from tcp://host.stg.fp.o:2001" totcp://*:2001``.

Like mediawiki and git do.

That way we don't have to maintain two separate sets of config files but just one but with variables/macros.

We want fedmsg daemons to pick up their configs with a little downtime as possible. If we can get them to do it without a formal restart, that would be awesome.

We need systemctl scripts for the long-running processes. Take a look at how moksha does it.

That message a consumer recieves should be this {"body": {"msg": "lub-dub"}} but it gets {"body": "{msg: "lub-dub"}"}

@rossdylan found it while working on fedbadges.

The config file is getting bloated. It would be nice to be able to install things piece-meal.

This should affect some of the work you're doing, @rossdylan.

15:45:30 < nirik> threebean: suggestion/RFE for git fedmsg messages: if they have more than one line, include the top line (like we do now), but add a "...more" or something to indicate the changelog is longer?

To start with, I saw this exception come up in /var/log/fedmsg/fedmsg-irc.log:

  File "/usr/lib64/python2.6/site-packages/twisted/python/log.py", line 84, in callWithLogger
    return callWithContext({"system": lp}, func, *args, **kw)
  File "/usr/lib64/python2.6/site-packages/twisted/python/log.py", line 69, in callWithContext
    return context.call({ILogContext: newCtx}, func, *args, **kw)
  File "/usr/lib64/python2.6/site-packages/twisted/python/context.py", line 59, in callWithContext
    return self.currentContext().callWithContext(ctx, func, *args, **kw)
  File "/usr/lib64/python2.6/site-packages/twisted/python/context.py", line 37, in callWithContext
    return func(*args,**kw)
  File "/usr/lib/python2.6/site-packages/txzmq/pubsub.py", line 60, in messageReceived
    self.gotMessage(message[1], message[0])
  File "/usr/lib/python2.6/site-packages/moksha/hub/zeromq/zeromq.py", line 143, in chain_over_moksha_callbacks
    f(_body, _topic)
  File "/usr/lib/python2.6/site-packages/moksha/hub/zeromq/zeromq.py", line 165, in intercept
    return callback(ZMQMessage(_topic, _body))
  File "/usr/lib/python2.6/site-packages/moksha/api/hub/consumer.py", line 100, in _consume_json
    self._consume(message_as_dict)
  File "/usr/lib/python2.6/site-packages/moksha/api/hub/consumer.py", line 109, in _consume
    self.consume(message)
  File "/usr/lib/python2.6/site-packages/fedmsg/consumers/ircbot.py", line 268, in consume
    terse=client.factory.terse,
  File "/usr/lib/python2.6/site-packages/fedmsg/consumers/ircbot.py", line 233, in prettify
    link=fedmsg.text._msg2link(msg, **self.hub.config),
  File "/usr/lib/python2.6/site-packages/fedmsg/consumers/ircbot.py", line 87, in ircprettify
    return fmt.format(title=title, subtitle=subtitle, link=link)
exceptions.UnicodeEncodeError: 'ascii' codec can't encode character u'\u0160' in position 5: ordinal not in range(128)

We have a pretty good idea, but we just need to write it down so others can check it out.

f17 python-twisted is 11.1.0 and rhel6 python-twisted is 8.2.0 (WHOAH)

The ircbot appears to be broken in stg until this is resolved, so we need to create a set of forward compat packages.