jonathanj / eliottree Goto Github PK

View Code? Open in Web Editor NEW

64.0 64.0 9.0 915 KB

Render Eliot logs as an ASCII tree

License: Other

Python 100.00%

eliot log-viewer python

eliottree's People

Contributors

Stargazers

Watchers

Forkers

tomprince jml itamarst mithrandi walkinreeds x0zzz emuhedo adiroiban cegfdb

eliottree's Issues

Factor out core functionality into a library

Currently, eliottree is a single script that does a bunch of stuff. If the core functionality were factored out into a library that the script used, it would be more maintainable (and testable!) and allow other tools to more easily build on top of eliottree.

Tree structure isn't always correct

Consider the following example:

4350e65c-bf31-4ecd-8d2d-b48a601dc6f6
    +-- check_links@1/started
        `-- url: http://localhost:8080
    +-- download@2,1/started
        `-- url: http://localhost:8080
        +-- download@2,2/succeeded
    +-- download@3,1/started
        `-- url: http://nosuchdomain
        +-- download@3,2/failed
            |-- exception: requests.exceptions.ConnectionError
            |-- reason: HTTPConnectionPool(host='nosuchdomain', port=80): Max retries exceeded with url: / (Caused by <class [...]
    +-- download@4,1/started
        `-- url: http://google.com
        +-- download@4,2/succeeded
    +-- check_links@5/succeeded

Notice that download@2,1 and friends are child actions of check_links@1, but that is not reflected in the structure.

Ability to customize default behaviour and colors

Sometimes the default colors don't work out well on certain themes (#78), sometimes a user might want a certain flag to be the default (--ascii), etc.

A comprehensive config file is probably the right solution for this.

Be able to chunk the display of the tree

I have some logs with really long tasks, that do not allow me to scroll all the way through the history.
I would like to be able to iterate the the log lines some chunk size at a time.
like

(pipeline_opt) λ eliot-tree logs/2020-08-19_0004.log -l 0 --chunk-size 10
2450eced-dde4-477d-96fc-64409a0b371f
└── pipeline_optimize.initialization.initialize_tanks.new_tank/1 ⇒ started 2020-08-19 23:33:03Z ⧖ 13.326s
    ├── fac: InletFacility(InletFacTag(site='SANEAST2', service='UIS', facility_name='THIS_FAC', property_num='121312'), [PumpFacility(FacilityTag(site='SANEAST2', service='UIS', facilityid='THIS_FAC_LACT'))]
    ├── initialization_days: 7
    ├── log_level: 30
    ├── pipeline_optimize.services.cygnet_services.cygnet.get_current_values_for_facility/2/1 ⇒ started 2020-08-19 23:33:03Z ⧖ 0.409s
    │   ├── fac: PumpFacility(FacilityTag(site='SANEAST2', service='UIS', facilityid='THIS_FAC_LACT'))
    │   ├── log_level: 30
    │   ├── pipeline_optimize.services.cygnet_services.cygnet.load_current_value/2/2/1 ⇒ started 2020-08-19 23:33:03Z ⧖ 0.017s
    │   │   ├── log_level: 30
Press any key to continue or 'X' to exit or 'T' for next task:

Eliot-tree fails to parse log that does not have task_level of 1

In long running (> 1 hour) Flocker compute nodes, Eliot logs rotate, resulting in latest logs not having task_level of 1: https://clusterhq.atlassian.net/browse/FLOC-2943

vagrant@node1:/var/log/flocker$ eliot-tree flocker-dataset-agent.log
Traceback (most recent call last):
File "/usr/local/bin/eliot-tree", line 11, in <module>
sys.exit(main())
File "/usr/local/lib/python2.7/dist-packages/eliottree/_cli.py", line 94, in main
display_task_tree(args)
File "/usr/local/lib/python2.7/dist-packages/eliottree/_cli.py", line 47, in display_task_tree
nodes=tree.nodes(tree.merge_tasks(tasks, filter_funcs())),
File "/usr/local/lib/python2.7/dist-packages/eliottree/tree.py", line 164, in merge_tasks
raise RuntimeError('Some tasks have no start parent', pending)

Would be useful to have eliot-tree help debug Flocker issues using production logs. Thanks a lot for any help fixing this issue!

All tasks are displayed if jmespath query matches nothing

The expected behaviour is, of course, that no tasks are displayed.

I believe the cause of this issue is that merge_tasks returns None if nothing matched; this value is then passed to nodes which interprets None as "all tasks". I think the fix is just to return [] instead of None.

Filtering by end fails on version 18.1.1

I get an error when I write the below query

λ eliot-tree logs\2020-06-01_0006.log --select "log_level=='DEBUG'" --end "2020-06-01T14:40:48"
Traceback (most recent call last):
  File "%userprofile%\.venv\Scripts\eliot-tree-script.py", line 9, in <module>
    sys.exit(main())
  File "%userprofile%\.venv\lib\site-packages\eliottree\_cli.py", line 178, in main
    human_readable=args.human_readable)
  File "%userprofile%\.venv\lib\site-packages\eliottree\_cli.py", line 92, in display_tasks
    colorize=colorize)
  File "%userprofile%\.venv\lib\site-packages\eliottree\_render.py", line 264, in render_tasks
    for task in tasks:
  File "%userprofile%\.venv\lib\site-packages\eliottree\_parse.py", line 19, in tasks_from_iterable
    for message_dict in iterable:
  File "%userprofile%\.venv\lib\site-packages\toolz\functoolz.py", line 488, in __call__
    ret = f(ret)
  File "%userprofile%\.venv\lib\site-packages\eliottree\filter.py", line 47, in _filter
    return _parse_timestamp(task[u'timestamp']) < end_date
TypeError: 'bool' object is not subscriptable

I conda installed it from conda-forge (thank you for putting it there btw).
the eliot-tree version is '18.1.1'

Parse errors lack sufficient context to track down the problem log/data

When eliot-tree encounters a parse error of some kind (perhaps the json is malformed, perhaps the structure of the encoded object violates some assumption, etc) eliot-tree mostly just tosses up an unhandled exception to the user. The exception may give some general idea about what went wrong (eg KeyError for timestamp - not bad - or some more obscure error from the json parser) but it doesn't seem to give much useful information about where it went wrong.

A line number along with such error reports would go a long way towards helping fix these problems. Better exception formatting for common cases (like "I wanted this key and it was missing") would be pretty nice too.

Color action start messages the same way the end messages are colored

It makes it a bit easier to identify the beginning and end of a failure/success.

Argument Error when use eliot tree

Test code:

from eliot import log_call, to_file

to_file(open('log.log', 'w'))

@log_call(action_type='CALC', include_args=['x'], include_result=True)
def calculate(x, y):
    return x * y

x = 2
y = 10
result = calculate(x, y)
print(result)

Then use eliot-tree log.log it returns an Argument Error.

Exceptions (1) occurred during processing:
Traceback (most recent call last):
  File "C:\Users\paule\Anaconda3\lib\site-packages\toolz\functoolz.py", line 781, in __call__
    return self.func(*args, **kwargs)
  File "C:\Users\paule\Anaconda3\lib\site-packages\eliottree\_render.py", line 117, in format_node
    format.escape_control_characters(node.root().task_uuid)))
  File "C:\Users\paule\Anaconda3\lib\site-packages\eliottree\_color.py", line 43, in __color
    return colored(text, fg, bg, attrs=attrs)
  File "C:\Users\paule\Anaconda3\lib\site-packages\eliottree\_color.py", line 30, in colored
    _colored.fg(fg) if fg is not None else u'',
  File "C:\Users\paule\Anaconda3\lib\site-packages\colored\colored.py", line 402, in fg
    return colored(color).foreground()
  File "C:\Users\paule\Anaconda3\lib\site-packages\colored\colored.py", line 22, in __init__
    self.enable_windows_terminal_mode()
  File "C:\Users\paule\Anaconda3\lib\site-packages\colored\colored.py", line 373, in enable_windows_terminal_mode
    hStdout = windll.kernel32.GetStdHandle(STD_OUTPUT_HANDLE)
ctypes.ArgumentError: argument 1: <class 'TypeError'>: wrong type

Failure trying to parse a task tree with missing messages

This often happens because of log rotation; an operation that straddles midnight will have the start action go into the previous day's log, but the end action into the next day's log.

The traceback looks like this:

Traceback (most recent call last):
  File "/appenv/bin/eliot-tree", line 11, in <module>
    sys.exit(main())
  File "/appenv/local/lib/python2.7/site-packages/eliottree/_cli.py", line 108, in main
    display_task_tree(args)
  File "/appenv/local/lib/python2.7/site-packages/eliottree/_cli.py", line 58, in display_task_tree
    human_readable=args.human_readable)
  File "/appenv/local/lib/python2.7/site-packages/eliottree/_cli.py", line 46, in build_task_nodes
    return tree.nodes(tree.merge_tasks(tasks, filter_funcs()))
  File "/appenv/local/lib/python2.7/site-packages/eliottree/tree.py", line 176, in merge_tasks
    pending = _merge(tasks)
  File "/appenv/local/lib/python2.7/site-packages/eliottree/tree.py", line 170, in _merge
    node.add_child(_TaskNode(task))
  File "/appenv/local/lib/python2.7/site-packages/eliottree/tree.py", line 80, in add_child
    _add_child(self, node.task['task_level'])
  File "/appenv/local/lib/python2.7/site-packages/eliottree/tree.py", line 77, in _add_child
    _add_child(children[level], levels)
  File "/appenv/local/lib/python2.7/site-packages/eliottree/tree.py", line 74, in _add_child
    level = levels.pop(0)
IndexError: pop from empty list

I didn't construct a minimal reproduction yet, but @jonathanj has access to the data that I used; if someone else wants to look into this, just yell at me, and I'll come up with an example.

Error occur when using Eliot-tree from docker container

Hi, I'm using Eliot-tree and it works very well.
The error occurs when I'm logging into a running docker container, and trying to produce the structured logs with Eliot-tree.
I'm getting the following error:

I'm using ubuntu 18.04, Docker version 19.03.12.

Any idea why do I get this error when using Eliot-tree inside a container?
Thanks!

On my terminal, prints white-on-white text

On my terminal, eliot-tree output looks like:

I find that UUID somewhat hard to read :-)

Human-readable values should only be formatted when rendered instead of modifying the tree data

The problem with modifying the tree data is that it makes it very difficult to write queries against it if eliot-tree is changing it in undisclosed ways to suit it's renderer.

Use Eliot's builtin parser

This would fix #43 and reduce the amount of code and complexity in eliottree.

Timestamp parsing and display handles timezones incorrectly

Currently the parsing code parses the timestamp as a local time and the display code displays it as though the data was in UTC.

Render `dict` values as sub-trees

Command prompt & python usage

Hi,

I am using jupyter lab notebook to execute the following:

from eliot import log_call, to_file

eliot_file =r"c:\Users\user\Downloads\eliot.log"
to_file(open(eliot_file, "w"))

@log_call
def add(a, b):
    return a + b

@log_call
def multiply(a, b):
    return a * b

multiply(add(1,2), 4)

Result is 12, and content of eliot_file is:

{"a": 1, "b": 2, "action_status": "started", "timestamp": 1578133156.6194398, "task_uuid": "ecd3e4f3-3f20-428f-8392-2c719e60e674", "action_type": "__main__.add", "task_level": [1]}
{"result": 3, "action_status": "succeeded", "timestamp": 1578133156.6204777, "task_uuid": "ecd3e4f3-3f20-428f-8392-2c719e60e674", "action_type": "__main__.add", "task_level": [2]}
{"a": 3, "b": 4, "action_status": "started", "timestamp": 1578133156.6204777, "task_uuid": "fcd2c4d6-83c0-4133-b5c6-d9049ed960fb", "action_type": "__main__.multiply", "task_level": [1]}
{"result": 12, "action_status": "succeeded", "timestamp": 1578133156.6214757, "task_uuid": "fcd2c4d6-83c0-4133-b5c6-d9049ed960fb", "action_type": "__main__.multiply", "task_level": [2]}

From windows 10, command line "eliot-tree eliot.log", situation is as following:

The picture above is with command prompt code page 437 (OEM - United States).
I tried to change code page to UTF-8 ("CHCP 65001"), but I still got special characters instead of symbols.

Question 1: How to set up windows 10 in order to get output of eliot-tree readable?

I tried to get the tree directly from python3, but I am not sure how to create tasks in:

import json, sys
from eliottree import tasks_from_iterable, render_tasks
render_tasks(sys.stdout.write, tasks, colorize=True)

I tried:

with open(eliot_file, "r") as f:
    x = f.readlines()

and then

render_tasks(sys.stdout.write, x, colorize=True)

y = tasks_from_iterable(x)
# or y = tasks_from_iterable(eliot_file)
render_tasks(sys.stdout.write, y, colorize=True)

but with no success ("").

Question 2: How to generate tasks from log file?

Regards.

Tasks reported out of order if some events are deleted / missing

Jonathan,

Fantastic tool! I have been making great use of it!

I have an application which logs debug events when a debug option is enabled. I've been experimenting with the idea of always logging the debug information, and filtering the debug events out of the log when I don't want/need to see them. I found that if I delete an event it can cause eliot-tree to render the associated task out of (chronological) order (presumably because the parser is waiting for the missing event to show up in the log data, and then eventually gives up?)

Here's an example:

from eliot import start_action, to_file, Message
import subprocess

to_file(open("normal.log", "w"))
with start_action(action_type="first_action"):
    Message.log(message_type="some_message", level='debug')
    Message.log(message_type="some_message", level='info')

with start_action(action_type="second_action"):
    pass

# Create a version of the log file in which the 'debug' event has been deleted
with open('normal.log') as file:
    with open('filtered.log', 'w') as filtered_file:
        for line in file:
            if 'debug' not in line:
                filtered_file.write(line)

subprocess.call('eliot-tree --color never normal.log > normal.tree', shell=True)
subprocess.call('eliot-tree --color never filtered.log > filtered.tree', shell=True)

Which results in:

normal.tree

ceed1fcb-51fc-42a6-8219-e3157e98b402
└── first_action/1 ⇒ started
    ├── timestamp: 2017-11-04 02:31:24.041994
    ├── some_message/2
    │   ├── level: debug
    │   └── timestamp: 2017-11-04 02:31:24.042184
    ├── some_message/3
    │   ├── level: info
    │   └── timestamp: 2017-11-04 02:31:24.042294
    └── first_action/4 ⇒ succeeded
        └── timestamp: 2017-11-04 02:31:24.042397

47d6d5a3-4f7c-4276-8bae-014a9a2e2875
└── second_action/1 ⇒ started
    ├── timestamp: 2017-11-04 02:31:24.042547
    └── second_action/2 ⇒ succeeded
        └── timestamp: 2017-11-04 02:31:24.042654

filtered.tree

47d6d5a3-4f7c-4276-8bae-014a9a2e2875
└── second_action/1 ⇒ started
    ├── timestamp: 2017-11-04 02:31:24.042547
    └── second_action/2 ⇒ succeeded
        └── timestamp: 2017-11-04 02:31:24.042654

ceed1fcb-51fc-42a6-8219-e3157e98b402
└── first_action/1 ⇒ started
    ├── timestamp: 2017-11-04 02:31:24.041994
    ├── some_message/3
    │   ├── level: info
    │   └── timestamp: 2017-11-04 02:31:24.042294
    └── first_action/4 ⇒ succeeded
        └── timestamp: 2017-11-04 02:31:24.042397

Perhaps I am crazy to be trying to filter events, but it seems so close to working that I thought I'd ask what you thought.

Cheers!

Tree node children are not correctly computed

Something something broken addChild code.

Strange format

Dear Jonathan,

I would like to ask for your kind help.

This is the first time that I try to use eliot whit eliot-tree.

The problem that the result formatted by eliot-tree looks very both in cmd, powershell and PyCharm's terminal.

I use Windows 10.

Thanks for your help in advance.

`iso8601.iso8601.Utc` is not reliably available

Version 0.1.12 doesn't expose this for Python 3.2.0 and newer, it looks like this is not really a public export anyway and we ought to be using UTC which is an instance of a UTC tzinfo implementation.

Unicode interface for `write` breaks backwards compatibility

15.3.0 requires a write that takes bytes, master currently requires a requires a write that takes Unicode.

Switch to `eliot.parse`, available in Eliot 1.6

Currently eliot-tree is relying on a private API, eliot._parse.

Prettier tree output

Over the weekend, I hacked together jml/tree-format, which renders Python trees in the format of the Unix tree command, e.g.

foo
├── bar
│   ├── a
│   └── b
├── baz
└── qux
    └── c

Would you be interested in a patch that adds this mode of output to eliottree?

Protect me against terminal hacking

Sometimes when I use eliottree to dump a tree to stdout, something like this happens:

    +-- network-client:create@3/succeeded
        |-- response_object:
            |-- apiVersion: v1
            |-- data:
                |-- .3: N\u2d1d\u42a3\uacfe\u7e8e\U000cb9dd
                |-- .3.g.h.a.2g.d-b.lk6tr.7.le0cajs.6dfq: \xa6\x8c\x96
                |-- .ai.dvvh6.1.zzxuy.7.ve: \xba
                |-- .sd.u3g0zc1.l.v: 
                |-- 473: 
                    [\U000e6273 \u03d7\u98cao$\u8c93\u9f53\u013b
                |-- 8ka7kk89vv.9tlhgrivtk: '\U000d2fff)
                |-- 8q.8.d: \u9974\U00021389
                    \U000b4e5a
                    9(\u80a1\┤b3ec\┤b7°3\┤4711
                ≠↑↑ ┴°≥d⎽┼11⎺┬│d: \┤7▮b7
                    \┤▮167H
                ◆↑↑ ≥⎼2↓⎺┼↑7d↓☃⎽: \┤17bb\U▮▮▮21588\U▮▮▮44cdc│
            ≠↑↑ ┐☃┼d: C⎺┼°☃±M▒⎻
            ◆↑↑ └e├▒d▒├▒:
                ≠↑↑ ▒┼┼⎺├▒├☃⎺┼⎽:
                ≠↑↑ c┌┤⎽├e⎼N▒└e: N⎺┼e
                ≠↑↑ de┌e├☃⎺┼G⎼▒cePe⎼☃⎺dSec⎺┼d⎽: N⎺┼e
                ≠↑↑ °☃┼▒┌☃≥e⎼⎽: []
                ≠↑↑ ±e┼e⎼▒├eN▒└e: N⎺┼e
                ≠↑↑ ±e┼e⎼▒├☃⎺┼: N⎺┼e
                ≠↑↑ ┌▒be┌⎽:
                    ≠↑↑ b: ┐
                    ≠↑↑ ─↓b┴┤▮├e26↓d▮49/⎽┐5⎺☃: 67▒e1c
                    ≠↑↑ ⎼±┌°└⎽┌9│: ├c
...
FAILED (e⎼⎼⎺⎼⎽=5← ⎽┤cce⎽⎽e⎽=84)
(├│┐┤be) e│▒⎼┐┤┼@b▒⎼≤⎺┼:·/W⎺⎼┐/Le▒⎽├A┤├▒⎺⎼☃├≤/├│┐┤be$

Now my terminal is unusable and the latter portion of the tree report has all been corrupted and is garbage.

eliottree should take care in formatting output so as not to provoke this behavior from terminals.

Render timestamp in a more consistent position

Currently the timestamp is rendered as a message field, depending on what other fields are present the visual position of this field will move around. An improvement would be to render the timestamp after the status (for an action) or level (for a message).

Support outputting timestamps in local timezone

UTC can sometimes be harder to interpret than local time, so would be nice if there was way to output in local timezone as well.

Original issue: itamarst/eliot#428

Handle exceptions during formatting more gracefully

Sometimes timestamps are missing / None for example:

  File "/home/mithrandi/code/eliottree/eliottree/format.py", line 47, in _format_timestamp_value
    result = datetime.utcfromtimestamp(float(value)).isoformat(' ')
TypeError: float() argument must be a string or a number

--raw will work around this but then you get no formatting for anything.

Support tailing logs

It would be really nice if this could incrementally render tailed logs.

Use Versioneer

Unicode error when piping to file

Hi, I'm trying to pipe the output of eliot-tree to a file on Windows 10, but I'm getting the following traceback:

Traceback (most recent call last):
  File "c:\users\zach\appdata\local\programs\python\python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\zach\appdata\local\programs\python\python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\Zach\AppData\Local\pypoetry\Cache\virtualenvs\ns-trcd-5J_fInKt-py3.7\Scripts\eliot-tree.exe\__main__.py", line 9, in <module>
  File "c:\users\zach\appdata\local\pypoetry\cache\virtualenvs\ns-trcd-5j_finkt-py3.7\lib\site-packages\eliottree\_cli.py", line 318, in main
    theme_overrides=config.get('theme_overrides'))
  File "c:\users\zach\appdata\local\pypoetry\cache\virtualenvs\ns-trcd-5j_finkt-py3.7\lib\site-packages\eliottree\_cli.py", line 126, in display_tasks
    theme=theme)
  File "c:\users\zach\appdata\local\pypoetry\cache\virtualenvs\ns-trcd-5j_finkt-py3.7\lib\site-packages\eliottree\_render.py", line 288, in render_tasks
    write(format_tree(task, _format_node, _get_children, make_options()))
  File "c:\users\zach\appdata\local\programs\python\python37\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\u21d2' in position 71: character maps to <undefined>

The file I'm using looks like this:

{"action_status": "started", "timestamp": 1582840009.482032, "task_uuid": "6ffa3c9b-3e24-4ee7-bb57-2e40e01171b6", "action_type": "_set_initial_widget_states", "task_level": [1]}
{"new_max": 5, "action_status": "started", "timestamp": 1582840009.482032, "task_uuid": "6ffa3c9b-3e24-4ee7-bb57-2e40e01171b6", "action_type": "update_max_measurements", "task_level": [2, 1]}
{"action_status": "succeeded", "timestamp": 1582840009.482032, "task_uuid": "6ffa3c9b-3e24-4ee7-bb57-2e40e01171b6", "action_type": "update_max_measurements", "task_level": [2, 2]}
{"action_status": "succeeded", "timestamp": 1582840009.482032, "task_uuid": "6ffa3c9b-3e24-4ee7-bb57-2e40e01171b6", "action_type": "_set_initial_widget_states", "task_level": [3]}
{"new_max": 50, "action_status": "started", "timestamp": 1582840013.952636, "task_uuid": "ab19b06e-fb4d-406f-be82-109b5530df3b", "action_type": "update_max_measurements", "task_level": [1]}
{"action_status": "succeeded", "timestamp": 1582840013.952636, "task_uuid": "ab19b06e-fb4d-406f-be82-109b5530df3b", "action_type": "update_max_measurements", "task_level": [2]}
{"action_status": "started", "timestamp": 1582840018.5424337, "task_uuid": "9a2da24e-bf98-442e-b158-7b84845f7a0b", "action_type": "set_time_axis", "task_level": [1]}
{"action_status": "succeeded", "timestamp": 1582840018.5424337, "task_uuid": "9a2da24e-bf98-442e-b158-7b84845f7a0b", "action_type": "set_time_axis", "task_level": [2]}
{"action_status": "started", "timestamp": 1582840023.4722626, "task_uuid": "6c6ecdac-b624-4e88-849a-91ffee883af5", "action_type": "close", "task_level": [1]}
{"action_status": "succeeded", "timestamp": 1582840023.4722626, "task_uuid": "6c6ecdac-b624-4e88-849a-91ffee883af5", "action_type": "close", "task_level": [2]}

I've tried making the encoding process as simple as possible by using the --ascii and --color never flags, but the issue persists. Am I missing something simple, or is this a bug?

Give me a way to select a time range of log messages

I have a big chunk of eliot logs. I have another log file with timestamps in it like "2015-10-29 12:45:16+0000" that identify interesting time ranges. I'd like to use eliot-tree (among other tools, perhaps) to examine the eliot logs specifically from those time ranges.

Something like:

eliot-tree --start "2015-10-29 12:45:16+0000" --end "..."

would be pretty great.

Something using the --select feature may well do the job too (though it's probably guaranteed to involve more typing - but shell aliases or something I guess 😉).

I tried --select "timestamp > ..." but even with a ton of extra support code to convert date formats and such, eliot-tree (or jmespath, dunno) eventually gets angry at comparison between a datetime and an int.

Include some examples of ``--select``.

It isn't clear how to write useful queries for `--select``, just from looking at the linked documentation.

Rendering problems with updated tree code

The new code apparently always assumes message 1 is the parent... but sometimes it isn't. In particular in the root of the action tree it might be a standalone message.

 $ python reproducer.py | eliot-tree 
b201ac9a-f6bb-4660-a0b4-c0ad2646659b
+-- sibling1@1
    `-- timestamp: 2015-08-08 14:52:33.515778
    +-- sibling2@2
        `-- timestamp: 2015-08-08 14:52:33.516109

$ cat reproducer.py 
from eliot import Message, to_file
from sys import stdout
to_file(stdout)

Message.log(message_type="sibling1")
Message.log(message_type="sibling2")

Allow selecting top-level trees by some kind of query

grep can't really help narrow down the output of eliottree in a useful way, what needs to happen is eliottree should prune the output according to some kind of query. http://jmespath.org/ might be quite useful for achieving this.

encoding error / windows traceback

I'm not entirely sure what's going on here, but on a GitHub Actions windows runner, I'm seeing this from eliot-tree:

Traceback (most recent call last):
  File "c:\hostedtoolcache\windows\python\3.8.10\x64\lib\runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\hostedtoolcache\windows\python\3.8.10\x64\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\hostedtoolcache\windows\Python\3.8.10\x64\Scripts\eliot-tree.exe\__main__.py", line 7, in <module>
  File "c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages\eliottree\_cli.py", line 308, in main
    display_tasks(
  File "c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages\eliottree\_cli.py", line 116, in display_tasks
    render_tasks(
  File "c:\hostedtoolcache\windows\python\3.8.10\x64\lib\site-packages\eliottree\_render.py", line 289, in render_tasks
    write(format_tree(task, _format_node, _get_children, options))
  File "c:\hostedtoolcache\windows\python\3.8.10\x64\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode characters in position 38-40: character maps to <undefined>

I've attached the eliot.log from the run (but when using eliot-tree on it on a Linux machine, I do not see the above error).

Regular messages aren't supported

E.g. try piping in:

{"task_uuid": "e96b14e5-21a6-421c-884e-af82e0018a85", "error": false, "timestamp": 1427725835.188358, "message": "Log opened.", "message_type": "twisted:log", "task_level": [1]}

and you'll get an error. Here's a fix:

diff --git a/eliottree.py b/eliottree.py
index 13af0b1..a9c1427 100644
--- a/eliottree.py
+++ b/eliottree.py
@@ -28,7 +28,7 @@ def _taskName(task):
     """
     level = u','.join(map(unicode, task[u'task_level']))
     messageType = task.get('message_type', None)
-    if messageType == u'eliot:traceback':
+    if messageType is not None:
         status = None
     elif messageType is None:
         messageType = task['action_type']

Add an option to print full failure messages

In this example, the interesting error is

exceptions.IOError: [Errno 2] No such file or directory: '/etc/flocker/agent.yml'\n"

But Eliot tree doesn't show it.

(1988)(flocker-ca-output-FLOC-1988 ? =)[~/.../HybridLogic/flocker]$ flocker-dataset-agent  > log
(1988)(flocker-ca-output-FLOC-1988 ? =)[~/.../HybridLogic/flocker]$ cat log
{"task_uuid": "cdeb220d-7605-4d5f-8341-1a170222e308", "error": false, "timestamp": 1432731519.112539, "message": "Log opened.", "message_type": "twisted:log", "task_level": [1]}
{"status": 1, "zfs_command": "zfs set readonly=on flocker", "output": "[Errno 2] No such file or directory", "timestamp": 1432731519.117719, "message_type": "filesystem:zfs:error", "task_uuid": "cdeb220d-7605-4d5f-8341-1a170222e308", "task_level": [2]}
{"status": 1, "zfs_command": "zfs set canmount=off flocker", "output": "[Errno 2] No such file or directory", "timestamp": 1432731519.122177, "message_type": "filesystem:zfs:error", "task_uuid": "cdeb220d-7605-4d5f-8341-1a170222e308", "task_level": [3]}
{"task_uuid": "cdeb220d-7605-4d5f-8341-1a170222e308", "error": true, "timestamp": 1432731519.123826, "message": "Unhandled Error\nTraceback (most recent call last):\n  File \"/home/richard/projects/HybridLogic/flocker/flocker/node/script.py\", line 404, in flocker_dataset_agent_main\n    options=options,\n  File \"/home/richard/projects/HybridLogic/flocker/flocker/common/script.py\", line 187, in main\n    self._react(run_and_log, [], _reactor=self._reactor)\n  File \"/home/richard/.virtualenvs/1988/lib/python2.7/site-packages/twisted/internet/task.py\", line 875, in react\n    finished = main(_reactor, *argv)\n  File \"/home/richard/projects/HybridLogic/flocker/flocker/common/script.py\", line 178, in run_and_log\n    d = maybeDeferred(self.script.main, reactor, options)\n--- <exception caught here> ---\n  File \"/home/richard/.virtualenvs/1988/lib/python2.7/site-packages/twisted/internet/defer.py\", line 140, in maybeDeferred\n    result = f(*args, **kw)\n  File \"/home/richard/projects/HybridLogic/flocker/flocker/volume/service.py\", line 452, in main\n    return self._volume_script.main(reactor, options, service)\n  File \"/home/richard/projects/HybridLogic/flocker/flocker/node/script.py\", line 366, in main\n    configuration = yaml.safe_load(agent_config.getContent())\n  File \"/home/richard/.virtualenvs/1988/lib/python2.7/site-packages/twisted/python/filepath.py\", line 290, in getContent\n    fp = self.open()\n  File \"/home/richard/.virtualenvs/1988/lib/python2.7/site-packages/twisted/python/filepath.py\", line 845, in open\n    return open(self.path, mode + 'b')\nexceptions.IOError: [Errno 2] No such file or directory: '/etc/flocker/agent.yml'\n", "message_type": "twisted:log", "task_level": [4]}
{"task_uuid": "cdeb220d-7605-4d5f-8341-1a170222e308", "error": true, "timestamp": 1432731519.124293, "message": "main function encountered error\nTraceback (most recent call last):\n  File \"/home/richard/projects/HybridLogic/flocker/flocker/node/script.py\", line 404, in flocker_dataset_agent_main\n    options=options,\n  File \"/home/richard/projects/HybridLogic/flocker/flocker/common/script.py\", line 187, in main\n    self._react(run_and_log, [], _reactor=self._reactor)\n  File \"/home/richard/.virtualenvs/1988/lib/python2.7/site-packages/twisted/internet/task.py\", line 875, in react\n    finished = main(_reactor, *argv)\n  File \"/home/richard/projects/HybridLogic/flocker/flocker/common/script.py\", line 178, in run_and_log\n    d = maybeDeferred(self.script.main, reactor, options)\n--- <exception caught here> ---\n  File \"/home/richard/.virtualenvs/1988/lib/python2.7/site-packages/twisted/internet/defer.py\", line 140, in maybeDeferred\n    result = f(*args, **kw)\n  File \"/home/richard/projects/HybridLogic/flocker/flocker/volume/service.py\", line 452, in main\n    return self._volume_script.main(reactor, options, service)\n  File \"/home/richard/projects/HybridLogic/flocker/flocker/node/script.py\", line 366, in main\n    configuration = yaml.safe_load(agent_config.getContent())\n  File \"/home/richard/.virtualenvs/1988/lib/python2.7/site-packages/twisted/python/filepath.py\", line 290, in getContent\n    fp = self.open()\n  File \"/home/richard/.virtualenvs/1988/lib/python2.7/site-packages/twisted/python/filepath.py\", line 845, in open\n    return open(self.path, mode + 'b')\nexceptions.IOError: [Errno 2] No such file or directory: '/etc/flocker/agent.yml'\n", "message_type": "twisted:log", "task_level": [5]}
{"task_uuid": "cdeb220d-7605-4d5f-8341-1a170222e308", "error": false, "timestamp": 1432731519.12443, "message": "Main loop terminated.", "message_type": "twisted:log", "task_level": [6]}
(1988)(flocker-ca-output-FLOC-1988 ? =)[~/.../HybridLogic/flocker]$ eliot-tree log 
cdeb220d-7605-4d5f-8341-1a170222e308
    +-- twisted:log@1/None
        |-- error: False
        |-- message: Log opened.
        |-- message_type: twisted:log
        `-- timestamp: 2015-05-27 13:58:39.112539
    +-- filesystem:zfs:error@2/None
        |-- message_type: filesystem:zfs:error
        |-- output: [Errno 2] No such file or directory
        |-- status: 1
        |-- timestamp: 2015-05-27 13:58:39.117719
        `-- zfs_command: zfs set readonly=on flocker
    +-- filesystem:zfs:error@3/None
        |-- message_type: filesystem:zfs:error
        |-- output: [Errno 2] No such file or directory
        |-- status: 1
        |-- timestamp: 2015-05-27 13:58:39.122177
        `-- zfs_command: zfs set canmount=off flocker
    +-- twisted:log@4/None
        |-- error: True
        |-- message: Unhandled Error...
        |-- message_type: twisted:log
        `-- timestamp: 2015-05-27 13:58:39.123826
    +-- twisted:log@5/None
        |-- error: True
        |-- message: main function encountered error...
        |-- message_type: twisted:log
        `-- timestamp: 2015-05-27 13:58:39.124293
    +-- twisted:log@6/None
        |-- error: False
        |-- message: Main loop terminated.
        |-- message_type: twisted:log
        `-- timestamp: 2015-05-27 13:58:39.124430

It'd be nice if it had an option to show full un-elipsized (is that the term) message field contents.
Something like journalctl s --full option, maybe?

Or have a way to show twisted:log@5/None in full?

Error running eliot-tree on Python 3

Versions:

eliot (0.8.0)
eliot-tree (15.3.0)

Python 3.4.3 (v3.4.3:9b73f1c3e601, Feb 23 2015, 02:52:03)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin

%> uname -a
Darwin dmt 14.5.0 Darwin Kernel Version 14.5.0: Wed Jul 29 02:26:53 PDT 2015; root:xnu-2782.40.9~1/RELEASE_X86_64 x86_64

%> eliot-tree
Traceback (most recent call last):
  File "/Users/vg/Devel/DetailsAuto/new/backend/.venv/bin/eliot-tree", line 9, in <module>
    load_entry_point('eliot-tree==15.3.0', 'console_scripts', 'eliot-tree')()
  File "/Users/vg/Devel/DetailsAuto/new/backend/.venv/lib/python3.4/site-packages/pkg_resources/__init__.py", line 474, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/Users/vg/Devel/DetailsAuto/new/backend/.venv/lib/python3.4/site-packages/pkg_resources/__init__.py", line 2582, in load_entry_point
    return ep.load()
  File "/Users/vg/Devel/DetailsAuto/new/backend/.venv/lib/python3.4/site-packages/pkg_resources/__init__.py", line 2265, in load
    return self._load()
  File "/Users/vg/Devel/DetailsAuto/new/backend/.venv/lib/python3.4/site-packages/pkg_resources/__init__.py", line 2268, in _load
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/Users/vg/Devel/DetailsAuto/new/backend/.venv/lib/python3.4/site-packages/eliottree/__init__.py", line 3, in <module>
    from eliottree.tree import Tree
  File "/Users/vg/Devel/DetailsAuto/new/backend/.venv/lib/python3.4/site-packages/eliottree/tree.py", line 129
    return sorted(nodes, key=lambda (_, n): n.task[u'timestamp'])
                                    ^
SyntaxError: invalid syntax

Messages without a message_type appear as unnamed tasks

Messages without a message_type appear as <UNNAMED TASK>s.

So the following code:

import eliot
import sys

eliot.to_file(sys.stdout)

with eliot.start_action(action_type=u"task"):
    eliot.Message.log(key=u"value")

Produces the following output:

7d074742-0116-4a57-a062-afa05bf13d5d
+-- task@1/started
    `-- timestamp: 2017-02-15 20:14:00.742261
    +-- <UNNAMED TASK>
        |-- key: value
        `-- timestamp: 2017-02-15 20:14:00.742498
    +-- task@3/succeeded
        `-- timestamp: 2017-02-15 20:14:00.742647

The message appears as an <UNNAMED TASK>. Providing a message_type field results in the message appearing as a message. This should be fixed or at least documented.

Exception tasks are not handled at all

conda-forge package

According to the documents on eliot
https://eliot.readthedocs.io/en/stable/quickstart.html

this was on conda-forge at one time, would it be possible to put it back?

Thanks

eliot:destination_failure doesn't have either (itamarst/eliot#165)
Messages with out a top-level action don't have a task_uuid.

pip install fails on python 3

Requirement already satisfied: jmespath>=0.7.1 in c:\anaconda3\lib\site-packages (from eliot-tree)
Collecting iso8601>=0.1.10 (from eliot-tree)
  Downloading iso8601-0.1.12-py3-none-any.whl
Collecting tree-format>=0.1.1 (from eliot-tree)
  Downloading tree-format-0.1.1.tar.gz
    Complete output from command python setup.py egg_info:
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "C:\Users\Gal\AppData\Local\Temp\pip-build-0inmxyd1\tree-format\setup.py", line 23, in <module>
        long_description = readme.read()
      File "C:\Anaconda3\lib\encodings\cp1252.py", line 23, in decode
        return codecs.charmap_decode(input,self.errors,decoding_table)[0]
    UnicodeDecodeError: 'charmap' codec can't decode byte 0x8f in position 223: character maps to <undefined>

    ----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in C:\Users\Gal\AppData\Local\Temp\pip-build-0inmxyd1\tree-format\

Render without graphics characters or display properly in less(1)

Maybe his is a help request for less(1), and not a complaint about eliot-tree, but I think it might be handy if either:

eliot-tree could be coaxed into rendering trees without VT100 graphics characters

mention could be made in the README of how to get less(1) to properly render those characters.