crossbario / autobahn-testsuite Goto Github PK

I tried to install the testsuite with pip install autobahntestsuite under a python 3.4 environement and got this error when running wstest:

➜  ~  wstest --help
Traceback (most recent call last):
  File "/usr/bin/wstest", line 9, in <module>
    load_entry_point('autobahntestsuite==0.7.1', 'console_scripts', 'wstest')()
  File "/usr/lib/python3.4/site-packages/pkg_resources/__init__.py", line 519, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python3.4/site-packages/pkg_resources/__init__.py", line 2630, in load_entry_point
    return ep.load()
  File "/usr/lib/python3.4/site-packages/pkg_resources/__init__.py", line 2310, in load
    return self.resolve()
  File "/usr/lib/python3.4/site-packages/pkg_resources/__init__.py", line 2316, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/lib/python3.4/site-packages/autobahntestsuite/__init__.py", line 22, in <module>
    import choosereactor # This MUST BE the FIRST file imported here! Do NOT touch.
ImportError: No module named 'choosereactor'

It looks like they should now be https://github.com/zaphoyd/wsperf.

Outline of WAMP v2 client library test mode.

To test a WAMP client library with Autobahn|Testsuite, one needs to implement a testee component using the client library.
A testee component is a WAMP application component that behaves in a defined way.
E.g. it needs to register a procedure “com.math.add2” which takes two numbers and returns their sum and it needs to publish an event to “wamp.testee.onready” when it has fully initialized.
The testee component then connects to a router, and a test driver component connected to the same router will then orchestrate the test.
During the test, a defined schedule of operations is carried out by the test driver on the testee component, and the testee component’s reactions and behavior is recorded.
The test recording is stored and analyzed. A test report is generated.

make above cases informational

I appreciate the work done for this tessuite. I use it to debug a websocket server I built. The problem is that there are many test cases and I want to narrow down the debugging to specific test cases. For example if the problem is on test case 9.2.3 I don't want to have to wait for all the tests for every attempt to fix my bug.

So is there a way to run a specific testcase?

Examples:

wstest --mode fuzzingclient --testcase 9.2.3
wstest --mode fuzzingclient --testcase 7-9 # this would run all test cases from 7.x.x to 9.x.x

I am trying to setup Autobahn test suite(http://autobahn.ws/testsuite/installation.html#installation) on my Windows and Ubuntu linux machine but getting following error when I try to get available options, wstest --help
C:\Python27\Scripts>wstest.exe --help
Traceback (most recent call last):
File "C:\Python27\Scripts\wstest-script.py", line 9, in
load_entry_point('autobahntestsuite==0.6.1', 'console_scripts', 'wstest')()
File "C:\Python27\lib\site-packages\pkg_resources.py", line 339, in load_entry
point
return get_distribution(dist).load_entry_point(group, name)
File "C:\Python27\lib\site-packages\pkg_resources.py", line 2470, in load_entr
y_point
return ep.load()
File "C:\Python27\lib\site-packages\pkg_resources.py", line 2184, in load
['name'])
File "C:\Python27\lib\site-packages\autobahntestsuite-0.6.1-py2.7.egg\autobahn
testsuite__init_.py", line 23, in
import wstest
File "C:\Python27\lib\site-packages\autobahntestsuite-0.6.1-py2.7.egg\autobahn
testsuite\wstest.py", line 40, in
import wamptestee
File "C:\Python27\lib\site-packages\autobahntestsuite-0.6.1-py2.7.egg\autobahn
testsuite\wamptestee.py", line 19, in
from autobahn.wamp1 import protocol as wamp
ImportError: No module named wamp1

Following are the instructions, I have followed:

1. Installed Python 2.7
2. Installed Twisted
3. Get latest package from git
   git clone git://github.com/tavendo/AutobahnTestSuite.git
   cd AutobahnTestSuite
   git checkout v0.6.1
   cd autobahntestsuite
   python setup.py install
4. On Command shell, using C:\Python27\Scripts>wstest.exe --help to see available options.

Also, I have installed WAMP server(Windows) and LAMP server(Linux) but still getting same error for both the platforms.

Please let me know if I am missing some steps during installation.

Thanks.

Recent report here: http://stackoverflow.com/questions/34157314/autobahn-websocket-issue-while-running-with-twistd-using-tac-file

I met a situation with test 1.1.6 that he can fail or pass completely dependent on if I force the mask nonce to 0x00000000 or not.

If I leave the mask nonce as random, he fails me with this in the return code

        0x0030:  ce14 4342 8855 03ef 656e 636f 756e 7465  ..CB.U..encounte
        0x0040:  7265 6420 696e 7661 6c69 6420 5554 462d  red.invalid.UTF-
        0x0050:  3820 7768 696c 6520 7072 6f63 6573 7369  8.while.processi
        0x0060:  6e67 2074 6578 7420 6d65 7373 6167 6520  ng.text.message.
        0x0070:  6174 2070 6179 6c6f 6164 206f 6374 6574  at.payload.octet
        0x0080:  2069 6e64 6578 2034 3039 37              .index.4097

I leave the nonce processing code alone on my side, just force it to zeros; he still goes through the steps the same.

I think there is no legitimate way to create this situation on my side.

I can see when the mask is random, the masked data content I am transmitting is correct, ie, all '*' XOR-masked.

Should be autobahn.wamp.test.

Migrate the WS testing machinery to the SQLite based storage that the new WAMP test machinery already uses.

After investigation I finally understand how to make "pass" 6.4.2 the way it's implemented in this test.

By itself the first frame cebae1bdb9cf83cebcceb5f4 are invalid / incomplet / badly cut UTF8 message. Because it's the first message ( opcode 1 ) and not the last (!fin). I wait for next message to confirm that the whole message concatenate from frame1 + frame2 are again not an UTF8. Now I can fail fast and pass the test.

Why first frame are accepted in testsuite?

For 6.4.3 and 6.4.4 could failed for the wrong reason because it's seem it didn't respect something else.

An unfragmented message consists of a single frame with the FIN
      bit set (Section 5.2) and an opcode other than 0.

Should those two test use opcode 1 instead of 0 ?

sudo easy_install autobahntestsuite
Searching for autobahntestsuite
Best match: autobahntestsuite 0.6.0
Processing autobahntestsuite-0.6.0-py2.7.egg
autobahntestsuite 0.6.0 is already the active version in easy-install.pth
Installing wstest script to /usr/local/bin

Using /Library/Python/2.7/site-packages/autobahntestsuite-0.6.0-py2.7.egg
Processing dependencies for autobahntestsuite
Searching for autobahn[twisted]>=0.8.1
Reading http://pypi.python.org/simple/autobahn/
Best match: autobahn 0.8.2
Downloading https://pypi.python.org/packages/source/a/autobahn/autobahn-0.8.2.zip#md5=a7cff61550397dc6d5e8da62f477304e
Processing autobahn-0.8.2.zip
Running autobahn-0.8.2/setup.py -q bdist_egg --dist-dir /tmp/easy_install-xEl1R2/autobahn-0.8.2/egg-dist-tmp-nQC1sP

The required version of setuptools (>=2.1) is not available,
and can't be installed while this script is running. Please
install a more recent version first, using
'easy_install -U setuptools'.

(Currently using setuptools 0.6c12dev-r88846 (/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python))
error: Setup script exited with 2

I'm attempting to run the fuzzingclient with the following config:

{
"options": {"failByDrop": false},
"outdir": "/var/www/reports",

"servers": [
{"agent": "AutobahnServer",
"url": "ws://localhost:8123",
"options": {"version": 18}}
],

"cases": ["*"],
"exclude-cases": [],
"exclude-agent-cases": {}
}

I always get back the following error on the first test case:

File "/usr/local/lib/python2.7/dist-packages/autobahntestsuite-0.5.1-py2.7.egg/autobahntestsuite/case/case.py", line 93, in onConnectionLost
if self.expectedClose["closedByMe"] != self.p.closedByMe:
exceptions.KeyError: 'closedByMe'

moved from: crossbario/autobahn-python#249

It appears that many implementations have Section 7.2.3.6 of permessage-deflate wrong.
https://tools.ietf.org/html/rfc7692#section-7.2.3.6

If two messages (from server to client) is sent like the following ...

TEXT ("Hello") / rsv1 / fin==false
41 0b f2 48 cd c9 c9 07 00 00 00 ff ff
CONTINUATION (1 byte following Section 7.2.3.6) / rsv1 / fin==true
c0 01 00
TEXT ("World") / rsv1 / fin==false
41 0b 0a cf 2f ca 49 01 00 00 00 ff ff
CONTINUATION (1 byte following Section 7.2.3.6) / rsv1 / fin==true
c0 01 00

Most browser will fail to parse/decode these 4 frames into 2 messages per the rules in Section 7.2.3.6
Some 3rd party websocket libraries will also fail.

Test case 7.1.6 returns informational:

Case outcome depends on implimentation defined close behavior. Message and close frame are sent back to back. If the close frame is processed before the text message write is complete (as can happen in asyncronous processing models) the close frame is processed first and the text message may not be recieved or may only be partially recieved.

(btw, quite some typos in this text)
The same thing also applies for these cases:
3.2, 4.1.3, 4.1.4, 4.2.3, 4.2.4, 4.2.5, 5.15 (maybe there are more)
But they return non-strict.

In these cases, the fuzzer sends a close frame (7.1.6) or an in valid frame directly after a valid message. My server runs in JavaScript and thus is asynchronous. When receiving a text message, the binary data is converted to a string in an async way (using FileReader), but the part that validates frames/processs close frames doesn't have callbacks. So the connection is already closed when trying to send the message back.

Case Outcome

Actual events differ from any expected.

Expected:
{'NON-STRICT': [('timeout', 'A'), ('timeout', 'B')], 'OK': [('timeout', 'A')]}

Observed:
[('timeout', 'A'), ('timeout', 'B')]

For some reason the event comparison code fails on my machine (OS X, Python 2.7.2): https://github.com/tavendo/AutobahnTestSuite/blob/13eb4de887c3dff481b7f2b8d0087773f042cee2/autobahntestsuite/autobahntestsuite/case/case.py#L82

The README.md file talks about checking out the v0.5.2 branch. It gives a strong instruction not to use master branch. However, there is only one branch on GitHub (which is master). Does the "Do NOT use current HEAD" warning still apply?

We're getting rid of the Autobahn.ws site. JS docs should live on GitHub. Adapt what we have to work here.

Parts of this are:

• replace macros with the end product (e.g. |ab| --> Autobahn|Testsuite)
• get the links working
• look into a rst --> markdown conversion

I upgraded autobahntestsuite to the latest version available on PyPI and I'm getting this exception:

% wstest --help
Traceback (most recent call last):
  File "/Users/myk/.virtualenvs/autobahn/bin/wstest", line 9, in <module>
    load_entry_point('autobahntestsuite==0.5.7', 'console_scripts', 'wstest')()
  File "/Users/myk/.virtualenvs/autobahn/lib/python2.7/site-packages/pkg_resources.py", line 353, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/Users/myk/.virtualenvs/autobahn/lib/python2.7/site-packages/pkg_resources.py", line 2321, in load_entry_point
    return ep.load()
  File "/Users/myk/.virtualenvs/autobahn/lib/python2.7/site-packages/pkg_resources.py", line 2048, in load
    entry = __import__(self.module_name, globals(),globals(), ['__name__'])
  File "/Users/myk/.virtualenvs/autobahn/lib/python2.7/site-packages/autobahntestsuite/__init__.py", line 23, in <module>
    import wstest
  File "/Users/myk/.virtualenvs/autobahn/lib/python2.7/site-packages/autobahntestsuite/wstest.py", line 40, in <module>
    import wamptestee
  File "/Users/myk/.virtualenvs/autobahn/lib/python2.7/site-packages/autobahntestsuite/wamptestee.py", line 93, in <module>
    class MyTopicService:
  File "/Users/myk/.virtualenvs/autobahn/lib/python2.7/site-packages/autobahntestsuite/wamptestee.py", line 100, in MyTopicService
    @wamp.exportSub("foobar", True)
AttributeError: 'module' object has no attribute 'exportSub'

This is reproducible in a fresh virtualenv:

mkvirtualenv autobahn
pip install --upgrade setuptools
pip install autobahntestsuite
wstest --help

I'm running your test suite against ws4py and I have a few test which are marked as failed because autobahn expects a connection to be immediately failed. Unfortunately it's unclear as to what this means.

The spec distinguishes between Failing and Aborting the connection. The former respects the closing handshake but is simply initiated due to an error whereas the later doesn't run the closing handshake at all but close the underlying connection directly.

I currently do not understand what AutobahnTestsuite expects here. For instance, case 6.6.1 requires the connection to be failed immediately because of an invalid UTF-8 payload. The spec says that in case of invalid UTF-8 the connection should be failed, not aborted.

http://tools.ietf.org/html/rfc6455#section-8.1

So what is expected from the testsuite?

See some results at http://www.defuze.org/oss/ws4py/testreports/servers/0.2.1/

Is there a reason why the testsuite didn't have origin header in handshake?

My server enforce origin to be present in the header, take me some time to figure that is what prevent autobahn to test my websocket server.

I have a hard time trying to install the testsuite. I follow the instructions here. I however get this error:

/tmp/AutobahnTestSuite/autobahntestsuite$ wstest --test
..
  File "/usr/local/lib/python2.7/dist-packages/autobahntestsuite-0.6.1-py2.7.egg/autobahntestsuite/wamptestee.py", line 19, in <module>
    from autobahn.wamp1 import protocol as wamp
ImportError: No module named wamp1

Is there a step missing from the instructions? I tried both with pip install and manually with setup.py but I get the same errors.

Description of case: Send 1000 compressed messages each of payload size 4096, auto-fragment to 0 octets. Use default permessage-deflate offer.

The very first frame send on this testcase sends 300 bytes of payload.
Its a text frame, with fin=true.

The server receives the 300 bytes, and inflates it to 4096 bytes of text.
The sha1 checksum for this 4096 bytes is 66976578abf9a7f1fc653c2bf9dcd40a9a8a19e2

However, the testcase assumes the sha1 checksum is 0c128dcadfd58d297bdb0e65946c1837c6acf9af

A raw capture of the autobahn sent frame in wireshark and a manual inflate of those raw bytes show the same sha1sum as the implementation as I'm testing.

I can provide the wireshark captured frame payload (300 bytes deflated), and the resulting inflated frame payload (4096 bytes) as examples of this result.

I send char 181 in my response header which is decoded properly in Edge, Firefox, Chrome and seems to be historically supported. Autobahn crashes due to failed Utf-8 parsing. The header is not supposed to be decoded as Utf-8, and it is perfectly legal to send char 181.

Actually, we should distinguish the following behaviors:

fail-per-message
fail-per-frame
fail-per-buffered
fail-per-first-octet (aka fail-fast)

"""
probably this is something which would be good reason to make that case section "informational" .. because the probably most sensible beh. (fail-per-buffered) is not canonical/well-defined anyway. mmh

it could be tested: start with N*valid char + invalid .. gradually increase N from 0 to 128k .. that should cover up most fail-per-buffered

actually: 6.4.3 should be extended so that it does that increasing N test. it would then report N*, the first N at which the peer detected invalid utf8.

for fail-fast: N* would be 0
for fail-per-buffer: N* would be the impl. buffer size

and it would fail if N* > N .. that is the impl. only does per-frame/per-msg val.
"""

Several of the test cases crash when run against a dummy server that just immediately closes the connection, such as:

import socket

s = socket.socket()
s.bind(('', 9001))
s.listen(1000)
while 1:
    csock, cinfo = s.accept()
    csock.close()

It would be better if these tests all failed (e.g. it would make test-driven development of servers easier).

(This was tested with "wstest -m fuzzingclient -s fuzzingclient.json" with fuzzingclient.json copied from the wstest docs).

systematic method of returning "values" to summary report from informational cases

Hello, I've successfully installed AuthobahnTestSuite but there's an error while running wstest:

Traceback (most recent call last):
  File "/usr/local/bin/wstest", line 9, in <module>
    load_entry_point('autobahntestsuite==0.6.1', 'console_scripts', 'wstest')()
  File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 356, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 2439, in load_entry_point
    return ep.load()
  File "build/bdist.linux-x86_64/egg/pkg_resources.py", line 2155, in load
    ['__name__'])
  File "/usr/lib/python2.6/site-packages/autobahntestsuite/__init__.py", line 23, in <module>
    import wstest
  File "/usr/lib/python2.6/site-packages/autobahntestsuite/wstest.py", line 37, in <module>
    import fuzzing
  File "/usr/lib/python2.6/site-packages/autobahntestsuite/fuzzing.py", line 40, in <module>
    from case import Case, \
  File "/usr/lib/python2.6/site-packages/autobahntestsuite/case/__init__.py", line 258, in <module>
    from case12_x_x import *
  File "/usr/lib/python2.6/site-packages/autobahntestsuite/case/case12_x_x.py", line 347, in <module>
    co_desc = "client offers (requestNoContextTakeover, requestMaxWindowBits): {}".format([(x.requestNoContextTakeover, x.requestMaxWindowBits) for x in co])
ValueError: zero length field name in format

I'm using python 2.6.6

No downloads available for autobahntestsuite 0.6.2.

wstest command was giving the following error:

Traceback (most recent call last):
File "/usr/local/bin/wstest", line 5, in
from pkg_resources import load_entry_point
File "/Library/Python/2.7/site-packages/setuptools-2.2-py2.7.egg/pkg_resources.py", line 2716, in

File "/Library/Python/2.7/site-packages/setuptools-2.2-py2.7.egg/pkg_resources.py", line 685, in require
def getstate(self):
File "/Library/Python/2.7/site-packages/setuptools-2.2-py2.7.egg/pkg_resources.py", line 588, in resolve
The plugin_env should be an Environment instance that contains
pkg_resources.DistributionNotFound: werkzeug

So I googled and then installed werkzeug module. Then it gave another error and I had to install "markupsafe" module. Essentially the following commands were required in order to get wstest working.

sudo /usr/local/bin/easy_install werkzeug
sudo /usr/local/bin/easy_install markupsafe

Is there a way to specify dependency on these modules so that the overall experience is simpler ?

Allowing to export test results from test DB 1 and import those into test DB 2 would allow us to build a centralized/consolidated DB of a large number of testees.

Exposing those on a Website (together with WS impl. features) would then finally allow users to go to 1 source of information rgd WS impl.

Add option to produce reports in JSON format

I'd like to propose that the Test cases 6.4.3 and 6.4.4 be flagged with informational content similar to how case 7.1.6 is labeled.

Testing my own WebSocket implementation in fuzzing client mode I noticed "Fail" on Case 7.5.1 - I assumed that closing frame body if it's present can be ANYTHING and MAY be UTF-8 encoded text (but can also be just regular binary blob - application decides what's expected).
I looked through RFC 6455 and I'm puzzled.

Section 1.2 specified only text frames as UTF-8 encoded, binary frames interpretation are left to applications, there's no information about interpretation of control frames payloads.

(...)there are
types for textual data (which is interpreted as UTF-8 [RFC3629]
text), binary data (whose interpretation is left up to the
application), and control frames (which are not intended to carry
data for the application but instead for protocol-level signaling,
such as to signal that the connection should be closed).

Section 5.5 clearly defines close frame as control frame:

Currently defined opcodes for control frames
include 0x8 (Close),(...)

Section 5.5.1. states close frames payload after 2B integer, body may or may not be present. Body itself may be UTF-8 encoded but it's interpretation is not defined.

Following the 2-byte integer, the body MAY contain UTF-8-encoded data
with value /reason/, the interpretation of which is not defined by
this specification.

In addition the same section warns later on close frame payload cannot be considered human-readable:

data is not guaranteed to
be human readable, clients MUST NOT show it to end users.

In my opinion close frames payload could contain UTF-8 text but it's not mandatory - it could also contain binary data.
Maybe my misunderstanding came from fact that I'm not native english speaker?

I am using the following config:

{
"options": {"failByDrop": false},
"outdir": "./reports/servers",

"servers": [
{"agent": "Autobahn Python 0.5.0",
"url": "ws://127.0.0.1:9003",
"options": {"version": 13}},

            {"agent": "CherryPy 3.2.2",
             "url": "ws://127.0.0.1:9000",
             "options": {"version": 13}}

          ],

"cases": ["*"],
"exclude-cases": [],
"exclude-agent-cases": {}
}

I run the following command:
$ wstest -m testeeserver -w ws://localhost:9003

I also start the CherryPy server:

Then:
$ wstest -m fuzzingclient -s fuzzingclient.json

Once the suite finishes, against the Autobahn server, the test 10.1.1, I get::

.venvs/default/local/lib/python2.7/site-packages/autobahntestsuite-0.5.0-py2.7.egg/autobahntestsuite/fuzzing.py", line 1140, in clientConnectionLost
connectWS(self, contextFactory = self.contextFactory)
exceptions.AttributeError: FuzzingClientFactory instance has no attribute 'contextFactory'

It happens if I'm using CherryOy as the first server as well. However when I'm using a single server (CherryPy or Autobahn) in the spec, things terminates properly.

It seems it breaks whenever you have more than one server in the spec.

Ideally, 1 line per WAMP message (each line is a serialized JSON with a WAMP msg)

TestSuite currently fails my library because it does not reject the following codes:

• 1004
• 1012
• 1013
• 1014
• 1016
• 1100
• 2000
• 2999

The rationale, I assume, is that anything reserved and not defined by RFC 6455 is considered "invalid". This puts me in an interesting position, though: if my library hardcodes these undefined opcodes to reject them, no one will be able to use it to design new extensions that use new opcodes.

I could always have a "developer mode" for the library that lets undefined opcodes through, and switch it off when running TestSuite. But that's further muddied by the fact that codes 1012 and 1013 are, in fact, defined as of May 2012.

Does Autobahn have a policy for evolving with new opcode definitions, and/or has there been any discussion on allowing a more relaxed test mode?

I have a question about the semantics of tests in the fuzzing client where one or more valid frames are sent prior to an invalid frame. This case is not a bug report or feature request at this point, but a request for clarification while using Autobahn TestSuite to write a server.

Test 3.2 is a good example (https://github.com/tavendo/AutobahnTestSuite/blob/master/autobahntestsuite/autobahntestsuite/case/case3_2.py#L32-33) where the expectation is Echo for first message is received, but then connection is failed immediately

A server written as a single-threaded RECV/SEND/RECV/SEND... would enforce FIFO request/reply sequence, and ought to pass this test. The client is not dictating the sequence, but rather the semantics programmed into the behavior of the server.

However, consider a server with async request handling within each connection -- the natural sequencing no longer exists. If a client sends a valid frame immediately followed by an invalid frame, it's a race condition whether or not a response to the first frame is received prior to the server failing the handshake.

Thoughts on this? Is a WebSocket server expected to handle all previous traffic prior to failing the connection on an invalid frame? Thanks!

And, highest praise on this TestSuite. The Tx/Rx wire log is brilliant.

WAMP testing won't be in AutobahnTestsuite for various reasons.

the following command:

wstest -m echoserver -w ws://localhost:9000

also opens a server on 8080. If there is already a server on 8080 this fails with an ugly error that boils down to [Errno 98] Address already in use. Is there a way to use the wstest echo server on a machine where 8080 is not available?

Hey,

I cannot find any info on how to get the fuzzingserver working in SSL mode. I just get this error:

Secure WebSocket listen requested, but no SSL context factory given

The usage documentation is pretty nice for non-SSL but is IMO lacking for SSL. I do pass key and cert as arguments (-k, -c) and I changed the url in the spec to wss.

When hosting generated test reports on a secure site (HTTPS), browsers will complain about "unsecure mixed content", since the image links point to the (currently) unsecured (HTTP) AutobahnTestsuite website.

We might just copy the images

• ws_protocol_test_report.png
• ws_protocol_test_report_autobahn.png

to the generated reports, and use local links like src="ws_protocol_test_report.png".

In those tests, the testsuite client sends a close frame while in the middle of another frame. A server cannot possibly read this close frame properly (because it cannot be expecting it). It isn't really relevant to the test semantics because the client sent invalid UTF-8 data before so what the test checks is that the server will timeout the connection after a while.

As the test case description doesn't mention this behavior it is confusing when trying to fix an implementation that fails at this test case.

Crash happens when targeting fuzzingclient at a server that incorrectly redirects ws://127.0.0.1:8080 to https://127.0.0.1:8443/.
System is macOS Sierra.

{
   "outdir": "./reports/servers",
   "servers": [
      {
         "url": "ws://127.0.0.1:8080"
      }
   ],
   "cases": ["*"],
   "exclude-cases": [],
   "exclude-agent-cases": {}
}

Servers = [u'ws://127.0.0.1:8080']
Unhandled Error
Traceback (most recent call last):
File "~/wstest/lib/python2.7/site-packages/autobahntestsuite/wstest.py", line 338, in run
start(options, spec)
File "~/wstest/lib/python2.7/site-packages/autobahntestsuite/wstest.py", line 272, in start
reactor.run()
File "~/wstest/lib/python2.7/site-packages/twisted/internet/base.py", line 1195, in run
self.mainLoop()
File "~/wstest/lib/python2.7/site-packages/twisted/internet/base.py", line 1204, in mainLoop
self.runUntilCurrent()
--- <exception caught here> ---
File "~/wstest/lib/python2.7/site-packages/twisted/internet/base.py", line 825, in runUntilCurrent
call.func(*call.args, **call.kw)
File "~/wstest/lib/python2.7/site-packages/twisted/internet/tcp.py", line 479, in connectionLost
self.connector.connectionLost(reason)
File "~/wstest/lib/python2.7/site-packages/twisted/internet/base.py", line 1088, in connectionLost
self.factory.clientConnectionLost(self, reason)
File "~/wstest/lib/python2.7/site-packages/autobahntestsuite/fuzzing.py", line 1292, in clientConnectionLost
self.createReports()
File "~/wstest/lib/python2.7/site-packages/autobahntestsuite/fuzzing.py", line 469, in createReports
self.createMasterReportHTML(self.outdir)
File "~/wstest/lib/python2.7/site-packages/autobahntestsuite/fuzzing.py", line 670, in createMasterReportHTML
agent_case_report_file = self.makeAgentCaseReportFilename(agentId, caseId, ext = 'html')
File "~/wstest/lib/python2.7/site-packages/autobahntestsuite/fuzzing.py", line 498, in makeAgentCaseReportFilename
return self.cleanForFilename(agentId) + "_case_" + c + "." + ext
File "~/wstest/lib/python2.7/site-packages/autobahntestsuite/fuzzing.py", line 487, in cleanForFilename
s0 = ''.join([c if c in "abcdefghjiklmnopqrstuvwxyz0123456789" else " " for c in str.strip().lower()])
exceptions.AttributeError: 'NoneType' object has no attribute 'strip'