Apologies if I'm missing something...

Prism will choke on the below with "Invalid Scenario format json: cannot unmarshal array into Go struct field .scenarios of type map[string]json.RawMessage.". If you change the scenario id from "100" to "foo", it will work.

{
  "scenarioVersion": "1.1",
  "name": "test collection",
  "before": {},
  "scenarios": {
    "100": {
      "name": "test",
      "steps": [
        {
          "id": "ryxe",
          "type": "http",
          "name": "",
          "input": {
            "method": "get",
            "url": "https://api.github.com/gists"
          }
        }
      ]
    }
  },
  "after": {},
  "utilities": {}
}

Lockdown initial runtime API

• $$.env

• $$.ctx #49

  • transforms
  • assertions
  • variables

• $$.request (same stuff has http input below)

• $$.response

• $.ctx

• input.query (http)

• input.cookies (http) #21

• input.host (http)

• input.path (http)

• input.scheme (http)

• SL.sleep()

• SL.spec.findOperation() - https://scenarios.stoplight.io/stoplight/specs/hubs/edit#/definitions/HTTP_Operation

• SL.schema.validate() validates a schema

• SL.schema.generateExample() generates a json example

JS runtime libraries

• lodash
• json schema faker
  • fix date format

Version: 2.0.0-alpha.3
Context: Conductor

Needs verification.

Scenario step sends HTTP request, with response status 204 (no content) after a DELETE operation.

The body shown in the UI is “undefined” whereas my API writes nothing at all on output. Therefore, the contract testing always fails.

Found a workaround in declaring an output model as “string” and “optional” but it’s not true.

I'm currently using prism to mock an api endpoint where one of the query parameters is called page[limit], with a type of integer. The mock api works fine if I omit this query param, but when I make a request with the query string ?page%5Blimit%5D=100, it throws this error:

{
  "errors": [
    {
      "context": "request.url.queryString",
      "description": "Invalid type. Expected: [array,integer], given: string",
      "field": "page[limit]",
      "location": "page[limit]",
      "type": "invalid_type"
    }
  ]
}

This error isn't thrown if I change the name of the param to limit, or if I change the type to string.

Am I doing something wrong here, or is this a bug?

Right now, if a scenario has 3 steps, and step 2 has a request body with invalid JSON, we fail out THE ENTIRE scenario without any sort of useful error, and no data returned for the UI.

Instead, we should probably:

1. add a log entry to the step with an error description ("Request body JSON is invalid", or something similar)
2. do not continue running the scenario
3. when the run is complete, return the result

Other than installing in mac or windows, I want to have the same experiencing under linux platform. And docker is the best option.

30 seconds? This will help to guard against infinite loops, etc.

Should be configurable.

Hi, I am using prism on Amazon Ec2 instance with following OS :
Distributor ID: Ubuntu Description: Ubuntu 16.04.1 LTS Release: 16.04 Codename: xenial

After few requests, as i checked its memory usage went upto 2GB's.

Right now the default request timeout is 30 seconds, and there is no way for the user to control.

In prism, while running scenarios with following example
{$.ctx.host}/users/{$.ctx.user}/profile, it does not find the api specification /users/{id}/profile.

If substituted with a actual data like {$.ctx.host}/users/1/profile it will pick up the api specification

• Should be possible to turn off automatic contract testing, we could still run it but it might just be a warning instead of an error.

302 response codes are often used to indicate browser redirection as part of oauth flows. Prism should support automatically using the Location response header to create a new request and "follow the redirect", as a browser would.

In some situations, it is not desirable to have all scenarios in a collection run concurrently.

Simplest configuration would be sequential versus parallel (what we have now).

Should support file uploads in multipart/form-data requests.

Add "setup" and "teardown" properties to collections.

{
    "setup": {...scenario}, // run at start collection
    "scenarios": [], // run concurrently, after setup
    "utilities": [], // not run with collection
    "teardown": {...scenario} // run after scenarios are all completed
}

• add to prism
• update docs
• update json schema / model in docs

• Print list of failed tests at the end so you can do a quick find in the terminal.
• Filter Results

We have a Rails app with prism as part of its test suite. The alphas run fine on images based on ruby:2.4-slim, but when I tried running one on an image based on alpine:3.6, I got No such file or directory errors. When I run ldd on the binary, I get this:

bash-4.3# ldd /usr/bin/prism
    /lib64/ld-linux-x86-64.so.2 (0x5588243ac000)
    libm.so.6 => /lib64/ld-linux-x86-64.so.2 (0x5588243ac000)
    libpthread.so.0 => /lib64/ld-linux-x86-64.so.2 (0x5588243ac000)
    libc.so.6 => /lib64/ld-linux-x86-64.so.2 (0x5588243ac000)
Error relocating /usr/bin/prism: __isnan: symbol not found
Error relocating /usr/bin/prism: __finite: symbol not found

In this page from the Docker alpine docs, it says that because Alpine uses musl libc instead of glibc, the binary would need to be compiled natively for musl in order to work. Is this something you could do? Or is there any other way we could run it in our alpine containers?

Should have option to log results output to file when running from terminal.

FAST

The swagger files contains the host as "www.abc.com", when running prism test with command "prism conduct scenario.json -s swagger.json --env host="localhost".

The api specification is not found.

• https://stoplight.io/prism

Additional runtime methods:

• $$.done() - stops collection run

• $.retry() - stops and retries scenario

• $.done() - stops scenario

• $.nextStep(stepId) - moves to next step, or specified id

• $.pushStep() - adds step to end of scenario

• $.retryStep() - stops and retries the current step

• SL.json2xml()

• SL.xml2json()

• fetch - need sync and async

Add back cookie helper api, similar to the previous version of prism.

Input: https://help.stoplight.io/prism/runtime-reference/api#requestcookieall-namestring-valuestring-pathstring-domainstring-expirestime-securebool-httponlybool

Output: https://help.stoplight.io/prism/runtime-reference/api#responsecookieall-namestring-valuestring-pathstring-domainstring-expirestime-securebool-httponlybool

This is a placeholder issue, more details coming soon.

Currently cleaning up everything and adding some tests.

With this YAML

swagger: '2.0'
info:
  version: '0.1'
  title: test
host: localhost:4010
basePath: /
schemes:
  - http
consumes:
  - text/plain
produces:
  - text/plain
paths:
  '/endpoint':
    get:
      parameters:
        - name: MyApp-MyHeader
          type: string
          in: header
          required: true
      responses:
        '200':
          description: ''

imported to and making request with Postman, Prism tells me (Wireshark stream dump):

GET /endpoint HTTP/1.1
cache-control: no-cache
Postman-Token: 90481f7f-64e7-4df4-9094-1851542c828a
MyApp-MyHeader: efuqhwfieuhqweiufhqwuiefh
User-Agent: PostmanRuntime/3.0.11-hotfix.2
Accept: */*
Host: localhost:4010
accept-encoding: gzip, deflate
Connection: keep-alive

HTTP/1.1 400 Bad Request
Content-Type: application/json
Prism-Request-Messages: [{"messageType":"error","text":"{\n  \"context\": \"request.headers\",\n  \"description\": \"MyApp-MyHeader is required\",\n  \"field\": \"MyApp-MyHeader\",\n  \"location\": \"(root)\",\n  \"type\": \"required\"\n}","createdAt":"2017-03-22T17:46:54.563860678Z"}]
Prism-Request-Validation: Failed
Prism-Response-Hijacked: true
Prism-Response-Mocked: true
Date: Wed, 22 Mar 2017 17:46:54 GMT
Content-Length: 206

{
  "errors": [
    {
      "context": "request.headers",
      "description": "MyApp-MyHeader is required",
      "field": "MyApp-MyHeader",
      "location": "(root)",
      "type": "required"
    }
  ]
}

Any idea what is trying to tell me and what to do about it? Prism console does not say anything useful even with debug parameter.

Mar 22 17:46:25.972 INFO: [ROUTER] {2} Rewriting request host to (localhost:4010)
Mar 22 17:46:26.398 INFO: [VM] Endpoint found (58d2b87277840441750ec7ff, /endpoint, GET, /endpoint)
Mar 22 17:46:26.398 INFO: [VM] Endpoint found (58d2b87277840441750ec7ff, /endpoint, GET, /endpoint)

Well maybe except the fact that it "finds" the endpoint twice for a single request.

Interesting side observation: if i don't use --mock parameter, Prism ends up in endless cycle on very first request:

Mar 22 17:48:14.523 INFO: [ROUTER] {2} Rewriting request host to (localhost:4010)
Mar 22 17:48:14.955 INFO: [VM] Endpoint found (58d2b8de77840441bab52cb4, /endpoint, GET, /endpoint)
Mar 22 17:48:14.955 INFO: [VM] Endpoint found (58d2b8de77840441bab52cb4, /endpoint, GET, /endpoint)
Mar 22 17:48:14.959 INFO: [ROUTER] {4} Rewriting request host to (localhost:4010)
Mar 22 17:48:15.333 INFO: [VM] Endpoint found (58d2b8de77840441bab52cb4, /endpoint, GET, /endpoint)
Mar 22 17:48:15.333 INFO: [VM] Endpoint found (58d2b8de77840441bab52cb4, /endpoint, GET, /endpoint)
Mar 22 17:48:15.335 INFO: [ROUTER] {6} Rewriting request host to (localhost:4010)
Mar 22 17:48:15.735 INFO: [VM] Endpoint found (58d2b8de77840441bab52cb4, /endpoint, GET, /endpoint)
Mar 22 17:48:15.736 INFO: [VM] Endpoint found (58d2b8de77840441bab52cb4, /endpoint, GET, /endpoint)
....

• add https://godoc.org/github.com/codahale/http-handlers/recovery
• log the stack trace to the console
• return a 500 response, and set a helpful message on the body.

Just an FYI: The new version looks like it has some cool features, but the documentation on stoplight.io is still referencing the previous version. I was able to work around this by modifying the local version of the install script, but I'm using the previous version. Would be nice to have documentation on the new version as well.

I've been using this snippet to get it working with Node on Windows:

prism.js

var exec = require('child_process').exec;

// Get the swagger json/yaml file passed in as an arg
var swaggerEndPoint = process.argv[2];

if(!swaggerEndPoint){
    console.error("No swagger endpoint given");
    return;
}

var prism = exec('prism_windows_386.exe run --mock --list --spec ' + swaggerEndPoint);  

prism.stdout.on('data', function(data) {
    console.log(data); 
});

and they use it with node like: C:\>node prism http://petstore.swagger.io/v2/swagger.json

Which works fine but I obviously need prism_windows_386.exe checked into source control. It would make this infinitely more usable if prism could just be installed via npm and used like a regular module.

• Using equal operation, assert null is broken, fails because undefined isn't equal to null.

Everything works on localhost just fine, we have deployed Prism on our server and when we access it from local network everything works just fine as well. But when I try to access it (same request) from outside network, the Prism will get (and print into console) the request, but does not send any response. Only later it will send 503 (see attached screenshot). Our network seems to be configured properly, do you have any idea where could be the problem?

--debug flag does not print anything into the console

CentOS 7 /usr/local/bin/prism SIGSEGV

Note: It works on macOS Sierra.

$ uname -r
3.10.0-327.el7.x86_64

$ cat /etc/centos-release
CentOS Linux release 7.2.1511 (Core)

Prism segmentation fault - Debug information

# prism
Segmentation fault

$ gdb prism
GNU gdb (GDB) Red Hat Enterprise Linux 7.6.1-80.el7
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-redhat-linux-gnu".
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>...
"/usr/local/bin/prism": not in executable format: File format not recognized
(gdb)

# strace /usr/local/bin/prism
execve("/usr/local/bin/prism", ["/usr/local/bin/prism"], [/* 30 vars */]) = -1 EFAULT (Bad address)
--- SIGSEGV {si_signo=SIGSEGV, si_code=SI_USER, si_pid=16211, si_uid=0} ---
+++ killed by SIGSEGV +++
Segmentation fault (core dumped)

$ file /usr/local/bin/prism
/usr/local/bin/prism: ERROR: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), dynamically linked (uses shared libs)error reading (Invalid argument)

$ hexdump -n 16 /usr/local/bin/prism
0000000 457f 464c 0102 0001 0000 0000 0000 0000
0000010

This would delay output until the entire collection is done, but would ensure consistent output that is in order.

Hello,

Thanks you guys for creating this. It's very helpful for mocking and creating apis.

Do you think it would be possible to add a command line arg, to define how much data is sent by the api ? This would be very helpful when we need some endpoint to return more than just one to 3 lines...

thanks in advance.

When running prism conduct scenario.json where scenario.json doesn't exist yields no output by prism. I would expect some error indicating that the file doesn't exist.

Likewise where scenario.json exists but is not valid, there's no output and I would expect an error indicating a problem with the file.

Some parsing libraries can't handle the generated timestamps, and they aren't very useful in unit tests. Lets just generate the timestamps +/- days from the current time, not years.

After internal testing overhaul, but before open sourcing:

Methods

• Synthetic benchmarks with go
• Realworld benchmarks with aws + some load testing product (blitz, loader?)

Examples

• Kong: https://getkong.org/about/benchmark/
• Tyk: https://tyk.io/tyk-api-gateway-benchmarks/

• Need to add support for binary format, causes validation to fail.

This will persist the ctx during a collection run. This will allow you to persist data in before scenarios.

If a response is defined with no schema, we should validate that the body is empty

http://swagger.io/specification/#responseObject

Umbrella task for prism helpers, like default mocking scenario.

• should work the same as prism v1, but needs to be built with a scenario.
  • for example, if there is no example defined generate a mocked response.

Should update this scenario: https://scenarios.stoplight.io/stoplight/collections/prism-helpers#/scenarios/mock/steps/0

needs #44