azolo / websockex Goto Github PK

It doesn't seem like it will be too hard.

send_frame/2 is handled with :gen.call, which just means I need to write it for cast/2 plus name registration.

Add the ability to pass the :name option and register as a named process.

Would be nice and match the functionality of other modules.

Reported via elixirforum WebSockex thread

Saša Jurić made some really good points about backing the process with a GenServer instead of implementing my own special process.

It will take some looking at and a different approach, but it may be a good idea to consider prior to a v1.0.0 stable release.

I don't know why, but the callback isn't being called consistently in tests.

First of all thanks for the library, it works great from phoenix as a client!

It's probably a newbie question that I'm missing something in how to use the library.

But I have this code:

def handle_frame({:text, "#" <> pingNum}, state) do
    Logger.info("Coinigy Ping ##{pingNum} Received")

    WebSockex.send_frame(self(), {:text, "##{String.to_integer(pingNum) + 1}"})

    {:ok, state}
  end

And I get timeout because it looks like the send_frame is not being sent to the server... This is what I receive on my terminate function:

{:timeout, {WebSockex, :call, [#PID<0.336.0>, {:text, "#2"}]}}

So my workaround is to use one of my servers (GenServer) to listen to whatever message is being sent or received from my WebSockex. Is there any better approach for that? Why can't I call send_frame from a handle_frame?

I'm trying to connect to the GDAX Websocket API, whose sandbox URL is "wss://ws-feed-public.sandbox.gdax.com"

Calling WebSockex.start_link with that URL terminates immediately with {:error, %WebSockex.ConnError{original: :closed}}

Connecting to the echo server at "wss://echo.websocket.org" gives the same error, while "wss://echo.websocket.org/?encoding=text" works just fine.

What's more strange, is that if I use the GDAX production URL at "wss://ws-feed.gdax.com", the connection terminates immediately with {:error, %WebSockex.RequestError{code: 400, message: "Bad Request"}}

This does not happen with websocket_client (https://hex.pm/packages/websocket_client/1.3.0), which works perfectly.

What's going on here?

It would be cool, if these two numbers could be part of the Conn-struct and interchangeable via options.

https://github.com/Azolo/websockex/blob/master/lib/websockex/conn.ex#L92
https://github.com/Azolo/websockex/blob/master/lib/websockex/conn.ex#L188

defmodule Foo.Client do
  use WebSockex
  require Logger

  @name __MODULE__
  @uri "..."

  def child_spec([]) do
    %{
      id: @name,
      start: {@name, :start_link, []},
    }
  end

  def start_link() do
    WebSockex.start_link(@uri, @name, :foo, name: @name)
  end

  def handle_frame({:text, msg}, state) do
    Logger.info msg
    raise "oops"
  end

If something unexpected happens in handle_frame, the exception is completely swallowed - the process crashes but nothing is logged to the console.

When invoking callbacks be sure to call terminate on the event that the callback raises some sort of error.

This may be working on the common_handle callbacks but isn't tested, so tests need to be written.

There will be 2 edge cases I can think of:

• The initial handle_connect callback.

• handle_disconnect, where the :handle_initial_conn_failure option is passed but the initial handle_connect hasn't been called.

• handle_cast/2

• handle_connect/2

• handle_disconnect/2

• handle_frame/2

• handle_info/2

• handle_ping/2

• handle_pong/2

• terminate/2

Reported via elixirforum WebSockex thread

You include in one of your examples the line:

WebSockex.start_link("wss://echo.websocket.org/?encoding=text", __MODULE__, :state)

When I run the same line of code, I get WebSockex.URLError{..}. I believe this code is because of the parse_uri/1 implementation:

    case URI.parse(url) do
      # This is confusing to look at. But it's just a match with multiple guards
      %URI{host: host, port: port, scheme: protocol}
      when is_nil(host)
      when is_nil(port)
      when not protocol in ["ws", "wss"] ->
        {:error, %WebSockex.URLError{url: url}}
      %URI{} = uri ->
        {:ok, uri}
    end

The result of calling URI.parse("wss://echo.websocket.org/?encoding=text") is going to return %URI{..., port: nil, ...}. Is there a reason you're asserting on the port being present? Or rather, is there a reason that the example code is compiling for you and not for me? Other URLs that I've tried have returned WebSockex.URLErrors as well, which is confusing me.

Hi there,

I've been using WebSockex for a project, but I've been having trouble having the socket know when we've lost connection.

For example, when I am connected to a web socket, and disconnect from wifi, handle_disconnect does not trigger. When I reconnect to wifi, it keeps receiving messages.

Is there anyway for the web socket to recognize the loss of connection in this case?

Thanks for the help!

Hello,
I'm trying to connect and test the "wss://api.bitfinex.com/ws/2"
Docs: https://bitfinex.readme.io/v2/reference#ws-public-ticker

Code:

iex(10)> uri = URI.parse "wss://api.bitfinex.com/ws/2"
%URI{
  authority: "api.bitfinex.com",
  fragment: nil,
  host: "api.bitfinex.com",
  path: "/ws/2",
  port: 443,
  query: nil,
  scheme: "wss",
  userinfo: nil
}
iex(11)> conn = WebSockex.Conn.new uri                
%WebSockex.Conn{
  cacerts: nil,
  conn_mod: :ssl,
  extra_headers: [],
  host: "api.bitfinex.com",
  insecure: true,
  path: "/ws/2",
  port: 443,
  query: nil,
  socket: nil,
  socket_connect_timeout: 6000,
  socket_recv_timeout: 5000,
  transport: :ssl
}
iex(12)> start_link = WS.start_link conn, :fake_state 
[info] ['TLS', 32, 'client', 58, 32, 73, 110, 32, 115, 116, 97, 116, 101, 32, 'cipher', 32, 'received SERVER ALERT: Fatal - Bad Record MAC', 10]
{:error, %WebSockex.ConnError{original: {:tls_alert, 'bad record mac'}}}
iex(13)> 

Its some limitation in lib or im doing something wrong?

Which makes the code_change callback useless.

Oops.

Reported via elixirforum WebSockex thread

It doesn't appear to me that there is any way to get information out of the module using WebSockex short of :sys.get_state(pid). I'm trying to use WebSockex with an events api where the local side of the connection builds up a rather large state of the world. Instead of dispatching the full state to separate worker processes when key events happen, I would like to ask for small bits of state when needed.

An example of what I'd like to do is get the profile of a user from the list of 1000s maintained in the module state. Since GenServer was a thought for implementation at one point, is the call functionality of GenServer something on the plate to add?

If not, is the intended interaction here for the WebSockex module to forward events as they are received to a separate process that can have the interface for querying reduced information of state?

I began noticing a strange issue when connecting / receiving messages after upgrading to erts-10.3. Originally noticed the issue when changing a docker image base from elixir:1.8.0-alpine to elixir:1.8.1-alpine, and did some troubleshooting to isolate that it was the erts-10.3 upgrade from the 10.2 series. Neither the underlying os change (base, -slim, etc) seems to matter, and in fact, the same issue occurs on Mac OSX between the different erts versions. In other words, Elixir 1.8.1 works ok, so long as erts-10.2.x is involved, instead of erts-10.3.

All tested versions of websockex appear to be affected (0.2.x, 0.3.x, 0.4.x).

I've assembled a toy example here:
https://github.com/bfolkens/socketfail

I chose a relatively active public websocket channel (BitMEX), and constructed a very bare test case using mostly example code from the Websockex and Elixir docs. After an initial burst of activity, the responses stop when using the elixir-1.8.1 base image. However, when the base image is switched to elixir-1.8.0, the example continues to run indefinitely.

handle_disconnect works perfectly for infinite reconnection like in issue 5 - when host got down after some time

but it does not handle connection error in WebSockex.start_link - so if there is no connection to host, application does not start:
21:18:29.799 [info] Application arbi exited: Arbi.Application.start(:normal, []) returned an error: shutdown: failed to start child: Arbi.WsSup ** (EXIT) shutdown: failed to start child: Arbi.WsWorker ** (EXIT) %WebSockex.ConnError{original: :econnrefused}

When :reconnect was returned by handle_disconnect or handle_connect_failure there is currently no way to know when the connection was established again.

Having a new optional callback like this could help:

      @doc false
      def handle_connect(_conn, state) do
        {:ok, state}
      end

My workaround for now is using handle_frame which is not very reliable.

Thanks for this great library!

I wanted to test disconnection behaviour and tried this:

1. Connect successfully with websocket and let the connection idle.
2. Disconnect the WIFI from my computer, which is the only internet connection it has.
3. Wait for handle_disconnect to be called, which is not the case.
4. Profit!

Did i do something wrong? Is there a other way to test this?

I also tried to use active: true mode, and fiddling with wait_for_response which did not help detecting the disconnect.

Hi,

I need to connect with Elixir to a remote Websocket using the client certificate. Would it be possible to put something about this into the documentation. How and where do you specify where is the TLS certificate that you would use at connection establishment?

I looked but can't really find anything on this in the docs.

Many thanks,

Tomaz

Another suggestion from @idi-ot on elixirforum.com.

In essence the spec says that it has to be valid UTF8, but only says to close the connection when receiving invalid UTF8 text.

Conclusion: It doesn't say I need to check to make sure I send valid UTF8 text.

For example, when connecting to a WS server behind CloudFlare, CloudFlare appends some extra headers to the response, such as cf-ray; it'd be nice to be able to be able to "see" these. I would file a PR, but I'm not really certain what the best way to do this would be :<

Hi, for me it's not clear how correct use websockex, I did an small websocket server which when receive a message reply with "hello you sent -> your_message", basically I need send a message and get the response

import * as express from 'express';
import * as http from "http";
import * as WebSocket from 'ws';

const app = express();
const server = http.createServer(app);
const wss = new WebSocket.Server({server});

wss.on('connection', (ws: WebSocket) => {
    ws.send("hi I'm ready");
    ws.on('message', (message: string) => {
        console.log(`received ${message}`);
        ws.send(`hello you sent -> ${message}`);
    })
});

server.listen(8999,()=>{
    //para correrlo se puede hacer con ts-node server.ts
    console.log("server running on port 8999")
})

now I'm trying to connect to this server with websockex with this code

defmodule WebsocketCommunicator do
  use WebSockex
  require Logger

  def start_link(opt \\ []) do
    WebSockex.start_link("ws://localhost:8999",__MODULE__,:fake_state,opt)
  end

  def handle_connect(_conn, state) do
    Logger.info("Connected!")
    {:ok, state}
  end

  def echo(client,message) do
    Logger.info("mandando mensaje #{message}")
    WebSockex.send_frame(client, {:text, message})
  end

  def handle_frame({:text,msg},:fake_state) do
    Logger.info("sending --> #{msg}")

    {:reply,{:text,msg},:fake_state}
  end
end

but when I try to run this

{:ok,pid} = WebsocketCommunicator.start_link()

I get

sending --> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent -> hello you sent

how can send and get the websocket response with websockex???...why after connect with start_link start sending messages?...

sorry for my noobs questions, thank you so much

Thanks for you changes. While looking through i found this:

The order of arguments in handle_connect(state, conn) does not match most other functions. The argument state is mostly the last argument. Expecting the conn as first argument would be more common for me.

Would do you think about that?

Hello! I need to send some params to websocket server to check them during the connection...
how can i do it?

A suggestion from @idi-ot from elixirforum.com.

There's no reason why it couldn't happen. Frame encoding pipeline needs to change a bit.

Problem

I wanted to send a message when I connect using websockets. Using your EchoClient example I added this callback

  def handle_connect(conn, state) do
    pid = self()
    EchoClient.echo(pid, "Hello")
    {:ok, state}
  end

Sending message: Hello is logged but nothing is received.
If I do

  def handle_connect(conn, state) do
    pid = self()
    spawn fn -> EchoClient.echo(pid, "Hello") end
    {:ok, state}
  end

Everything works as expected, the message is sent and received.
I don't understand why I needed to do the spawn ? Is this an issue with the library or with my understanding of processes ?

I'm experiencing issues where larger frames (50KB+) delivered via fragments don't complete.

Using trace:

*DBG* #PID<0.218.0> received frame: {:fragment, :text ....
*DBG* #PID<0.218.0> received frame: {:continuation, ...
--- gets stuck without more :continuation or :end ----

I was wondering if this was a known issue.

Using elixir-socket library, I am able to receive every fragment.

I'd like to write unit tests around my web socket message handling and I haven't been able to think of a good way to assert on the response my code sends back to the socket. It looks like in the tests for websockex you implement a fake server and connect a client to it to test things. Could you expose that server so that consumers of their library can make use of it as well?

I'm also open to other suggestions as well if anyone has other ideas.

Look at adding a format_status function for the get_status system message.

Then adding a callback for user supplied format_status functions. Like GenServer's format_status callback

Reported via elixirforum WebSockex thread

Looks to be similar and tangentially related to #45.

Here's the relevant excerpt from my logs -

2018-05-03 22:39:32.637 [warn] websocket disconnected with reason {:remote, 1001, ""}, attempting reconnect
2018-05-03 22:39:32.919 [error] No handle_info/2 clause in Elixir.Nostrum.Shard.Session provided for {:ssl_closed, {:sslsocket, {:gen_tcp, #Port<0.16490>, :tls_connection, :undefined}, #PID<0.4982.0>}}

I'm able to trigger this by returning {:close, state} from a handle_cast/2 callback, although I get it when the remote party disconnects as well as is the case with the log above.

I've confirmed that the issue is happening in the close_loop/4 function by adding the following match to the receive. Besides the print it's all just copied from the :tcp_closed match 🙃.

{:ssl_closed, ^socket} ->
  IO.inspect "received ssl_closed in close loop"
  new_conn = %{conn | socket: nil}
  debug = Utils.sys_debug(debug, :closed, state)
  purge_timer(timer_ref, :"websockex_close_timeout")
  state = Map.delete(state, :timer_ref)
  on_disconnect(reason, parent, debug, %{state | conn: new_conn})

As expected this prints the message and seems to be handled correctly. If this is how it should be handled (I'm unsure if we need to do anything special for ssl connections) I can open a PR. I'd probably need a little guidance to properly set up a test though.

Consider allowing the use of a WebSockex.Conn struct as the first parameter of start/4 and start_link/4.

Currently a url string is converted into a WebSockex.Conn and passed around. Allowing the use of Conn directly would allow a more fine tuned connection.

Does the library support reconnect? When the server goes down, it'd be nice if the websocket client tried to reconnect indefinitely (or have some sort of backoff strategy).

The Erlang :ssl module is very strict in its certification checking.

The result of that is that, if I understand correctly, verify_peer option works when everything is basically done correctly.

What I found out is that the internet is obviously broken everywhere, the :ssl module reliably connects to just about zero of the sites I tried when verifying the peer with CA certs from erlang-certifi.

So I tried using the verify_fun option with ssl_verify_fun.erl, plus the partial_chain based off of hackney's implementation... But it didn't work.

After that I tried a bunch of other stuff and made some progress, changed some more stuff only to find that it still didn't work. So I threw my hands up and made everything insecure by default.

TLDR;

I'm not smart enough to make the :ssl module's verify_peer work correctly.

I need to open a websocket connection through a http proxy. The proxy does support websockets when I use a web browser as client.

Is there a way to configure WebSockex to use an http proxy?

Hi @Azolo,

When you get the chance could you please tag and publish a new version to hex?

There have been several changes/bug fixes since 0.4.1 earlier this year (Jan 22 2018). Packages that depend on websockex can't publish a new release against these changes using a reference to a github repo.

🍻

defmodule WebSocketClient do
  use WebSockex

  def start_link(url, state) do
    WebSockex.start_link(url, __MODULE__, state)
  end

  def handle_info(event, state) do
    IO.puts("Handle info event is called.")
    {:ok, state}
  end
end

iex(1)> {:ok, pid} = WebSocketClient.start_link("ws://echo.websocket.org", %{})
iex(2)> GenServer.call(pid, {:some_random_event})
Handle info event is called.
** (exit) exited in: GenServer.call(#PID<0.413.0>, {:some_random_event}, 5000)
    ** (EXIT) time out
    (elixir) lib/gen_server.ex:924: GenServer.call/3

iex process then hangs. I tried looking in the source code but I wasn't really sure what I was looking at.

When running echo_client example in iex I get an error: %WebSockex.ConnError{original: :closed} from start_link()

It works fine if I replace wss:// with ws://

def start_link(opts \\ []) do
    WebSockex.start_link("ws://echo.websocket.org/?encoding=text", __MODULE__, :fake_state, opts)
  end

This hasn't been done because I'm lazy. Sorry.

See handle_debug/4 and the Erlang debug structure.

Reported via elixirforum WebSockex thread.

Special processes should exit when their parent exits according to OTP principles and WebSockex does not.

Reported via elixirforum WebSockex thread

I found this project via the Elixir Forum - great work!

As was discussed on your announcement thread, there are multiple other Websocket clients for Elixir and Erlang:

• https://github.com/sanmiguel/websocket_client
• https://github.com/meh/elixir-socket
• https://github.com/ninenines/gun

You touched on your motivation for creating Websockex on the thread. It would be great to have a comparison section in the README describing the differences between your project and the others.

It would be cool if send_frame had a mechanism to inform a process if a Frame was sent successfully.

I'm running dialyzer on a project using websockex and running into an error. Does anyone have any suggestions on how to fix it?

Cheers

lib/websockex.ex:8:callback_type_mismatch
Callback mismatch for @callback handle_ping/2 in WebSockex behaviour.

Expected type:

  {:close, _}
  | {:ok, _}
  | {:close, {integer(), binary()}, _}
  | {:reply, {:binary, binary()} | {:ping, binary()} | {:text, binary()}, _}


Actual type:
{:reply, :pong | {:pong, _}, _}

You mentioned your plan to switch to a GenServer on the Elixir Forum thread. Adding a tracking issue for it. :)

Is there a way to disable auto ping by websockex?

Hey, I've been using websockex in a project and it's working great (thanks for writing it). I do have one problem though.

I initiate a Supervisor with a simple_one_for_one that is responsible for initializing a websockex. It calls websockex with start (not start_link), but whenever the socket in which I'm listening is closed websockex will raise an exception, that in production gets logged as this:

CRASH REPORT==== 11-Nov-2017::12:47:29 ===
crasher:
initial call: Elixir.WebSockex:init/5
pid: <0.1340.0>
registered_name: []
exception exit: {remote,closed}
in function 'Elixir.WebSockex':terminate/4 (lib/websockex.ex, line 870)
ancestors: ['Elixir.Channeler.Monitor','Elixir.Channeler.Supervisor',
<0.1284.0>]
message_queue_len: 0
messages: []
links: []
dictionary: []
trap_exit: false
status: running
heap_size: 28690
stack_size: 27
reductions: 33897241
neighbours:

I have a terminate callback and I can see on the reason {:remote, :closed}, but would like to know how I can handle the :closed event so that it doesn't technically "crash"? I'm fine with writing the explanation to the docs and PR'ing if you would like, along with a section making notice of both start and start_link. Thanks

There's a function clause error with conn_module/2 when using any other protocol than ws or wss. conn_module/2 should at least return nil.

Though, I'm grappling with allowing protocols other than those two with start and start_link.

I regularly get an erl-dump with a badmatch for lib/websockex/frame.ex on line 144.

    pid: <0.1331.0>^M
    registered_name: []^M
    exception error: no match of right hand side value ...

I don't know if this is my fault or a known bug. If I can provide any further information, let me know.

defmodule GdaxStream.Client do
  use WebSockex

  def start_link(url, state) do
    WebSockex.start_link(url, __MODULE__, state)
  end

  def handle_connect(conn, state) do
    IO.puts("connected!")
    {:ok, state}
  end

end

What can I do if I want to send a subscription frame just after connected?

  def subscription_frame() do
    data =
      Poison.encode!(%{
        type: "subscribe",
        product_ids: ["BTC-USD"],
        channels: ["ticker"]
      })

    {:text, data}
  end

  def handle_connect(conn, state) do
    WebSockex.send_frame(self(), subscription_frame())
    {:ok, state}
  end

doing so doesn't work, {:error, %WebSockex.CallingSelfError{function: :send_frame}}

I want to make send_frame synchronous in the sense that I want to block until I get the server response for the request, not just the frame sending ack.

The frame contains a json with a requestId UUID and the server respond with the same requestId.

With GenServer I could do something like that: https://elixirforum.com/t/wrap-async-process-for-genserver-call/2248/3. Do you think something similar is possible with your process ?

When any of the :gen modules terminate the create an error log. WebSockex should do that too.