pants.http.websocket

pants.http.websocket implements the WebSocket protocol, as described by RFC 6455, on top of the Pants HTTP server using an API similar to that provided by pants.stream.Stream.

Using WebSockets

To start working with WebSockets, you’ll need to create a subclass of WebSocket. As with Stream, WebSocket instances are meant to contain the majority of your networking logic through the definition of custom event handlers. Event handlers are methods that have names beginning with on_ that can be safely overridden within your subclass.

Listening for Connections

WebSocket is designed to be used as a request handler for the Pants HTTP server, pants.http.server.HTTPServer. As such, to begin listening for WebSocket connections, you must create an instance of HTTPServer using your custom WebSocket subclass as its request handler.

from pants.http import HTTPServer, WebSocket
from pants import Engine

class EchoSocket(WebSocket):
    def on_read(self, data):
        self.write(data)

HTTPServer(EchoSocket).listen(8080)
Engine.instance().start()

If you need to host traditional requests from your HTTPServer instance, you may invoke the WebSocket handler simply by creating an instance of your WebSocket subclass with the appropriate pants.http.server.HTTPRequest instance as its only argument:

from pants.http import HTTPServer, WebSocket
from pants import Engine

class EchoSocket(WebSocket):
    def on_read(self, data):
        self.write(data)

def request_handler(request):
    if request.path == '/_ws':
        EchoSocket(request)
    else:
        request.send_response("Nothing to see here.")

HTTPServer(request_handler).listen(8080)
Engine.instance().start()

WebSocket and Application

WebSocket has support for pants.web.application.Application and can easily be used as a request handler for any route. Additionally, variables captured from the URL by Application will be made accessible to the WebSocket.on_connect() event handler. The following example of a WebSocket echo server displays a customized welcome message depending on the requested URL.

from pants.http import WebSocket
from pants.web import Application

app = Application()

@app.route("/ws/<name>")
class EchoSocket(WebSocket):
    def on_connect(self, name):
        self.write(u"Hello, {name}!".format(name=name))

    def on_read(self, data):
        self.write(data)

app.run(8080)

WebSocket Security

Secure Connections

WebSocket relies upon the pants.http.server.HTTPServer instance serving it to provide SSL. This can be as easy as calling the server’s startSSL() method.

To determine whether or not the WebSocket instance is using a secure connection, you may use the is_secure attribute.

Custom Handshakes

You may implement custom logic during the WebSocket’s handshake by overriding the WebSocket.on_handshake() event handler. The on_handshake handler is called with a reference to the HTTPRequest instance the WebSocket handshake is happening upon as well as an empty dictionary that may be used to set custom headers on the HTTP response.

on_handshake is expected to return a True value if the request is alright. Returning a False value will result in the generation of an error page. The following example of a custom handshake requires a secret HTTP header in the request, and that the connection is secured:

from pants.http import WebSocket

class SecureSocket(WebSocket):
    def on_handshake(self, request, headers):
        return self.is_secure and 'X-Pizza' in request.headers

    def on_connect(self):
        self.write(u"Welcome to PizzaNet.")

Reading and Writing Data

WebSockets are a bit different than normal Stream instances, as a WebSocket can transmit both byte strings and unicode strings, and data is encapsulated into formatted messages with definite lengths. Because of this, reading from one can be slightly different.

Mostly, however, the read_delimiter works in exactly the same way as that of pants.stream.Stream.

Unicode Strings and Byte Strings

WebSocket strictly enforces the difference between byte strings and unicode strings. As such, the connection will be closed with a protocol error if any of the following happen:

1. The string types of the read_delimiter and the buffer differ.
2. There is an existing string still in the buffer when the client sends another string of a different type.
3. The read_delimiter is currently a struct and the buffer does not contain a byte string.

Of those conditions, the most likely to occur is the first. Take the following code:

from pants.http import WebSocket, HTTPServer
from pants import Engine

def process(text):
    return text.decode('rot13')

class LineOriented(WebSocket):
    def on_connect(self):
        self.read_delimiter = "\n"

    def on_read(self, line):
        self.write(process(line))

HTTPServer(LineOriented).listen(8080)
Engine.instance().start()

And, on the client:

<!DOCTYPE html>
<textarea id="editor"></textarea><br>
<input type="submit" value="Send">
<script>
    var ws = new WebSocket("ws://localhost:8080/"),
        input = document.querySelector('#editor'),
        button = document.querySelector('input');

    ws.onmessage = function(e) {
        alert("Got back: " + e.data);
    }

    button.addEventListener("click", function() {
        ws.send(input.value + "\n");
    });
</script>

On Python 2.x, the read delimiter will be a byte string. The WebSocket will expect to receive a byte string. However, the simple JavaScript shown above sends unicode strings. That simple service would fail immediately on Python 2.

To avoid the problem, be sure to use the string type you really want for your read delimiters. For the above, that’s as simple as setting the read delimiter with:

self.read_delimiter = u"\n"

WebSocket Messages

In addition to the standard types of read_delimiter, WebSocket instances support the use of a special value called EntireMessage. When using EntireMessage, full messages will be sent to your on_read event handler, as framed by the remote end-point.

EntireMessage is the default read_delimiter for WebSocket instances, and it makes it dead simple to write simple services.

The following example implements a simple RPC system over WebSockets:

import json

from pants.http.server import HTTPServer
from pants.http.websocket import WebSocket, FRAME_TEXT
from pants import Engine

class RPCSocket(WebSocket):
    methods = {}

    @classmethod
    def method(cls, name):
        ''' Add a method to the RPC. '''
        def decorator(method):
            cls.methods[name] = method
            return method
        return decorator

    def json(self, **data):
        ''' Send a JSON object to the remote end-point. '''
        # JSON outputs UTF-8 encoded text by default, so use the frame
        # argument to let WebSocket know it should be sent as text to the
        # remote end-point, rather than as binary data.
        self.write(json.dumps(data), frame=FRAME_TEXT)

    def on_read(self, data):
        # Attempt to decode a JSON message.
        try:
            data = json.loads(data)
        except ValueError:
            self.json(ok=False, result="can't decode JSON")
            return

        # Lookup the desired method. Return an error if it doesn't exist.
        method = data['method']
        if not method in self.methods:
            self.json(ok=False, result="no such method")
            return

        method = self.methods[method]
        args = data.get("args", tuple())
        kwargs = data.get("kwargs", dict())
        ok = True

        # Try running the method, and capture the result. If it errors, set
        # the result to the error string and ok to False.
        try:
            result = method(*args, **kwargs)
        except Exception as ex:
            ok = False
            result = str(ex)

        self.json(ok=ok, result=result)


@RPCSocket.method("rot13")
def rot13(string):
    return string.decode("rot13")

HTTPServer(RPCSocket).listen(8080)
Engine.instance().start()

As you can see, it never even uses read_delimiter. The client simply sends JSON messages, with code such as:

my_websocket.send(JSON.stringify({method: "rot13", args: ["test"]}));

This behavior is completely reliable, even when the client is sending fragmented messages.

WebSocket

class pants.http.websocket.WebSocket(request, *arguments)

An implementation of WebSockets on top of the Pants HTTP server using an API similar to that of pants.stream.Stream.

A WebSocket instance represents a WebSocket connection to a remote client. In the future, WebSocket will be modified to support acting as a client in addition to acting as a server.

When using WebSockets you write logic as you could for Stream, using the same read_delimiter and event handlers, while the WebSocket implementation handles the initial negotiation and all data framing for you.

Argument	Description
request	The HTTPRequest to begin negotiating a WebSocket connection over.

is_secure

Whether or not the underlying HTTP connection is secured.

allow_old_handshake

Whether or not to allow clients using the old draft-76 protocol to connect. By default, this is set to False.

Due to the primitive design of the draft-76 version of the WebSocket protocol, there is significantly reduced functionality when it is being used.

1. Binary data cannot be transmitted. All communications between the WebSocket instance and the remote end-point must take place using unicode strings.
2. Connections are closed immediately with no concept of close reasons. When you use close() on a draft-76 WebSocket, it will flush the buffer and then, once the buffer empties, close the connection immediately.
3. There are no control frames, such as the PING frames created when you invoke ping().

There are other missing features as well, such as extensions and the ability to fragment long messages, but Pants does not currently provide support for those features at this time.

buffer_size

The maximum size, in bytes, of the internal buffer used for incoming data.

When buffering data it is important to ensure that inordinate amounts of memory are not used. Setting the buffer size to a sensible value can prevent coding errors or malicious use from causing your application to consume increasingly large amounts of memory. By default, a maximum of 64kb of data will be stored.

The buffer size is mainly relevant when using a string value for the read_delimiter. Because you cannot guarantee that the string will appear, having an upper limit on the size of the data is appropriate.

If the read delimiter is set to a number larger than the buffer size, the buffer size will be increased to accommodate the read delimiter.

When the internal buffer’s size exceeds the maximum allowed, the on_overflow_error() callback will be invoked.

Attempting to set the buffer size to anything other than an integer or long will raise a TypeError.

close(flush=True, reason=1000, message=None)

Close the WebSocket connection. If flush is True, wait for any remaining data to be sent and send a close frame before closing the connection.

Argument	Default	Description
flush	True	Optional. If False, closes the connection immediately, without ensuring all buffered data is sent.
reason	1000	Optional. The reason the socket is closing, as defined at RFC 6455#section-7.4.
message	None	Optional. A message string to send with the reason code, rather than the default.

local_address

The address of the WebSocket on the local machine.

By default, this will be the value of socket.getsockname or None. It is possible for user code to override the default behaviour and set the value of the property manually. In order to return the property to its default behaviour, user code then has to delete the value. Example:

# default behaviour
channel.local_address = custom_value
# channel.local_address will return custom_value now
del channel.local_address
# default behaviour

on_close()

Placeholder. Called after the WebSocket has finished closing.

on_connect(*arguments)

Placeholder. Called after the WebSocket has connected to a client and completed its handshake. Any additional arguments passed to the WebSocket instance’s constructor will be passed to this method when it is invoked, making it easy to use WebSocket together with the URL variables captured by pants.web.application.Application, as shown in the following example:

from pants.web import Application
from pants.http import WebSocket

app = Application()
@app.route("/ws/<int:id>")
class MyConnection(WebSocket):
    def on_connect(self, id):
        pass

on_handshake(request, headers)

Placeholder. Called during the initial handshake, making it possible to validate the request with custom logic, such as Origin checking and other forms of authentication.

If this function returns a False value, the handshake will be stopped and an error will be sent to the client.

Argument	Description
request	The pants.http.server.HTTPRequest being upgraded to a WebSocket.
headers	An empty dict. Any values set here will be sent as headers when accepting (or rejecting) the connection.

on_overflow_error(exception)

Placeholder. Called when an internal buffer on the WebSocket has exceeded its size limit.

By default, logs the exception and closes the WebSocket.

Argument	Description
exception	The exception that was raised.

on_pong(data)

Placeholder. Called when a PONG control frame is received from the remote end-point in response to an earlier ping.

When used together with the ping() method, on_pong may be used to measure the connection’s round-trip time. See ping() for more information.

Argument	Description
data	Either the RTT expressed as seconds, or an arbitrary byte string that served as the PONG frame’s payload.

on_read(data)

Placeholder. Called when data is read from the WebSocket.

Argument	Description
data	A chunk of data received from the socket. Binary data will be provided as a byte string, and text data will be provided as a unicode string.

on_write()

Placeholder. Called after the WebSocket has finished writing data.

ping(data=None)

Write a ping frame to the WebSocket. You may, optionally, provide a byte string of data to be used as the ping’s payload. When the end-point returns a PONG, and the on_pong() method is called, that byte string will be provided to on_pong. Otherwise, on_pong will be called with the elapsed time.

Argument	Description
data	Optional. A byte string of data to be sent as the ping’s payload.

read_delimiter

The magical read delimiter which determines how incoming data is buffered by the WebSocket.

As data is read from the socket, it is buffered internally by the WebSocket before being passed to the on_read() callback. The value of the read delimiter determines when the data is passed to the callback. Valid values are None, a string, an integer/long, a compiled regular expression, an instance of struct.Struct, an instance of netstruct.NetStruct, or the EntireMessage object.

When the read delimiter is the EntireMessage object, entire WebSocket messages will be passed to on_read() immediately once they have been received in their entirety. This is the default behavior for WebSocket instances.

When the read delimiter is None, data will be passed to on_read() immediately after it has been received.

When the read delimiter is a byte string or unicode string, data will be buffered internally until that string is encountered in the incoming data. All data up to but excluding the read delimiter is then passed to on_read(). The segment matching the read delimiter itself is discarded from the buffer.

Note

When using strings as your read delimiter, you must be careful to use unicode strings if you wish to send and receive strings to a remote JavaScript client.

When the read delimiter is an integer or a long, it is treated as the number of bytes to read before passing the data to on_read().

When the read delimiter is an instance of struct.Struct, the Struct’s size is fully buffered and the data is unpacked before the unpacked data is sent to on_read(). Unlike other types of read delimiters, this can result in more than one argument being sent to the on_read() event handler, as in the following example:

import struct
from pants.http import WebSocket

class Example(WebSocket):
    def on_connect(self):
        self.read_delimiter = struct.Struct("!ILH")

    def on_read(self, packet_type, length, id):
        pass

You must send binary data from the client when using structs as your read delimiter. If Pants receives a unicode string while a struct read delimiter is set, it will close the connection with a protocol error. This holds true for the Netstruct delimiters as well.

When the read delimiter is an instance of netstruct.NetStruct, the NetStruct’s minimum_size is buffered and unpacked with the NetStruct, and additional data is buffered as necessary until the NetStruct can be completely unpacked. Once ready, the data will be passed to on_read(). Using Struct and NetStruct are very similar.

When the read delimiter is a compiled regular expression (re.RegexObject), there are two possible behaviors that you may switch between by setting the value of regex_search. If regex_search is True, as is the default, the delimiter’s search() method is used and, if a match is found, the string before that match is passed to on_read(). The segment that was matched by the regular expression will be discarded.

If regex_search is False, the delimiter’s match() method is used instead and, if a match is found, the match object itself will be passed to on_read(), giving you access to the capture groups. Again, the segment that was matched by the regular expression will be discarded from the buffer.

Attempting to set the read delimiter to any other value will raise a TypeError.

The effective use of the read delimiter can greatly simplify the implementation of certain protocols.

remote_address

The remote address to which the WebSocket is connected.

By default, this will be the value of socket.getpeername or None. It is possible for user code to override the default behaviour and set the value of the property manually. In order to return the property to its default behaviour, user code then has to delete the value. Example:

# default behaviour
channel.remote_address = custom_value
# channel.remote_address will return custom_value now
del channel.remote_address
# default behaviour

write(data, frame=None, flush=False)

Write data to the WebSocket.

Data will not be written immediately, but will be buffered internally until it can be sent without blocking the process.

Calling write() on a closed or disconnected WebSocket will raise a RuntimeError.

If data is a unicode string, the data will be sent to the remote end-point as text using the frame opcode for text. If data is a byte string, the data will be sent to the remote end-point as binary data using the frame opcode for binary data. If you manually specify a frame opcode, the provided data must be a byte string.

An appropriate header for the data will be generated by this method, using the length of the data and the frame opcode.

Arguments	Description
data	A string of data to write to the WebSocket. Unicode will be converted automatically.
frame	Optional. The frame opcode for this message.
flush	Optional. If True, flush the internal write buffer. See pants.stream.Stream.flush() for details.

write_file(sfile, nbytes=0, offset=0, flush=False)

Write a file to the WebSocket.

This method sends an entire file as one huge binary frame, so be careful with how you use it.

Calling write_file() on a closed or disconnected WebSocket will raise a RuntimeError.

Arguments	Description
sfile	A file object to write to the WebSocket.
nbytes	Optional. The number of bytes of the file to write. If 0, all bytes will be written.
offset	Optional. The number of bytes to offset writing by.
flush	Optional. If True, flush the internal write buffer. See flush() for details.

write_packed(*data, **kwargs)

Write packed binary data to the WebSocket.

Calling write_packed() on a closed or disconnected WebSocket will raise a RuntimeError.

If the current read_delimiter is an instance of struct.Struct or netstruct.NetStruct the format will be read from that Struct, otherwise you will need to provide a format.

Argument	Description
*data	Any number of values to be passed through struct and written to the remote host.
format	Optional. A formatting string to pack the provided data with. If one isn’t provided, the read delimiter will be used.
flush	Optional. If True, flush the internal write buffer. See flush() for details.

EntireMessage

pants.http.websocket.EntireMessage

EntireMessage is a unique Python object that, when set as the read_delimiter for a WebSocket instance, will cause entire messages to be passed to the on_read() event handler at once.