Custom Devices - implemented externally or in p44script

What are "Custom Devices"?

"Custom devices" are devices that behave in the system (Digital Strom with P44-DSB-xx, or via matter integration) exactly like other devices, but can be implemented by the user in a simple way. In this way, special hardware can also be easily integrated into the Smart Home, for example via an http API or (e.g. on P44-DSB-X) via direct hardware extensions (RaspberryPi shields, chips connected to GPIO, etc.).

How are "Custom Devices" created?

"Custom Devices" can be implemented in two different ways:

• Starting with firmware version 2.6.0 directly in the P44-xx device as p44script. This kind of external devices can be created completely from the web interface of P44-xx devices, and are also saved in the configuration backups as part of the device configuration. p44script provides a lot of convenient functions to control different kinds of devices: from functions for accessing HTTP APIs (geturl, posturl, puturl), direct access to digital (digitalio) and analog (analogio) inputs and outputs, to motors with limit switches and current monitoring (dcmotor), to the graphics subsystem lrgraphics for SmartLED chains, a lot is possible. Here are easy to implement/extend examples to find.

• Already since firmware 1.5.0.8 there is the external device API. With this API an external script or application can connect to a P44-xx device via a TCP socket and easily integrate devices into the smarthome system. This way is useful e.g. to establish connections between different systems, or to integrate devices with very complex implementation that would go beyond the scope of p44Script performance. Here are several examples to find how the external device API can be used.

These two possible implementation types work almost identically in concept, the only difference is where the implementation runs (inside the P44-xx device as p44script, or outside and connected via TCP socket). In both cases, however, the communication "language" of the device implementation with the system is the same, namely the external device API. For p44script implementations, the message() function is used to receive and send external device API messages. For fully external devices, messages are exchanged over TCP/IP.

Similarities between p44script and external implementations

• The device is described to the system by the so-called init message, which takes the form of a JSON object.
• In the init-message you can choose between a very simple text variant and a more detailed JSON variant for further communication. By default JSON is used, which is very easy to read and write in p44Script.
• The implementation only has to take care of passing output values given by the system to the hardware and reporting input values to the system. All the complexity of scenes, room assignments, click detection, dimming behavior, sensor value filters, etc. is handled by the p44-xx device.

Differences between p44script and external implementations

• For p44script implementations, the init message is entered once when the device is created in the web interface. For external devices, the init message must be submitted at the beginning of communication (each time the TCP connection is re-established).
• For p44script implementations, the init message may be written on multiple lines and may even contain C-style comments /* ... */. For external implementations, the whole init-message must be sent on a single line and as pure JSON without comments (because the end of line is taken as a message separator).
• In external implementations, it is possible to run multiple devices over a single TCP connection; the messages are then assigned to different devices via so-called tags. In p44script implementations there is no such multiple device mechanism, and therefore no tags. The references to tag parameters in the following description of the external device API are therefore irrelevant for p44script implementations.
• In p44script implementations the messages initvdc, log and bye are not available.

Using the external device API from p44script implementations

Creating p44script based devices

To create a p44script based device, use the "+ Scripted" button in the P44-xx web interface. You can then enter a init message (specification below), which is then immediately used to create a device.

Operation of p44script implemented devices

The device must be implemented in a script that basically does the following

• initialize the device (set-up hardware, initialize parameters etc.)

• If the device has outputs, make sure to have a handler processing incoming messages, like:

on (message()) as m {
  // check the message and act upon channel changes etc.
  if (m.message="channel") {
    // ...update the channel in the hardware
  }
}

• If the device has buttons, inputs or sensors, make sure to use message(msg_to_send) to report to the system when inputs change. This can be done from a permanently running main loop (but do not forget to include enough delay() to avoid using excessive CPU time and slow down the entire device!), or on() {...} handlers based on input events or regular timing (e.g. using the every() function), for example

on (every(0:05)) {
  // every 5 minutes, poll a sensor
  var s = getmysensorvalue()
  var m = { 'message': 'sensor', 'index':0 }
  m.value = s;
  message(m)
}

• either keep running (e.g. in a loop) or return true. Note that returning true (or keep running) is essential because all other ways a script ends is treated as a possibly temporary problem, which means the script is re-started. Only if the implementation scripts returns true, this is treated as an OK signal meaning that the device can operate without a script running permanently (i.e. only via on(...) {} handlers that trigger some code when needed)

For more information, see the examples.

Using the external device API via TCP-IP

To host external devices, vdcd must be started with the --externaldevices option, providing a port number or absolute unix socket path for device implementation to connect to. For security reasons, it is recommended to run the scripts and programs implementing devices on the same device as vdcd itself. However for development purposes the --externalnonlocal command line option can be specified to allow device API connections from non-local clients.

The plan44.ch Digital Strom products P44-DSB-xx and P44-DSB-xx support the external device API from version 1.5.0.8 onwards. Multiple devices sharing a single connection is supported from version 1.5.2.0 onwards. Single device support is supported from 1.6.0.2 onwards. However, the external device API is active by default only for P44-DSB devices enabled for "testing"/beta (available upon request), or for devices with userlevel set to 1 or higher (default for production devices is 0, can be set to 1 via remote support upon request). In the free "P44-DSB-X" plan44.ch image for RaspberryPi, the external device API is always enabled. By default, vdcd uses port 8999 for the external device API

Operation of TCP-IP connected external devices

Each external device implementation needs to

• open a connection to the TCP port or unix socket specified with the --externaldevices vdcd command line option (default is port 8999).
• optionally, send a initvdc message to adjust vdc-level data (like vdc model name and icon)
• send a init message declaring the properties of the device (specifying outputs, inputs, names, default group membership etc.). The init message uses JSON syntax. However, no JSON support is actually needed in a device implementation, because the init message can specify to use a extremely simple text protocol for any communication beyond the init message itself. And the init message is usually a constant string that can be sent easily using any programming language. (Note that for single devices, which have complex actions and events, and for some advanced features like output transition times, only the JSON syntax is supported).
• enter a loop, waiting for messages from the vdcd indicating output channel changes, or sending messages to the vdcd indicating input changes.
• When connection closes (due to error or when vdcd explicitly closes it), the device implementation should restart, see first bullet point. This can be achieved within the device implementation itself, or by having the device implementation run as a daemon under control of a daemon supervisor like runit or procd or systemd, which re-start daemons when they terminate.
• from version 1.5.2.0 onwards, multiple devices can be created on the same TCP connection. To distinguish them, each one must be assigned a tag in the init message, which must/will then be used in all further messages to/from that device.

Message Format over TCP/IP

Messages consist of strings, delimited by a single LF (0x0A) character. The init message must always be in JSON format. Further messages are either JSON or simple text messages, depending on the protocol option in the init message (see below).

Initvdc Message (only for external devices over TCP/IP)

The optional initvdc message can be sent by an external device implementation to set some vdc-global parameters:

Sending the Init Message over TCP/IP

The init message is sent by a external device implementation as the first message after opening the socket connection. It needs to be formatted as a single line JSON object.

To initialize multiple devices sharing the same connection, multiple init messages can be put into a JSON array. This is required in particular to create multiple devices that are meant to communicate via the simple text API - because after the first message, the protocol will be switched to simple text and would not allow further JSON init messages.

A simple init message for a light dimmer might look like (on a single line):

{'message': 'init','protocol':'simple','output':'light','name':'ext dimmer','uniqueid':'myUniqueID1234'}

A init message for two light dimmers on the same connection might look like (on a single line):

[{'message': 'init', 'tag':'A', 'protocol':'simple', 'output':'light', 'name':'ext dimmer A', 'uniqueid':'myUniqueID1234_A'}, {'message': 'init', 'tag':'B', 'protocol':'simple', 'output':'light', 'name':'ext dimmer B', 'uniqueid':'myUniqueID1234_B'}]

For more information, see the examples.

External device API specification

The rest of this document specifies the external device API protocol and applies to both p44script implemnented devices (where messages are exchanged via the message() function) and externally running implenmentations communicating via TCP/IP.

Init message structure

The following tables describes all possible fields of the init message JSON object:

Button object in the buttons field of the init message

Input object in the inputs field of the init message

Sensor object in the sensors field of the init message

Action object in actions field of the init message

Standardaction object in standardactions field of the init message

Configuration object in configurations field of the init message

Dynamicaction object in dynamicactions field of the init message and in the dynamicAction message

State object in states field of the init message

Event object in events field of the init message

Property object in properties field of the init message

value description fields (for action parameters, states, properties)

Messages from vdcd to device(s)

vdcd sends (depending on the features selected in the init message) the messages shown in the table below.

If devices were created with a tag (which is required when creating multiple devices on a single connection), all JSON protocol messages will include the tag field, and all simple text protocol messages will be prefixed by tag plus a colon.

Operations related to multiple device configurations are only available in the JSON protocol as follows:

Operations related to 'single' devices are only available in the JSON protocol as follows:

Messages from device(s) to vdcd

the device(s) can send (depending on the features selected in the init message) the messages shown in the table below.

If devices were created with a tag (which is required when creating multiple devices on a single connection), all JSON protocol messages must include the tag field, and all simple text protocol messages must be prefixed by tag plus a colon to identify the device.

Some more specialised operations, e.g. those relating to 'Single Devices', are only available in the JSON protocol as follows: