18 KiB
Guide to implement standard interfaces
This guide is for devlopers intending to add HMIs for custom hardware or designing interfaces for network enabled devices, in particular for low spec hardware.
Checklist / Requirements
This document assumes that there are certain basics / best practices already implemented :
- Consistent calling convention
- Consistent error codes and handling across the entire program
- The programs public interface uses minimal, distinct and consistent function signatures
- Shared brokers or gateways
- Consistent and simplyfied lifecycle methods, eg: Addon::setup, Addon::loop, Addon::destroy
- Handling of IO timeouts and retry
- Message resiliance for subsequent devices / IO
This basics make further binding to a protocol or intermediate abstraction relative easy.
Protocols
This chapter is dedicated to the implmentation of popular protocols. Some of these ideally don´t require additional changes in the application and some can be supported by minimal tweaks or sufficient documentation about how to accuire those (The goal of this guide is to support all of them with minimal effort).
A minimum and but yet practical protocol should be chosen wisely and up front. A custom protocol comes often with penalties which are hard to document, communicate and to support, unless this is part of the business but as often seen, it does not pay out for the vendor. In many cases a protocol as Modbus is sufficient since there are lot's of existing client software one can refer to. It makes it easy to test the product, outside of any noise as custom client implementations. Other than software to test and being used as reference, there should be also enough off the shelf hardware to extend wire lengths but also to convert it to other protocols (there is a big market around Modbus or Canbus). The very choice has to be part of the product design and business strategy.
1. Serial (custom messaging and data structures)
The protocol (rfc1963) is being widely used for applications which need only a short cable and can be found in 3D-Printers, HiFi amplifiers and even larger machines as older CNC mills or lathes - mostly for configuration and servicing tasks. Due to its limitations in cable length (2.5m) and transport, it's not meant to build larger networks, especially in environments with electromagnetic emissions (EMI) since data transmission is realized using low voltages, prone to external EMI (5V - 12V). The protocol supports basic parity checks (YTube - Ben Eater but more reliable CRC checks have to be built upon in the application. Please see more about on YTube - Ben Eater.
Implementation
In case of a simple application, it´s relatively easy to parse incoming messages and then call the right functions but this can be fast unmanageable as soon as the API grows.
To invoke global functions or class member functions from incoming strings, eg: MyClass::MemberFunction::Argument, a registry with function pointers can be used. This method works sufficient enough as long function signatures don´t differ too much from each other since for each different signature, a function pointer has to be declared. Using reinterpret cast to cast a void* pointer (isocpp) is possible but can leave the application quickly in an undefined state. This is because some of the architectures and suited compiler features don´t support advanced casting.
Example of a class::fn registry (This is prove of concept code, don´t use this in production!)
Recommended data structure for requests
To facilitate debugging but also human readable messaging (ASCII code as GCode). The following message structure turned out to be realible and versatile
<< id,verb,flags,version,payload >>>
Encapuslating it with a prefix '<<' and suffix '>>' helps to distinguish these messages from other data on the same connection (ie: debug statements as Serial.print)
-
id: optional, as in TCP or other high level protocols, this is a sort of transaction id to track the message though it's lifecycle which enables to detect dropped or lost messages -
verb: optional, the type of the call, application specific but in more complex scenarios this designates the message type. In particular these types have been practical (security, compilation and implementation wise):
enum ECALLS
{
// global function
EC_COMMAND = 1,
// addon method
EC_METHOD = 2,
// external function
EC_FUNC = 3,
// user space (addons)
EC_USER = 10
};
flags : optional but also useful for prototyping and development. Flags which can be used to alter the response and command behaviour
enum MessageFlags
{
// local flag for the recipient
NEW = 1 << 1,
// local flag for the recipient
PROCESSING = 1 << 2,
// local flag for the recipient
PROCESSED = 1 << 3,
// sender flag, turn on debugging
DEBUG = 1 << 4,
// sender flag, instructing the recipient to send a reciept
RECEIPT = 1 << 5
};
version : optional but recommended as soon it's clear that the product may go through different revisions. This can avoid issues when backporting hotfixes, despite a firmware ideally should expose it's version and compilation configuration, via initial handshake or explicit call. Doing this on a message level also enables easier testing for different implementations per version.
payload : The actual data. In the case of bound class functions, eg: <<1;2;0;1;Power:on:1>>. It should be noted that using strings on low spec devices comes with memory problems but also often missing basic implementations of string functions. This document will explore later on more optimized invocation methods using numbers only (as seen in Modbus).
Remarks
- Serial communication can be traced and debugged on modern scopes but also with software(kernel, DLL hooking or WinUSBCap). Obfuscating can be archived by hashing but will require significant more resources and yet, it's a matter of time to tap into it.
- The concepts above are only for educational purposes. Simple applications as serial invocations of commands (GCode) can perfectly deal with simple return messages as echoing the request (as in Modbus). However, it seemed a reasonable introduction into the nuts and bolts of advanced messaging, mandatory requirements for the next chapters.
- SO : Messing with member function pointers, SE : Fishy reninterpret_cast
2. Modbus RTU over RS485 (2 wire)
To implement Modbus, please consult first the Specification and Implementation Guide from Modbus.org - PDF
The challanges begin when multiple devices have to managed on one device(often an I/O controller).
2.1. Example : Print head
This utilizes a VFD and PID controller via Modbus-RTU-RS485 and a few other components via analog or digital signals. Low level I/O handling happens ideally on the I/O controller. The 'Remote Manager' acts as client and consumes a high level interface via Modbus-TCP.
+--------------------+
| |
| Remote Manager |
| |
+---------+----------+
|
|
| Modbus RTU / or raw TCP
|
|
+---------+----------+
| |
| I/O Controller |
| |
+--+------+----------+
| |
| | Modbus RS485 (2Wire)
| |
+--------------+ | | +-----------------------+
| | | | | |
| Sensor +---+ +----+ Device Nr. 1 (VFD) |
| | | | | |
+--------------+ | | +-----------------------+
| |
+--------------+ | | +-----------------------+
| | | | | |
| Stepper +---+ +----+ Device Nr. 2 (PID) |
| | | | |
+--------------+ | +-----------------------+
|
| +-----------------------+
| | |
+----+ Device Nr. 3 (PID) |
| |
+-----------------------+
The VFD and the PIDs have their own vendor specific interfaces. The controller reads and writes those registers, relevant to the application and then exposes a new interface on Modbus TCP or plain TCP sockets.
Most Modbus implementations offer only a very slim API to query a device(slave). A query (or request) can be a read- but also a write - command, one at a time. It's the application's responsibilty to manage this queries in an ordered fashion. For this the concept of a message queue is being utilized. A message queue can be shared among different owners (classes) where all subscribers can queue new requests.
The organization of functionallity (classes and functions) should be designed optimized for it's usage. Equal devices should be grouped and managed by a parent class to assist for instance in broadcasting (eg : all on or all off). Furthermore, it's wise to consider that multiple but equal devices will be added over time.
In our example, we're using an unknown number of PID controllers and expose them in a managed interface:
class TemperatureControl : public Addon, ModbusClient {
OmronPID pids[NUMBER_OF_PIDS];
stop();{ each pids => p => pid->stop} // stop all PIDS
run(); {each pids => p => pid->run} // run all PIDS
isHeating(); // returns true if any of the PIDs is heating, used for a feedback LED on the control panel
Modbus *rs485; // the Modbus RS485 gateway
Mudbus *modBusTCP; // the Modbus TCP gateway
}
This way the OmronPID class doens't need to know anything about Modbus and acts merely as storage for values.
Printhead register design
The more important information as status, state, etc... should be easy to fetch using the READ_HOLDING_REGISTER Modbus command. The command accepts a number to specify an address range. To preserve bandwidth and avoid collisions, the very range should be kept small. Populating the first 10 registers is a common practice.
Print head holding read registers
// System Globals
- 0x0 : This is normally unused
- 0x1 : System state : RUNNING|STOPPED|POWERED|ERROR - any sub device activity will raise the flags - used for feedback and main control
- 0x2 : Last Error Code
- 0x3 : Reserved
- 0x4 : Reserved
// Sub systems or devices
- 0x5 : VFD - State : RUNNING|STOPPED|ACCELERATING|DECELERATING|ERROR
- 0x6 : PID 1 - State : RUNNING|STOPPED|HEATING|ALARM|ERROR
- 0x7 : PID 2 - State : RUNNING|STOPPED|HEATING|ALARM|ERROR
- 0x8 : PID 3 - State : RUNNING|STOPPED|HEATING|ALARM|ERROR
- 0x9 : System Sensor Flags : ENCLOSURE_OPEN|OVERHEAT|...
That's it, 10 registers with 2 bytes each - ideal to be fetched by clients frequently. Any less frequent requested details are placed in higher addresses. Those are ideally grouped in similar register pages:
// Holding registers 0xA (10) - 0xE (14) are used for the VFD details
- 0xA : VFD - Last Error Code
- 0xB : VFD - Target Frequency
- 0xC : VFD - Current Frequency
- 0xD : VFD - Current (Amperage)
- 0xE : VFD - Reserved
To gain more flexibilty about address ranges for such groups, offsets and macros can be used :
#define SYSTEM_MAIN_START 0x0
#define VFD_MAIN_OFFSET SYSTEM_MAIN_START + 10
// VFD HOLDING READ REGISTERS (0x03)
#define VFD_RREGISTER_LAST_ERROR VFD_MAIN_OFFSET + 1
Depending on the system size, these registers might go under duplicate checks preferably at compile time. For C++ there are a number of macros implemented to provide array access- at low cost. See ./examples/commons/macros.h#85. Those macros can also assist safe pin assignment.
Common used address range assignment for read registers
Read Holding Registers Poll Priority Address Ranges
+-----------------------------+------------------+------------------+
| Main System Monitors | | |
| | 1 | 0 |
| Recommended size : 10 | | |
+------------------------------------------------+------------------+
| Main System Detail Registers| | |
| | 2 | 0x10 |
| Recommended size : 10 | | |
+------------------------------------------------+------------------+
| | | |
| Sub System 1 Main Monitors | On Demand | 0x20 |
+------------------------------------------------+------------------+
| | | |
| Sub System 2 Main Monitors | On Demand | 0x30 |
+-------------------------------------------------------------------+
| | | |
| Main Advanced Level | On Demand | 0x1000 |
| | | |
+-------------------------------------------------------------------+
| | | |
| Sub System Advanced Level | On Demand | 0x2000 |
| | | |
| | | |
| | | |
+-----------------------------+------------------+------------------+
- Message queue overview
- Race conditions
- Split feedback and control via read and write registers
Example References
Remarks
- Modbus for Dummies - YT
- All You Need to Know About Modbus RTU - YT. Watch his other videos, incl. the justified rant to sell MQTT as the new IoT (aka Internet of sh** - TW).
- Serial Communication RS232 & RS485 - YT
3. Modbus RTU - TCP
4. TCP (Custom)
After implementing Modbus RTU for TCP/Serial, a custom TCP protocol can be added for clients who don't have a Modbus implementation (eg: ABB and other robot systems). This enables inheritance of error codes & message resiliance. It's becomes a matter of documentation only.
In example, our simple print-head firmware manages RS485 devices (ie: VFD & PID controllers) and proxies them via Modbus-RTU on TCP using a simple server implementation (see './examples/modbus-tcp').
To determine the exact byte sequence to be sent to the I/O controller, a simple Modbus client software can be used :
In this particular case, we're setting the target temperature of a PID controller to 100, using address 17, eg: 01 06 00 11 00 64 D8 24
01: slave id06: Modbus verb / function code, in this case WRITE HOLDING REGISTER11: address (17)00 64: value (100), 2 bytesD8 24: CRC, 2 bytes. Since it's TCP, this isn't evaluated and can be ignored, see TCP Specs
As next, a packet tracer is used to see what is actually send over the TCP wire and thus the TCP protocol overhead can determined.
The image shows where the Modbus message begins and ends. Wireshark also enables to investigate preceeding bytes and helps to understand those.
d4 95 00 00 00 06 01 06 00 11 00 64
In order to fake a Modbus message, all we need is 01 06 00 11 00 64 but we also have to prefix it with the TCP overhead (d4 95 00 00 00 06)
|-- TCP Overhead----- | -------- Modbus ---- |
d4 95 00 00 00 06 | 01 06 00 11 00 64
In example, we can send this now via Hercules to emulate a Modbus Message over TCP.
The TCP overhead (d4 95 00 00 00 06) is created as follow:
d4 95: Transaction identifier, 2 bytes00 00: Protocol identifier, 2 bytes00 06: Length of the message, 2 bytes
Viola
5. MQTT
6. Via Proxy : Profibus
7. Via Proxy : CAN
Preface
- State/register race condidtions
- Wiring (max. cable length, max. devices)
- High spec protcols & low spec devices
- External protocol proxies
- Documentation and custom firmware configuration
- User space (extension points & callbacks)
- Security
- Feeback & debugging
Abstractions
- IO state persistence across different protocols
- States
- Registers and custom address mapping
- Calling conventions (incl. user function to select responses)
- Message queues