Atlas Copco Open Protocol: Fix Split MID 0061 Results

Here is how the fault shows up. Short messages from the tightening controller parse cleanly every time: communication start, program select, keep-alive, socket selector, all under 100 characters. The 386-character tightening result (MID 0061) sometimes arrives whole and sometimes in pieces. Your log shows 256 characters and then the rest. On another cycle you get 64, 128, 192 and 2. On a third you get 320 and 66. Your script never builds a valid result, so it never sends the acknowledgement (MID 0062). The controller waits several seconds, resends the result, and the station loses cycle time on every bad receive.

You have not lost data. Your receive loop is the fault.

Skip the Fixes That Don't Work

Most engineers try one of these first. None of them fixes the problem.

• Raising the recv() buffer size. The loop already calls client_socket.recv(1024), and 1024 is larger than 386. The size argument is only a maximum. recv() returns whatever bytes the OS has buffered when it is called, which can be 1 byte or the full frame. A larger number changes nothing.
• Proceeding once more than N characters arrive. One common heuristic is "if I have more than 220 characters, the result data is in there, so parse it." It works until a split lands before character 220. It also leaves the tail of the frame in the socket, and the next read treats that tail as the start of a new message.
• Adding a sleep before recv(). A delay makes a complete frame more likely to be sitting in the buffer. It does not guarantee it, and it adds latency to every message, including keep-alives.
• Relying on the controller's resend. The controller does resend an unacknowledged result, so no result is lost. The resend takes several seconds, though, and that delay is the cycle-time loss you are trying to remove.
• Decoding each chunk and passing it straight to the parser. This is the loop you have now, with if data: followed by decode() and process_data(). Every fragment is processed as if it were a complete message and then discarded.

Understand Why the Result Frame Splits

Start here: TCP is a byte stream. It does not preserve message boundaries. The controller writes one Open Protocol frame. The network stack can deliver it to your socket in any number of pieces, and each recv() call returns whatever has arrived so far.

• Short frames (under about 100 bytes) nearly always travel in one segment and land in the buffer together. That is why only the long message fails.
• The 386-byte MID 0061 frame is large enough that segmenting, write timing on the controller side, and the moment your thread wakes up all affect how many bytes are waiting.
• The reverse also happens. Two frames sent close together, such as a keep-alive followed by a result, can arrive in a single recv(). A one-read-per-message parser then treats two frames as one.

An application protocol on TCP needs its own framing. Open Protocol provides it: the first 4 bytes of every frame are the frame length in ASCII digits. That length does not count the NULL terminator byte at the end of the frame. The MID sits in bytes 4 to 7, which is the slice your code already checks for keep-alive 9999.

So the correct read is always: 4 bytes of length, then keep reading until you hold length + 1 bytes in total. After that, parse.

Match the Symptom to the Cause

Read Frames by the Length Header

This loop reads exactly one frame at a time and never over-reads into the next one. It is written for Ignition's Jython 2.7 scripting runtime, where recv() returns a str. Open Protocol frames are ASCII, so no decode step is needed. If you port it to CPython 3, build a bytes buffer and decode it once the frame is complete, not chunk by chunk.

def recv_exact(sock, n):
    # Block until exactly n bytes are read, or the peer closes
    buf = ''
    while len(buf) < n:
        chunk = sock.recv(n - len(buf))
        if not chunk:
            raise IOError('controller closed the connection')
        buf += chunk
    return buf

def get_message(client_socket, controller_id):
    try:
        while True:
            header = recv_exact(client_socket, 4)      # ASCII length
            if not header.isdigit():
                raise ValueError('bad length field: %r' % header)
            length = int(header)                       # excludes NUL
            frame = header + recv_exact(client_socket, length + 1 - 4)
            msg = frame.rstrip('\x00')                 # drop terminator
            mid = msg[4:8]
            if mid != '9999':                          # skip keep-alive logging
                print('Rx {}: {} chars - {}'.format(controller_id, len(msg), msg))
            process_data(msg, controller_id)
    except Exception as e:
        print('I/O error {}: {}'.format(controller_id, e))
        # close the socket here and hand off to your reconnect routine

Deploy it in this order:

1. Replace the single recv(1024) and if data: block with the header-then-body read above. Use it for every MID, not only 0061, so the stream stays aligned.
2. Remove the "more than 220 characters" heuristic and any similar length guesses from process_data(). Every frame the parser receives is now complete.
3. Remove any sleep you added before recv(). The loop blocks exactly as long as it needs to.
4. Send MID 0062 only after process_data() has parsed a full 0061 frame. An early ack on a partial frame means you acknowledged data you never received.
5. Run one reader loop per controller socket in its own thread. recv_exact() blocks, and one slow controller must not stall the others.

Harden the Reader Before It Hits Production

The core framing fix is small. These are the edge cases that cause trouble later.

• Disconnects. A zero-length recv() means the controller closed the socket. A bare break in an inner loop only exits that loop, and int() then runs on a half-filled buffer. Raise an exception, as recv_exact() does, so the whole read aborts cleanly.
• Socket timeouts. If you set a timeout so the thread can check a stop flag, a timeout in the middle of a frame must not throw away the bytes already read. Either keep the partial buffer across retries, or treat a mid-frame timeout as a desync and reconnect.
• Desync recovery. Once you have discarded a fragment, the next 4 bytes are no longer a length field. Don't try to scan for the next valid header. Close the socket, reconnect, and repeat your communication start and subscriptions. The controller resends any unacknowledged result.
• Chunk-wise decoding. Calling decode('utf-8') on each chunk works only by luck on ASCII data. With a multi-byte character it fails as soon as a split falls inside that character. Decode once, on the complete frame.
• NULL terminator. The length field excludes it, but it is still in the stream. If you don't consume that extra byte, it becomes the first byte of the next "length" field and every later read is misaligned.

Verify the Fix Under Load

A single good cycle proves nothing. The original code produced correct frames some of the time too.

1. Log three values for every frame: the header length, the actual received length (before stripping the NUL), and the MID. Every line must show received == length + 1.
2. Run a long burst of tightenings. Deliberately bad results (NOK) are the fastest way to generate MID 0061 traffic because you don't have to complete a good joint each time. Every result frame should log the full 386 characters.
3. Repeat with OK tightenings and exercise every other MID you use (program select, socket selector, keep-alive) to confirm that the framing holds across message types.
4. Watch for result resends. With 0062 going out immediately after each complete 0061, the controller should never resend, and the multi-second gap between tightening and ack should disappear.
5. Pull the Ethernet cable or power-cycle one controller during the test. The reader should log a closed connection, reconnect, and resume without affecting the other controllers' threads.

Know Which Fixes Waste Your Time

• Wireshark captures showing the 0061 frame split across TCP segments don't point to a controller or switch fault. Segmenting is normal, and the reader has to handle it.
• Nagle and settings change how the sender batches writes. You can't control the controller's socket options from Ignition, and changing yours doesn't reassemble incoming data.
• Controller firmware updates and message-format changes are unnecessary for this symptom. The frame already carries its own length. Use it.

FAQ