Simulating OPC UA Faults: Client and Server Test Methods

1. Overview and Scope

OPC UA (IEC 62541) communication is deterministic in spec but rarely so in the field. Production networks drop packets, switches fail, controllers reboot mid-session, certificates expire unnoticed, and firewalls silently close idle TCP sockets. A client that behaves correctly against a healthy server can still fail catastrophically the first time a wire is pulled. Simulating these faults before deployment is the only way to produce code that survives a real plant.

This reference covers practical fault injection for OPC UA across four layers:

• Network/transport layer – TCP RST, packet loss, latency, link down
• Session/service layer – ServerHalted, ServiceFault with Bad_* codes, timeout
• Security layer – certificate rejection, expired trust, SecurityPolicy mismatch
• Discovery layer – missing LDS endpoint, deregistered server, DNS failure

Procedures below cover OPC Foundation reference implementations, the Prosys Simulation Server, Siemens Process Simulate 13.0, and Python-based custom fault servers built on opcua-asyncio. Wherever the source material leaves status code values ambiguous, the article labels the ambiguity and points back to the OPC UA specification instead of inventing values.

2. OPC UA Communication Stack Recap

Fault injection must target the right layer or the test is meaningless. A "ping fail" proves nothing about an OPC UA subscription channel; a Bad_ConnectionClosed at the session layer requires a different test rig than a TCP ECONNRESET at the transport layer.

Verify port assignments and SecurityPolicy names against the OPC UA specification reference pages on the OPC Foundation site:

• OPC UA Part 4 – Services
• OPC UA Part 6 – Mappings
• OPC UA Part 7 – Profiles

The Prosys OPC UA Simulator documentation and the OPC Foundation GitHub repositories provide working examples for every layer.

3. Fault Taxonomy and Acceptance Targets

Each fault must be mapped to a measurable client response. A client that ignores Bad_ServerHalted and never reconnects is as broken as one that crashes on a TCP reset. Define acceptance targets in advance.

4. Network-Level Fault Simulation

Network-level faults are the simplest and the most realistic. They break the transport without any cooperation from the server, so they exercise the client's reconnect logic exactly as a plant failure would.

4.1 Virtual Machine Disconnect (Cross-Platform)

The lightest-weight approach is to run the OPC UA server and client on separate virtual machines and toggle the virtual NIC. When the cable goes away, the client must reconnect once it returns.

1. Install the OPC UA server (Prosys Simulator or your own) on VM-A.
2. Install the OPC UA client under test on VM-B.
3. Establish a baseline session and subscription. Confirm ServerState = Running.
4. On VM-A, disable the virtual NIC (VMware: Edit > Virtual Network Editor > Disconnect; Hyper-V: Get-VMNetworkAdapter -VMName OPCUA-Server | Disconnect-VMNetworkAdapter; VirtualBox: VBoxManage controlvm OPCUA-Server setlinkstate off).
5. Observe client behavior for the configured keep-alive interval plus grace.
6. Reconnect the NIC. Confirm the client re-establishes session and resubscribes.

This method captures TCP RST/FIN, keep-alive timeouts, and full link loss. It cannot inject packet-level impairments such as reordered or corrupted frames.

4.2 Linux tc/netem for Latency, Loss, and Corruption

When you need sub-RTT impairment (jitter, 1% loss, 200 ms latency), use the Linux kernel's netem queuing discipline. It runs on the server host or on a dedicated bridge VM between client and server.

# Add 200 ms one-way delay and 50 ms jitter on port 4840 outbound
sudo tc qdisc add dev eth0 root netem delay 200ms 50ms

# Add 1% packet loss
sudo tc qdisc change dev eth0 root netem delay 200ms 50ms loss 1%

# Add 0.1% corruption
sudo tc qdisc change dev eth0 root netem delay 200ms 50ms loss 1% corrupt 0.1%

# Capture statistics
tc -s qdisc show dev eth0

# Remove rule
sudo tc qdisc del dev eth0 root

netem is ideal for testing publish-interval tolerance, monitored-item sampling jitter, and keep-alive behavior under degraded links. Confirm tc support in the kernel with tc qdisc show before relying on it in CI.

4.3 Windows Firewall Port Block

On a single Windows host, block port 4840 with the Windows Defender Firewall to simulate a network ACL change mid-session.

netsh advfirewall firewall add rule name="OPCUA-Block-4840" dir=in action=block protocol=TCP localport=4840

# Observe client behavior; remove the rule to restore
netsh advfirewall firewall delete rule name="OPCUA-Block-4840"

This is the closest simulation to an industrial firewall policy applied between zones (for example, the cell network blocking the control network on the wrong port). It will not trigger an immediate TCP RST; the client observes timeouts instead. Make sure the test client has a timeout shorter than the firewall's idle session reaper, otherwise the socket stays "half-open" for hours.

4.4 Disabling the Network Adapter

The bluntest tool. Disable the adapter in ncpa.cpl or via PowerShell:

Disable-NetAdapter -Name "Ethernet0" -Confirm:$false
# ... test window ...
Enable-NetAdapter -Name "Ethernet0"

This triggers NDIS-level link loss. It is the strongest test of a client's "I lost my network entirely" path – including its ability to come back when the adapter is re-enabled with a new DHCP lease.

5. Application-Level Fault Injection

Network faults prove the transport path. Application-level faults prove the client reads StatusCode results and reacts correctly. These require a server you control – the OPC Foundation Sample Server or a custom script.

5.1 ServiceFault with Bad_* Status Codes

Any OPC UA service request can return a ServiceFault with a ResponseHeader.ServiceResult carrying a status code. Inject a fault by configuring the server to return a chosen code on the next request, then measure the client.

Common codes to inject:

• Bad_CommunicationError – generic transport-level failure surfaced to the application
• Bad_Timeout – operation took too long server-side
• Bad_ServerHalted – server is shutting down gracefully
• Bad_OutOfService – node is intentionally disabled for maintenance
• Bad_UserAccessDenied – authentication valid but authorization failed
• Bad_EncodingLimitsExceeded – message size or array bounds violated

Clients must distinguish a Bad_* return on the ResponseHeader (request never executed) from a Bad_* in the body (request executed, individual value bad). The Siemens SIMATIC client libraries and the OPC Foundation .NET Standard stack expose both on ServiceResult and per-attribute StatusCode.

5.2 Server Halted and ServerState Transitions

An OPC UA server transitions through ServerState values of Running, ShuttingDown, NotReachable, and Halted. The graceful-shutdown sequence is: server sends a ServiceFault with Bad_ServerHalted on outstanding requests, sends a final CloseSecureChannel, then exits. A well-behaved client observes the ServiceFault, reads Server_ServerStatus_State via a fresh read if possible, schedules a reconnect, and re-issues all subscriptions with the same client handle on the new session.

Test rig: trigger a server shutdown, observe that the client logs Bad_ServerHalted (not CommunicationError), and confirm the subscription handles still map to the correct monitored items after reconnect.

5.3 Discovery Failures

Local Discovery Server (LDS) failures are common in plants where DNS or multicast is restricted. To inject:

1. Stop the LDS service (net stop "OPC Foundation Discovery Service" on Windows, or kill the lds process on Linux).
2. Confirm the client's discovery client falls back to a cached endpoint or surfaces an explicit discovery error.
3. Restart LDS and confirm re-registration.

The OPC UA specification requires an mDNS announcement on UDP 5353; on networks where mDNS is filtered, only well-known static endpoint URLs work. Build that requirement into the test.

6. OPC Foundation Sample Server and Client

The reference implementations from the OPC Foundation are the safest starting point. They are free, signed by the OPC Foundation, and exercise every layer including reverse connect and HTTPS endpoints. The UA-.NETStandard repository includes:

• Quickstarts.SampleServer – console server exposing a few hundred nodes
• Quickstarts.SampleClient – console client with browse, read, subscribe
• ReferenceServer – full-featured server with security and LDS support

6.1 Build and Run

1. Clone the repository and checkout the latest tagged release (verify the tag against the OPC Foundation release notes).
2. Open UA .NET Standard.sln in Visual Studio 2022 or build with dotnet build from the command line.
3. Run SampleServer on one machine (or as one process) with default configuration: opc.tcp://localhost:4840.
4. Run SampleClient, discover opc.tcp://localhost:4840, and click Connect.

6.2 HTTPS Endpoints for Fault Scope

The OPC Foundation UA-.NETStandard issue #411 documents successful Sample Client / Sample Server communication over HTTPS endpoints. Use HTTPS endpoints to test TLS-layer faults separately from binary-channel faults: certificate rotation, cipher suite mismatch, and client-cert revocation.

6.3 Reverse Connect

Both Sample Server and Sample Client support reverse connect, where the server initiates the TCP connection back to the client. This is the only way to traverse a strict outbound-only firewall. Faults to inject:

• Client listener down at startup – server should retry on backoff
• Client listener down mid-session – server should reconnect; client must reconcile its session list
• Server behind NAT – tests STUN-like discovery behavior

7. Prosys OPC UA Simulation Server

The Prosys OPC UA Simulation Server is a commercial (free demo available) Windows application with hundreds of pre-built nodes, full security configuration, and a scripting interface. It is widely used in Siemens, ABB, and Rockwell test rigs because it models realistic process variables rather than flat counters.

7.1 Configuration for Fault Injection

1. Install the simulator and license it (trial license covers all features for 30 days).
2. Open Simulation window and confirm Simulation.Simulation.Servers shows the local endpoint.
3. In Server Settings > Endpoints, add a second endpoint on a non-standard port (for example 4841) so the test rig can toggle it.
4. Use the built-in Errors simulation: right-click a node, choose Simulate > Error, and select a status code from the dialog. The simulator will return that code for the next read or subscription sample.
5. To simulate a server halt, choose File > Stop Server. The simulator announces ServerState = ShuttingDown on outstanding sessions and then closes the channel.

7.2 Connection Fault Mode

Prosys Simulator supports dropping the underlying TCP socket at configurable intervals. Set Server Settings > Advanced > Force Disconnect Every (seconds) to a value just greater than the client's keep-alive interval. The test client must distinguish this from a network-side disconnect: both produce a TCP RST at the client, but only the application-level fault triggers a Bad_ServerHalted on the in-flight service.

7.3 Compatibility with Siemens Toolchain

Prosys Simulator exports its endpoint URL and a self-signed certificate trusted by default in TIA Portal V17+ and WinCC Unified V18+ when added under OPC UA > Trusted Certificates. This is the recommended pairing for testing a Siemens OPC UA client without a live S7-1500 CPU on the bench.

8. Siemens Process Simulate 13.0 OPC UA Integration

Process Simulate 13.0 (part of the Tecnomatix portfolio) exposes an External Connection for OPC UA. The connection is defined under Resources > External Connections and binds a Process Simulate signal (a property, event, or operation parameter) to an OPC UA node on a remote server.

8.1 Establishing a Connection

1. In Process Simulate, open the active study or station and navigate to Resources > External Connections.
2. Click New, select OPC UA, and enter the endpoint URL of the simulator (for example, opc.tcp://localhost:4840).
3. Click Validate. Process Simulate performs a GetEndpoints, displays the available SecurityPolicies, and prompts for trust of the server certificate.
4. Map a Process Simulate property to an OPC UA node ID using the browse dialog. The node ID must be of a type Process Simulate can consume (Boolean, Integer, Real, String).
5. Save the connection. Process Simulate activates the session and starts reading or writing the node at the configured update interval.

8.2 Fault Behavior in Process Simulate 13.0

When the OPC UA server is unreachable, Process Simulate flags the connection with an error indicator (red badge) and surfaces the underlying status code in the connection properties dialog. The error indicator clears automatically when the server becomes reachable again; no manual re-import is required.

8.3 Mapping to a TIA Portal S7-1500 OPC UA Server

Process Simulate also consumes OPC UA servers exposed by the S7-1500 CPU with firmware V2.9 or later. Enable the OPC UA server in TIA Portal under CPU Properties > OPC UA Server, set the port (default 4840), and select the security policies Process Simulate will negotiate. See the S7-1500 system manual entry for "OPC UA server" on the Siemens support portal for firmware-specific limitations (for example, maximum simultaneous sessions per CPU).

9. Python-Based Custom Fault Injection with opcua-asyncio

When no off-the-shelf server exposes the exact fault you need, write one. The opcua-asyncio library on PyPI implements a server in roughly 200 lines and lets you intercept any service call to inject faults.

# fault_server.py
import asyncio
from asyncua import Server, ua

ENDPOINT = "opc.tcp://0.0.0.0:4855"
NAMESPACE = "urn:bench:opcua:fault"

async def main():
    server = Server()
    await server.init()
    server.set_endpoint(ENDPOINT)
    uri = await server.register_namespace(NAMESPACE)

    # Build a tiny address space
    objects = server.nodes.objects
    sensor = await objects.add_variable(
        ua.NodeId("Sensor.Temp", uri), "Sensor.Temp", 25.0
    )
    await sensor.set_writable()

    # Fault injection: fault_count > 0 returns Bad_OutOfService for that many reads
    fault_count = 0

    async def fault_before_read(node):
        nonlocal fault_count
        if fault_count > 0 and node == sensor:
            fault_count -= 1
            raise ua.UaStatusCodeError(ua.StatusCode(ua.StatusCodes.BadOutOfService))

    server.beforeread.add_handler(fault_before_read)

    async with server:
        print(f"Fault server running on {ENDPOINT}")
        while True:
            await asyncio.sleep(1)
            # Toggle: set fault_count = 5 to inject 5 Bad reads, then 0
            # Drive from a CLI, an env var, or a second OPC UA node for full automation
            fault_count = int(__import__("os").environ.get("FAULT_COUNT", "0"))

if __name__ == "__main__":
    asyncio.run(main())

Drive FAULT_COUNT from the test harness. The same pattern supports injection of Bad_Timeout (delay before responding), Bad_CommunicationError (drop the response), and Bad_ServerHalted (raise in before_shutdown).

For full channel-level control, fork opcua-asyncio and override the message dispatch in ua_binary_server. This is the only practical way to inject malformed chunk headers, exceeding MaxChunkCount, and other protocol-edge cases that commercial simulators do not expose.

10. Test Scenarios and Validation Matrix

Compose the methods above into named scenarios. Each scenario must have a pass/fail criterion before it runs. The matrix below is a starting point; trim or extend for the system under test.

11. Verification and Acceptance Checklist

Run this checklist at the end of every fault-injection campaign. Treat each line as a binary gate; do not promote a build with any unchecked line.

1. Confirm a baseline ServerState = Running snapshot before each fault.
2. Confirm the client logs the specific status code or socket error for the fault (not just a generic "error").
3. Confirm the client's error counter increments and is visible at the HMI/SCADA level.
4. Confirm the client does not retry indefinitely; backoff is observable.
5. Confirm the client does not crash; process remains responsive.
6. Confirm subscriptions are restored with the same client handles on reconnect.
7. Confirm certificates are not silently added to the trust store (security regression).
8. Confirm no leaked file descriptors or threads after 100 fault cycles (check with Process Explorer or lsof).
9. Confirm Process Simulate 13.0 / TIA Portal OPC UA clients flag the fault within the configured diagnostic interval.
10. Capture a packet trace (Wireshark with the OPC UA dissector) for one fault to confirm the wire-level behavior matches the spec.

12. Platform-Specific Notes

12.1 S7-1500 OPC UA Server (TIA Portal V18/V19)

The S7-1500 CPU acts as an OPC UA server natively. Key fault-relevant limits per firmware:

• Maximum simultaneous sessions per CPU (typically 20 on S7-1515, 40 on S7-1518) – verify against the CPU's data sheet for your article number.
• Maximum subscribed monitored items per session (firmware-dependent, often 500).
• Server does not gracefully halt by default; stopping the CPU yields an abrupt socket close. Test both Stop CPU and OPC UA Server > Disable paths.

Trust list management is under CPU Properties > OPC UA Server > Client Certificates. A client whose certificate is rejected produces BadCertificateUntrusted; the test rig must observe this and fail the run.

12.2 WinCC Unified / TIA Portal OPC UA Client

WinCC Unified V18/V19 uses the OPC UA .NET Standard stack and shares its reconnect logic with the rest of TIA Portal. Configure the connection's Reconnect Interval and Watchdog Timeout under the HMI tag properties. Fault tests must demonstrate the configured values are actually honored – the default 5 s reconnect is too aggressive for slow radios and too slow for a tripped breaker.

12.3 Beckhoff TwinCAT OPC UA

TwinCAT 3 exposes an OPC UA server on port 4840 with Basic256Sha256 by default. TwinCAT's reconnect behavior is driven by the TF6100 PLC library; inject faults into the Beckhoff side using the same network-level methods (Section 4) and observe that the TwinCAT client libraries restart sessions automatically.