An Ignition 7.8.0 Gateway polling remote PLCs through TOP Server 5.19 can keep its OPC UA session in a degraded state after the site VPNs drop and reconnect. Tags stay bad or stale even though the Gateway and TOP Server share a LAN that never went down. Disabling and re-enabling the OPC connection in Gateway Config restores data immediately. That behavior points to the Ignition OPC UA client, and 7.8.1 contains OPC UA client fixes that cover this failure.
What path does a tag read take from the Gateway to a remote PLC?
A tag value crosses two independent legs, and only one of them failed. The UA session runs over the LAN leg. The VPN leg sits behind the driver inside TOP Server.
| Hop | From | To | Transport | Affected by VPN outage? |
|---|---|---|---|---|
| 1 | Ignition Gateway (OPC UA client) | TOP Server UA endpoint | OPC UA binary over TCP, local network | No, the link stays up |
| 2 | TOP Server UA server layer | TOP Server driver channel | Internal to the server process | Indirectly, because reads wait on the driver |
| 3 | Driver channel | Remote PLC | Site VPN tunnel | Yes, drops every 7-14 days in this installation |
TOP Server is Kepware KEPServerEX licensed through Software Toolbox. The same failure appears on Ignition 7.8.0 connections to KEPServerEX, and everything below applies to both.
Check 1: Is the Gateway-to-TOP Server LAN leg actually healthy?
Rule out hop 1 first so you do not chase a network fault that does not exist.
- Ping the TOP Server host from the Gateway host. Then confirm a TCP connection to the UA endpoint port with any port-test tool.
- Open the OPC connection status page in the Gateway web interface and read the reported state.
- Open the Gateway OPC Quick Client and try to browse the TOP Server connection.
| Reading | Meaning | Next check |
|---|---|---|
| Ping or TCP fails | Real LAN or firewall fault on hop 1 | Fix the network before anything else |
TCP succeeds, status shows Connected, browse works, tags good |
No fault right now | Wait for the next VPN event and capture logs |
TCP succeeds, status shows Connected, browse fails or hangs, tags bad or stale |
Transport is fine but the UA session is broken | Check 2 |
The failing installations match the third row. The Gateway reports the connection as connected, the Quick Client lists it but cannot browse into it, and tags sit at bad or stale quality. A "connected" label means only that the TCP socket or secure channel exists. It does not prove that requests on that channel are getting answers.
Check 2: Which log signature appears after the VPN comes back?
Filter the Gateway log to the UaTcpClientSymmetricHandler logger and to channel-state messages. Note the timestamps and compare them against the VPN drop and restore times.
| Log line | What the client is reporting |
|---|---|
Error decoding Symmetric Message |
A message chunk arrived on the secure channel, and the client could not parse or decrypt it under the current symmetric security state. The client's view of the channel no longer matches the server's. |
No UaRequestFuture for requestId=N |
A response arrived for a request ID the client no longer tracks. Usually the client already timed out that request and discarded its pending future, and the server answered late. |
Channel went Inactive (GatewayIPAddr:port -> TopserverIPAddr:port) |
The client-side TCP channel closed. A reconnect and session re-establishment should follow. |
Here is how a remote outage produces these errors on a local session. When a VPN drops, the driver channels serving that site stop getting PLC replies. Each device request then waits out its driver timeout and retries. UA Read requests and subscription publishes that touch those items take longer to answer. The Ignition client applies its own request timeout. When that timer expires first, the client drops the request and later receives a response it cannot match, which logs the No UaRequestFuture line. Under enough load the channel state falls out of step and the client closes it.
A healthy client then reconnects, recreates or transfers the session, and rebuilds its subscriptions and monitored items. On 7.8.0 the client does not return to a clean state. The same errors keep repeating after the VPNs are back, and data never resumes.
| Log pattern | Interpretation | Next check |
|---|---|---|
| Errors only during the VPN outage, stopping once the VPN restores | Normal timeout behavior, and the client recovered | Tune timeouts only if the noise matters |
| Errors start with the VPN drop and keep repeating after restore | Client session did not recover | Check 3 |
| Errors unrelated to VPN events | A different fault on hop 1 or in the server | Capture a trace on the LAN leg |
Check 3: Does TOP Server itself see the PLCs again?
Next, find out whether the stall is on the server side of hop 1 or the client side. Take this reading on the TOP Server host, not through Ignition.
- Open the TOP Server/KEPServerEX runtime event log and confirm the devices on the affected channels reconnected after the VPN restore.
- Launch the server's own OPC Quick Client on the TOP Server machine. Read the same tags Ignition shows as bad or stale.
- Check that values update and that the quality reads good.
| Server-side reading | Meaning | Next check |
|---|---|---|
| Devices still in error, tags bad in the server's Quick Client | The driver or VPN leg has not recovered. The fault is on hop 3. | Troubleshoot the VPN, the PLC, or the driver channel |
| Devices good, tags good locally, but Ignition still bad or stale | Server recovered, and the UA client session is the problem | Check 4 |
Check 4: Does disabling and re-enabling the connection clear it?
In Gateway Config, open the OPC connections list, disable the TOP Server connection, save, then enable it and save again. This tears down the client's channel, session, and subscriptions and builds them from scratch.
| Symptom | Likely cause | Resolving branch |
|---|---|---|
| Data returns immediately after disable/enable | Ignition 7.8.0 OPC UA client state stuck after channel loss | Upgrade to 7.8.1 |
| Data does not return after disable/enable, server Quick Client good | Certificate, security policy, or endpoint mismatch introduced separately | Re-check the endpoint and trust settings on both sides |
| Tags STALE rather than BAD, Quick Client lists the connection but cannot browse | Same stuck-session condition, seen on KEPServerEX connections | Upgrade to 7.8.1 |
| Recovery works without intervention, but outages are long | Driver timeouts and retries on dead sites delaying the whole server | Isolate sites per channel (see next section) |
An immediate recovery on toggle confirms that nothing on hop 1, hop 2, or hop 3 needs repair at that moment. The client just needed its session rebuilt, and that is the defect 7.8.1 addresses.
How do you keep a dead site from stalling the server until the upgrade lands?
In 7.8.0 there is no scripting function to restart an OPC connection. The Gateway's OPC connection Enabled status tags are read-only, so you cannot write to them from a PLC heartbeat alarm or a timer script. Until you upgrade, the toggle stays manual. Reduce how often you need it by shortening how long a dead site holds up the server.
- Give each VPN site its own driver channel. In KEPServerEX and TOP Server, devices on one channel are polled in series. A dead device on a shared channel delays every other device on that channel. With one channel per site, a single VPN outage stays contained to that site's items.
- Enable auto-demotion on remote devices. After a set number of failed attempts, the driver stops polling a non-responsive device for a period. It then tries again instead of burning a full timeout-and-retry cycle on every scan. This cuts the time UA requests wait on dead sites. Read the available demotion settings in the device properties of your driver.
-
Review device timeout and retry counts. Timeout multiplied by retries sets how long each failed read blocks its channel. Compare that product against the Ignition connection's request timeout. When the server routinely answers slower than the client waits, you get the
No UaRequestFuturepattern. - Keep the heartbeat alarms as the trigger for manual action. When every site heartbeat recovers on the TOP Server side but Ignition tags stay bad, toggle the connection. Checks 3 and 4 give the on-call engineer a two-minute runbook.
How do you upgrade to 7.8.1 and prove the session recovers on its own?
- Take a Gateway backup from the Gateway Config backup/restore page before you change versions.
- Install Ignition 7.8.1 over the 7.8.0 Gateway with the standard upgrade installer, then confirm the version on the Gateway status page.
- Confirm the TOP Server OPC UA connection comes up, the Quick Client browses into it, and all site tags read good.
- Leave the TOP Server channel and device settings unchanged for the first test so the result isolates the client fix. Apply channel isolation and auto-demotion afterward.
- Plan a controlled outage on one site. Take its VPN tunnel down, or block its route at the firewall, for longer than the driver's full timeout-and-retry cycle.
- During the outage, confirm the Gateway log may show
UaTcpClientSymmetricHandlertimeout messages. Confirm that tags on the other sites keep updating. - Restore the tunnel. In the server's own Quick Client, confirm the site's devices recover and tags read good.
- Watch the Gateway log.
No UaRequestFutureandError decoding Symmetric Messagelines must stop after the restore, not keep repeating. - Without touching the connection's enable setting, confirm that the site's Ignition tags return from bad or stale to good and that the PLC heartbeat alarm clears. If they recover on their own, the upgrade resolved the fault. If they stay bad until a disable/enable, capture the Gateway log from drop to restore and open a case with Inductive Automation support, noting both versions: Ignition 7.8.1 and TOP Server 5.19.
FAQ
Can I restart an Ignition 7.8 OPC UA connection from a script or tag write?
Not in 7.8.0. No scripting function restarts an OPC connection, and the Gateway's OPC connection Enabled tags are read-only. The only reset is disabling and re-enabling the connection in Gateway Config, and upgrading to 7.8.1 removes the need for it.
Does TOP Server behave differently from KEPServerEX with Ignition OPC UA?
No. TOP Server is Kepware's KEPServerEX licensed through Software Toolbox. The same Ignition 7.8.0 stuck-session behavior, with No UaRequestFuture log floods and tags that stay bad or stale, shows up on both, and the same 7.8.1 fix applies.
Does a STALE tag quality in Ignition mean the PLC is offline?
Not necessarily. STALE means the Gateway stopped receiving updates. If the server's own Quick Client shows the PLC tags good, the fault is in the OPC UA client session rather than the PLC or VPN, and a connection disable/enable or the 7.8.1 upgrade clears it.