Ignition Vision Client Bad_Stale Reconnects: Fix One PC

Daniel Price12 min read
HMI / SCADAOther ManufacturerTroubleshooting
Licensed PE Working through this on a live machine? A Maine-licensed engineer can take it from here — included with IMD hardware, by the hour for everything else. Book an engineer

Where does the request stop?

Follow the packet. A Vision client polls its gateway on a fixed schedule. The request leaves the client JVM, passes through the Windows TCP/IP stack, the NIC driver, the physical adapter (onboard, dock, USB-C dongle, or Wi-Fi), the access switch, and any routed hops, and ends at the gateway's HTTPS listener. When one client of twelve drops out and the other eleven stay connected, the shared hops are healthy: the gateway, its listener, and the core network. The fault sits on the part of the path used only by the failing client.

The installation in this case:

Item Value
Ignition Platform 8.1.35 (b2023120517)
Vision Module 11.1.35 (b2023120517)
OPC-UA Module 9.1.35 (b2023120517)
DNP3 Driver Module 4.1.35 (b2023120517)
MQTT Engine Module 4.0.20 (b2023112216)
Gateway host Windows 10, under 5% CPU
Connected clients 12
Failing client One Windows 11 laptop (new, high-end office PC)
Healthy clients All others, on Windows 10 or Ubuntu
Symptom rate A reconnect about every other minute
Root cause found Hardware, drivers, or both on the Windows 11 laptop

Redundancy was configured but not working. That is a separate fault, covered below. It does not produce this symptom.

What does the client log show, event by event?

Read the client console log as a timeline, not as a pile of stack traces. One full cycle, reconstructed from the log:

Time Thread Log event Meaning
12:39:05.089 GatewayConnection-1 Skipping local challenge - directory doesn't exist. Informational. Part of the re-login sequence.
12:39:05.101 GatewayConnection-1 Updated login state. Logged in? true, Username: monitor The client re-authenticated and got a new session.
12:39:05.123 GatewayConnection-1 Stopping reconnect thread. Recovery from the previous drop is complete.
12:39:05.128 AWT-EventQueue-0 No session found. You must re-login. (QueryTagHistory) A sparkline history query went out on the old session.
12:39:05.318 ClientExecEngine-4 Subscription crc mismatch, will resubscribe. The client rebuilds its tag subscriptions on the new session.
12:39:40.130 ClientExecEngine-1 Connection to Gateway lost, due to exception ... Connect timed out The periodic poll could not open a TCP connection to the gateway.
12:39:40.130 ClientExecEngine-1 Changing connected quality to 'Bad_Stale' Every subscribed tag on the client goes stale.
12:39:40.133 ClientExecEngine-1 Pausing scripts... / Starting reconnect thread. Client scripts halt and the reconnect loop starts.
12:39:41.148 QueryExecuter 2 Session no longer valid. The gateway no longer recognises the old session.
12:40:47.335–.546 GatewayConnection-1 Login state updated, reconnect thread stopped, crc mismatch resubscribe The next re-login completes.

Every symptom downstream of 12:39:40.130 follows from one event: a TCP connect that did not complete. The Bad_Stale quality, paused scripts, failed history queries, and resubscription are all consequences. Fix the connect timeout and the rest goes away.

Why does "Connect timed out" point below the application layer?

Layer one first. The stack trace ends in java.net.SocketTimeoutException: Connect timed out, thrown from sun.nio.ch.NioSocketImpl.timedFinishConnect. It was reached through SSLSocketImpl.connect and HttpsClient, and was triggered by GatewayInterface.ping called from GatewayTagInterface.runPoll.

That chain tells you exactly how far the request got:

  • Connect timeout. The TCP three-way handshake (SYN, SYN-ACK, ACK) did not finish within the client's connect timeout. No TLS handshake happened, no HTTP request was sent, and the gateway application never saw the attempt.
  • Read timeout (not seen here). The TCP connection opened, the request was sent, and the gateway was slow to answer. That is the signature of an overloaded gateway, a blocked servlet thread, or a slow database behind a query.

The operating system kernel completes the TCP handshake, not the Ignition application. A gateway at under 5% CPU completes handshakes in microseconds. A handshake that never completes means SYN or SYN-ACK packets were lost or held somewhere on the path. With eleven other clients connected, that somewhere is almost always one of the following:

  • The client NIC entering a low-power state or losing link.
  • A NIC driver stall or adapter reset.
  • A dock or USB Ethernet adapter re-enumerating.
  • A Wi-Fi roam or power-save doze.
  • A filter driver (VPN client, endpoint security) holding outbound connections.
  • A faulty patch cable or switch port used only by that client.

A dead cable or wrong route stays broken. A path that disappears and returns in about a second, on a repeating cycle, behaves like a power-management or driver event on the host.

Why do sparkline charts throw "No session found" after the link returns?

A Vision client holds an authenticated session on the gateway. When the connection drops, the client starts its reconnect thread. When the path returns, the client learns the old session is gone (Session no longer valid.) and logs back in. The log shows the client re-authenticating as user monitor with Roles: [] and Security Zones: null.

Between the path returning and the re-login completing, bindings keep firing on their poll schedule. Each sparkline in the log is bound to a tag history query:

  • The binding polls every 5000 ms (@5000ms).
  • Example path: prov:default:/tag:sensors/cloud/oven_blue/tempc.

Any query sent in that window goes out on the dead session. The gateway rejects it with No session found. You must re-login. (QueryTagHistory). The stack trace runs through TagHistoryCache, SubCache, and StandardCacheBucket, so the client-side history cache tried to fill a bucket and the gateway call failed.

These errors are secondary. Do not chase them in the history provider, the database, or the chart configuration. The same applies to Subscription crc mismatch, will resubscribe.: the client detects that its subscription set no longer matches the gateway's view after the new session and rebuilds it. That is normal recovery behaviour.

Is the red client diagnostics graph the cause or a consequence?

The client diagnostics Performance tab turned red at the same moments the log recorded reconnects. The decision rule is order in time:

Observation Interpretation Next check
Red appears at or after the Connect timed out timestamp Consequence. Poll latency spikes because the request could not connect. Continue on the network and hardware path.
Client CPU or memory climbs for several seconds before the timeout Possible local cause, such as JVM memory pressure or host CPU starvation stalling the poll thread. Check Task Manager and the client memory graph against the same timestamps.
Red with no Connect timed out in the log Slow responses, not lost connections. Look at gateway-side query and script load instead.

A connect timeout comes from the socket layer. Even a starved JVM would normally show a read or response delay, not a failed handshake. Treat the red trace as a consequence unless the resource graphs lead the timeout.

Does the broken redundancy explain the reconnects?

No. Two separate observations get conflated here:

  • The master shows activity level "Active". That is the normal state of a healthy master. It does not mean a failover occurred. Nothing in the client log shows a switch to a backup address; every reconnect returns to the same gateway.
  • Clients do not list a backup gateway, and a forced failover leaves clients dead because the backup does not respond. This is a real redundancy fault. Clients have no backup address to move to, and the backup gateway is not serving. It belongs with whoever owns the redundancy configuration (here, the integrator).

Redundancy misconfiguration affects every client equally. It cannot make one laptop out of twelve lose its TCP handshake every other minute while the master stays Active. Fix it on its own schedule, and keep it out of this diagnosis.

Which fixes are on the table, and how do they compare?

Approach Targets the connect-timeout mechanism? Blast radius Effort Fit for this symptom
Raise client timeouts or change poll behaviour in the project No. It widens tolerance, so short outages go unnoticed, but packets are still lost. Every client of the project Low Masks the fault. Longer outages still go stale, and failure detection gets slower for healthy clients.
Repair gateway redundancy No. The master is Active and never failed over. Whole system Medium, integrator-owned Needed, but separate.
Tune or restart the gateway No. The gateway is at under 5% CPU and serves eleven clients without drops. Whole system Low to medium Poor. Disrupts healthy clients for no gain.
Reinstall the Vision client launcher or clear its cache Only if the client software stack is faulty. The exception is raised in the OS socket layer. One client Low Secondary. Try after hardware isolation if nothing else changes.
Isolate client hardware, NIC driver, adapter, and power settings Yes. It acts on the hop where the handshake is lost. One client Medium Root cause here: hardware and/or drivers on the new laptop.

Why isolate the client hardware first?

Isolate the client hardware first because the evidence pattern points there:

  • Only one host fails.
  • It is the only host of its kind: a new Windows 11 laptop.
  • The failure is a TCP connect timeout, so the application never received the attempt.
  • The gateway is idle, and the other eleven clients hold their sessions.

Swap tests on that host split the path into halves quickly, and each test needs no change to the production gateway or project.

The OS difference (Windows 11 against Windows 10 and Ubuntu) is a correlation, not the mechanism. Here the cause traced to the laptop's hardware and drivers, not to Windows 11 as such. New high-end laptops commonly ship with the following, all of which can drop a connection attempt for a second or so:

  • Aggressive NIC power management.
  • Docking stations or USB-C Ethernet adapters with their own firmware.
  • Modern standby behaviour.
  • OEM network utilities or security agents.

How do I isolate the fault on the client?

Collect timestamps first, then change one variable at a time. Keep the Vision client running during every test so its log records reconnects on the same clock as your probe.

  1. Confirm the gateway port. The stack shows HTTPS (HttpsClient, SSLSocketImpl). Read the host and port from the gateway address this client launches against, in the Vision client launcher configuration. Do not assume a default.
  2. Run a timestamped TCP connect probe from the laptop to that host and port (script below). A failed TCP connect in the probe at the same second as a client Connect timed out confirms the problem is below Ignition.
  3. Swap the physical path. Plug a known-good Windows 10 client into the laptop's exact cable, dock, and switch port, then run the Vision client there. Plug the laptop into a port and cable that a healthy client uses. If the fault follows the laptop, the cable, port, and switch are cleared.
  4. Swap the adapter. If the laptop connects through a dock or USB-C Ethernet dongle, connect straight to the onboard NIC, or the reverse. If it is on Wi-Fi, run the test wired. A fault that disappears with one adapter identifies the hardware or driver at fault.
  5. Read the Windows System event log at the recorded timestamps. Look for network adapter disconnect, reset, or link-state events from the NIC driver, and for power-state transitions. Events that line up to the second with the client reconnects close the case on the adapter.
  6. Disable NIC power saving. In Device Manager, open the adapter's Power Management tab and clear the option that lets Windows turn the device off to save power. On the Advanced tab, disable energy-efficient Ethernet and similar power-saving properties; names vary by NIC vendor, so read the list the driver exposes. Set the Windows power plan so the network adapter is not throttled while plugged in.
  7. Update or roll back the NIC and dock drivers and firmware using the laptop manufacturer's packages, not only the generic driver Windows Update installs. Apply the dock firmware if a dock is in the path.
  8. Test filter drivers. If site policy allows, temporarily disconnect the VPN client or pause the third-party network filter or endpoint agent and repeat the probe. A filter that holds outbound SYNs produces exactly this connect-timeout signature.
  9. Reinstall the Vision client launcher only if the probe shows clean TCP connects while the Vision client still logs Connect timed out. That split points back into the client software stack.

What probe script gives a timeline to match against the client log?

Run this in PowerShell on the failing laptop. It opens a raw TCP connection to the gateway's HTTPS port about once a second and logs the result with milliseconds. Replace the host and port with the values from step 2.


Read the result against the client log:

Probe result at a client reconnect Location of the fault
tcp=False icmp=False The link or adapter went away: NIC power state, driver reset, dock, Wi-Fi, or cable.
tcp=False icmp=True Layer three is up but TCP to that port is blocked or held: a filter driver, host firewall, or security agent.
tcp=True icmp=True throughout The OS path is clean. Look at the Vision client JVM and launcher on that host.

It is not the Ignition client's connect timeout.

How do I verify the fix?

  1. Search the client console log over the same window. Pass when there are no Connect timed out, no Changing connected quality to 'Bad_Stale', no Starting reconnect thread., and no No session found. You must re-login. (QueryTagHistory) entries.
  2. Watch the client diagnostics Performance tab for the same period. Pass when there are no red excursions.
  3. Undo the one change that made the difference (re-enable the power-saving option, go back to the old adapter, or reinstate the old driver) for a short test window. Confirm the drops return, then reapply the fix. A reproducible on/off result proves the root cause. Record that setting in the laptop build standard so a reimage or driver update does not bring it back.

FAQ

How do I tell if Ignition Bad_Stale on a Vision client is a client or gateway problem?

Check how many clients drop, and check the exception text. If only one client drops while others on the same gateway stay connected, and its log shows Connect timed out (a TCP handshake failure) rather than a read or response timeout, the fault is on that client's network path or hardware. A gateway at low CPU completes TCP handshakes regardless of application load.

How do I stop "No session found. You must re-login. (QueryTagHistory)" errors on sparkline charts?

Stop the connection drops that invalidate the session. These errors come from history bindings (polling at @5000ms in this case) firing on the old session between the path returning and the client re-login completing. Once the client stops logging Connect timed out, the history query errors stop with it.

How do I test the TCP connection from a Vision client PC to the Ignition gateway?

Run a timestamped TCP connect loop from the client to the gateway's HTTPS host and port, taken from the launcher configuration, alongside an ICMP ping. Then match failures to the client log timestamps. TCP and ICMP failing together points to the link or adapter; TCP failing alone points to a filter driver or firewall on the host.

Does a broken Ignition redundancy setup cause Bad_Stale on only one client?

No. A master showing activity level "Active" is normal and means no failover occurred. Redundancy faults, such as clients showing no backup gateway or a backup that does not answer on forced failover, affect every client equally, so fix them separately from a single-client reconnect problem.

Back to blog