Troubleshooting S7-400 CP443-1 Communication Delay Between PLCs

Problem Details

A SIMATIC S7-400 station built around a CPU 416-3 DP with a CP 443-1 Industrial Ethernet communications processor exchanges a motor speed setpoint with a second S7-400 acting as a slave. The slave uses the received speed to position a steam control valve. The user issues the setpoint from an HMI connected to the master.

Observed symptom:

• The setpoint written from the HMI appears at the slave's input area with a 15–30 second latency.
• The behavior is intermittent: a recent Ethernet patch-cable replacement (cable between the slave's CP 443-1 and the plant hub) appeared to fix the issue for 2–3 days, after which the delay returned with the same magnitude.
• STEP 7 / NetPro configuration shows no errors. The connection is established; the diagnostic buffer of the CP shows the connection as ESTABLISHED.
• The slave's control valve reacts to the late-arriving value, producing an oscillation in steam flow and unstable process pressure downstream.

Latency in the 15–30 s band is far above the normal cycle time of an S7-400 PUT/GET or USEND/URCV exchange (typically 50–500 ms over a healthy Ethernet segment). It almost always points to one of the following mechanisms: a transport-level retry, a TCP keep-alive gap, an application-level polling, or a programmed conditional gating of the SEND/RECV call.

Root Cause Analysis

Before changing any hardware, classify the failure by reading the status outputs of the active communication block. Per the Siemens SIMATIC Communication manual (CPU-CPU Communication with SIMATIC Controllers, document 78028908), every S7-400-to-S7-400 exchange over an ISO-on-TCP or TCP connection uses one of four block families:

For a motor speed setpoint that the master pushes to the slave, the typical implementation is one of:

1. Master: PUT block, triggered cyclically in OB1 or OB35.
2. Master: USEND + Slave: URCV pair, triggered cyclically.

All four families expose the same output pattern: DONE, ERROR, STATUS, plus the ACT (USEND) or REQ (PUT/GET) input. A latency of 15–30 s is rarely a network fault at the physical layer; it is almost always one of the following five root causes.

Cause 1 — Trigger gate disables SEND/RECV for long stretches

If the master code gates the USEND or PUT call behind a condition (e.g. A "HMI_speed_changed" in ladder, or a IF bSpeedChg THEN in SCL), the block does not actually call the CP. The 30 s gap then represents the period between manual setpoint changes, not a fault. Verify by reading the ACT/REQ input in VAT or by tracing the call in the cross-reference.

Cause 2 — TCP keep-alive gap on the CP 443-1

Per the CP 443-1 manual (SIMATIC NET S7-CPs for Industrial Ethernet CP 443-1, document 27013386), the CP supports ISO-on-TCP (RFC 1006) and TCP. When a TCP connection sits idle, the operating layer sends keep-alive probes. The default keep-alive timer on older CP 443-1 firmware (EX10, EX11, EX20) is 30 s for the first probe, with up to 9 retries at 75 s intervals — exactly matching the observed 15–30 s gap. If USEND/URCV is being used, every gap longer than 2 s can trigger a probe burst, during which data is queued.

Cause 3 — Hub / switch port mis-configuration or half-duplex mismatch

The user replaced the patch cable and saw a 2–3 day improvement. That signature is classic for a degrading SFP/port or a port that is auto-negotiating to 10 Mbit/s half-duplex. The CP 443-1 attempts to send, gets late collisions, retries, and the ISO-on-TCP timer expires — at which point the S7-400 communication layer drops the connection and re-establishes it, producing the visible 15–30 s gap.

Cause 4 — Memory overlap at the partner DB

When the slave's receive area overlaps with a region that is being written by another OB (commonly OB1 background, OB35, OB82, OB121), the value is overwritten by the OB that runs after the receive completes. The application then appears to "lose" the new value for several cycles until the next URCV arrives. Cross-reference both DBs and any BLD / BLKMOV in OB1 to confirm the receive area is exclusive.

Cause 5 — HMI polling locks the HMI connection

WinCC / ProTool / TIA Panel HMI on the master can hold the only ISO-on-TCP connection resource on the CP 443-1. The CP 443-1 (EX20) has a hard-coded limit of 64 connection resources total, of which 1 is reserved for the PG/OP and 1 for the S7 routing. If the HMI uses 4–8 tags each on its own connection (a common misconfiguration with WinCC flexible), the master has no free resource for the PUT/GET. The CP then queues the request and only services it when an HMI tag changes, producing a 15–30 s apparent delay.

Solution

Resolve the issue in the following order. Each step can be confirmed before moving to the next.

Step 1 — Confirm the connection is healthy at the CP

1. Open SIMATIC Manager > Online > Accessible Nodes and ping the slave CP from the master PG.
2. Open the master CP in NetPro, right-click the connection to the slave, select Connection diagnostics. The state must read ESTABLISHED. Record the "Bytes sent / received" counters — non-zero and incrementing values confirm the link is alive.
3. Open the slave's diagnostic buffer (CPU > Module Information > Diagnostic Buffer). Look for event IDs 0x4543 (connection interrupted) and 0x4544 (connection re-established). A pair appearing every 15–30 s confirms keep-alive drops.

Step 2 — Convert the exchange from USEND/URCV to PUT/GET (or vice versa)

If the current implementation uses USEND/URCV:

• Replace with PUT (FB 15) on the master and remove the URCV on the slave. The PUT block is acknowledged at the application layer; if the slave is unreachable, the master's STATUS output returns immediately with error 0x0001–0x000F, allowing a clean retry every cycle.
• Re-trigger PUT on every OB1 pass with a REQ := TRUE pulse driven by a 100 ms clock merker. Do not gate it on "speed changed".

If the implementation is already PUT/GET, switch to BSEND/BRCV with a small LEN := 4 for the speed value (DINT). The BSEND family uses an ISO-on-TCP handshake that is more robust to short network interruptions than PUT.

Step 3 — Tune the CP 443-1 keep-alive and watchdog

1. Open the CP 443-1 in HW Config, double-click the CP, select the Options tab.
2. Set Keep-alive time to 10 s.
3. Set Number of keep-alive retries to 3.
4. Set Connection monitoring time to 30 s.

These values are persisted in the CP's configuration; a download + power-cycle of the CP is required to activate.

Step 4 — Free connection resources on the master CP

1. In NetPro, count the connections owned by the master CP 443-1. If more than 30 active connections exist, consolidate HMI tags onto a single named connection using the S7 Communication driver (not the default Softnet driver).
2. Verify in the CP's Connection statistics online view that Free resources > 4 at all times.

Step 5 — Replace the hub with a managed switch

A shared 10/100 hub is the prime suspect given the cable-replacement history. Replace with a managed switch (e.g. Siemens SCALANCE XC-208 6GK5208-0BA00-2AC2) and force both ports to 100 Mbit/s full-duplex with auto-negotiation disabled. Confirm in the switch port statistics that 0 CRC errors, 0 late collisions, and 0 runts are observed over a 30 min window.

Step 6 — Cross-reference the slave receive DB

1. Right-click the slave receive DB in STEP 7, choose Cross-references.
2. Filter for Writes. Every entry outside the URCV call site is a candidate to overwrite the received speed.
3. Move the receive area to a DB that is read-only outside the application OB, or use a temporary ANY pointer at the start of OB1, then copy to the application DB after OB1 completes.

Step 7 — Add application-level monitoring

Add a watchdog merker in the slave that is toggled by the master every cycle. If the merker is not toggled for 5 s, the slave raises a process alarm. The alarm log entry will show the exact second the data stopped arriving, allowing direct correlation with the switch port statistics and the diagnostic buffer.

Verification

After applying the changes, run the following checks.

1. Cycle-time check. Force a step change on the HMI setpoint, then read the slave's receive DB with VAT. The new value must appear within 2 OB1 cycles (typical OB1 = 100 ms → 200 ms).
2. Diagnostic buffer. No new 0x4543 / 0x4544 entries during a 1-hour observation window with continuous setpoint changes.
3. Switch port statistics. 0 CRC errors, 0 input/output discards, line utilization < 5 %.
4. CP connection list. In the master's CP online diagnostics, the connection to the slave shows Bytes sent and Bytes received incrementing at the expected cyclic rate (e.g. every 100 ms × 4 bytes = 40 B/s).
5. Process behavior. Steam control valve tracks the setpoint with a steady-state error < 1 % and no oscillation. Plant DCS log shows no "speed late" alarms during a 24-hour shift.

Connection Configuration Reference

The reference configuration for an S7-400 to S7-400 ISO-on-TCP connection is summarized below.

The CP 443-1 always operates as the server for incoming connections when the partner uses fetch/write, as stated in the CP 443-1 manual. In an S7-to-S7 PUT/GET scenario, only one side (the master) is configured as the active partner; the slave's connection is auto-generated by NetPro and is implicitly passive.

PUT/GET Block Parameter Reference

Two failure signatures are diagnostic:

• STATUS = 0x0081, ERROR = TRUE, BUSY = FALSE: the partner CPU rejected the request. Typically an ANY pointer error or a TSAP mismatch.
• STATUS = 0x0007, ERROR = TRUE: the CP has run out of connection resources. Consolidate connections on the master (Step 4 above).

USEND/URCV Block Parameter Reference

Diagnostic Buffer Event IDs

Firmware and Catalog Reference

Cycle-Time and Latency Math

Expected end-to-end latency for a single PUT call over a healthy 100 Mbit/s ISO-on-TCP link:

• OB1 cycle: 100 ms (master)
• Block run-time: 2–8 ms (PUT)
• TCP send on CP 443-1: 1–3 ms
• Network one-way: < 1 ms (100 m Cat 5e)
• Slave OB1 receive: 100 ms
• Total: ≈ 200–215 ms

Anything above 1 s indicates an application-level or transport-level issue. 15–30 s points to a periodic retry / re-establishment mechanism, not a steady-state performance problem.

Field-Proven Diagnostic Procedure

1. Open VAT on the master and add DB100.DBD0 (send area), FB15.DB15.REQ, FB15.DB15.BUSY, FB15.DB15.DONE, FB15.DB15.ERROR, FB15.DB15.STATUS.
2. Force a step change on the HMI setpoint from 0 → 1500 RPM.
3. Time how long DONE takes to flash TRUE after the change. If it is < 500 ms, the master side is healthy. The issue is on the slave side (overwrite, conditional URCV, OB1 ordering).
4. If BUSY stays high for 15–30 s before DONE, the CP is re-establishing the connection — go to the diagnostic buffer on both CPUs and correlate timestamps with the switch port statistics.
5. On the slave, add DB200.DBD0 to VAT. If the value appears correctly within 1 s of DONE on the master, the receive path is healthy and the issue is the valve control logic (e.g. integrator windup, valve hysteresis).

Common Pitfalls

• Two HMI connections and a single PUT. WinCC can open a connection per tag if the project was migrated from a ProTool configuration. Each connection consumes a CP 443-1 resource.
• ISO-on-TCP TSAP collision. If the slave also acts as a server for a second master, both masters must use a different TSAP. The default TSAP 10.00 on both ends is fine if there is only one master.
• CP firmware < V2.4 on EX11. Older firmware does not honor keep-alive time changes; upgrade to ≥ V2.6.
• Auto-MDIX disabled on the switch port. The CP 443-1 does not support Auto-MDIX on all firmware versions; use a straight-through cable on uplink ports.
• Loop through a managed switch with STP enabled. When the switch re-elects the root bridge (typically every 30 s if the bridge priority is unstable), the ISO-on-TCP connection drops. Disable STP on the S7-400 port or enable Fast Spanning Tree on the switch.