Resolving PCS 7 S7-400H CPU Switchover Detection on OS Clients

When a redundant S7-400H automation station performs a CPU master/reserve switchover, operator stations (OS clients) connected through the terminal bus can lose the process picture for several minutes before recovering. The root cause is almost always a missing or incorrectly configured S7 Fault-Tolerant Connection (S7H) between the OS server and the AS pair, combined with terminal-bus components that are not configured to handle the H-system failover path. This reference walks through the diagnosis, NetPro configuration, WinCC redundancy settings, and acceptance test required to bring failover recovery down to the expected sub-30-second window.

1. Architecture Context: Plant Bus vs. Terminal Bus

In a PCS 7 plant, three physical Ethernet networks usually coexist, and confusing them is the single largest source of failover-detection failures on operator stations.

The CPU switchover is detected on the plant bus, where two CP443-1 modules — one in each H-CPU — share a logical MAC/IP and present the AS to the OS server through an S7 fault-tolerant connection. The OS server's job is to detect the CPU change and inform the connected OS clients over the terminal bus. If the plant-bus side does not run an S7H connection, the OS server never registers the redundancy loss and the clients fall into the 5–6 minute timeout window observed in the field.

2. Problem Description

The failure mode reported on installed PCS 7 systems with three redundant OS clients, two redundant terminal-adapter cards per client, and one Engineering Station with two CP1613 cards presents as follows:

1. During steady state, process pictures update normally on all OS clients.
2. Operator triggers or maintenance forces a CPU 0 ↔ CPU 1 switchover (button on CPU, STOP/RUN, fiber-optic test, or actual hardware fault).
3. The active CPU becomes standby; the standby becomes master.
4. The OS clients immediately display a blank process picture with no live values and no update.
5. After approximately 5–6 minutes, the OS clients resume updates using the new active path.
6. The OS server, by contrast, logs only a brief "H-system: CPU redundancy loss" system message and recovers in < 30 s.

The five-to-six-minute window is the default OS-client retry interval when the server's redundancy state cannot be reconciled — this is not a sign of slow recovery but a sign that the OS server never communicated the failover to the clients.

3. Root Cause Analysis

Five conditions must be satisfied simultaneously to obtain sub-30-second failover detection across the entire OS client fleet. Any missing item reproduces the symptom above.

The most frequent root cause in the field is condition C2: an ordinary S7 connection is created in NetPro instead of an S7 connection — fault-tolerant. S7 connections do not participate in H-system failover detection. The OS server therefore waits for the TCP keep-alive to expire before treating the path as broken, and only then attempts to reconnect — hence the 5–6 minute blackout.

4. Required Hardware, Firmware, and Software

5. NetPro Configuration: Building the S7H Connection

5.1 AS side (HW Config)

1. Open the AS station in HW Config (SIMATIC Manager → AS project → HW Config).
2. Confirm the rack contains two CPUs (slots 3 and 4 for S7-400H) and that both racks have a CP 443-1 in identical slot positions.
3. Double-click CP443-1 #1 → properties → set IP address 192.168.0.10, subnet mask 255.255.255.0, no router.
4. Open CP443-1 #1 → Properties → Operating Mode and check "Activate CPU redundancy". Repeat for CP443-1 #2 with IP 192.168.0.11.
5. In the AS station, set the H parameters: H-CPU pair, sync module port, monitoring time (default 2.0 s is acceptable).
6. Save and translate HW Config.

5.2 OS server side (HW Config)

1. Insert a SIMATIC PC station with a CP1613 / IE PCIe module.
2. Assign the CP1613 an IP address on the same plant-bus subnet as the CP443-1 pair, e.g. 192.168.0.20, subnet mask 255.255.255.0.
3. If the OS server uses the same PC for both plant and terminal bus (single-PC OS server), insert a second CP1613 for the terminal bus and assign a terminal-bus IP, e.g. 192.168.1.20 on subnet 192.168.1.0/24.

5.3 Creating the S7H connection

1. In NetPro, select the OS server's CP1613 plant-bus interface.
2. Right-click → Insert New Connection.
3. Partner: the AS station. Connection type: S7 connection, fault-tolerant (German: "S7-Verbindung, fehlertolerant").
4. Local end: OS server CP1613.
5. Partner end: enter both CP443-1 modules by clicking "Add" once for each; NetPro stores the S7H partner pair CP443-1#1 / CP443-1#2.
6. Connection ID is auto-assigned; note the ID for diagnostic use (e.g. S7H_conn_1 = ID 0x0010).
7. Save and compile (NetPro → Save and Compile). This produces SDB entries for the OS server.
8. Download to AS (HW Config) and download SDBs to the OS PC (Station → Download → to PC station).

6. WinCC Server Redundancy Configuration

The S7H connection on the plant bus gives the OS server a fast CPU-switchover signal. The OS server must then inform the OS clients. In a single-server (non-redundant-server) topology, WinCC already handles this internally through its Server-Client subscription channel, but only if the OS server's WinCC redundancy option is correctly configured.

6.1 Single OS server (no redundant server)

1. Open WinCC Explorer on the OS server PC.
2. Right-click Computer → Properties → tab Startup.
3. Confirm WinCC Runtime launches; the OS server project name must match the WinCC project on the server.
4. On each OS client: WinCC Explorer → Computer → Properties → tab Server. Add the OS server's host name (e.g. OSSRV01) as preferred server.
5. Verify that the OS client's picture tree references the server's tags, not local tags.

6.2 Redundant OS server pair

1. In WinCC Explorer → Computer → Properties → Redundancy, enable Activate redundancy on both servers.
2. Set the redundancy partner (e.g. OSSRV02 for OSSRV01 and vice versa).
3. Set Synchronization cycle to 10 s (default) for fast tag updates, or 30 s if CPU load is constrained.
4. Set Update time for tag status "bad" to a value lower than the client timeout — 5 s recommended.
5. Verify that the OS clients have BOTH servers listed in Computer → Properties → Server, in the order preferred server first.

7. Terminal Bus Configuration with Two CP1613 per Client

With three OS clients, each equipped with two CP1613 cards, and one ES with two CP1613 cards (CP1613 cards in total = 8), the terminal bus must support NIC teaming or a redundant ring. The hardware alone is not enough — the driver layer must bind both cards.

7.1 Driver configuration (Windows / SIMATIC NET)

1. Install the SIMATIC NET driver (bundled with PCS 7). The CP1613 module is identified as "CP 1613" under Windows Device Manager.
2. Open SIMATIC NET Configuration → PC Station.
3. For each CP1613, configure the same logical station name (e.g. OSCLT01) but different physical IPs, OR configure NIC teaming with one virtual IP and two physical MACs.
4. Recommended approach: assign the same IP address to both CP1613 adapters (allowed only if the active path is selected via MAC failover, not ARP) and use the SIMATIC NET Fault Tolerance driver to manage primary/backup roles.

7.2 Switch configuration

1. If using a redundant ring (SCALANCE XR-400 with HRP manager), place all 8 CP1613 in the ring and assign one switch as ring manager.
2. If using line topology, ensure each CP1613 has its own switch port and the switches are interconnected via single trunk (no ring, no redundancy manager).
3. Enable LLDP on the switches; PCS 7 V8+ relies on LLDP for diagnostics.

8. Step-by-Step Resolution Procedure

Apply the following procedure to convert a PCS 7 OS client fleet from blank-screen-on-failover behavior to sub-30-second recovery.

8.1 Inventory the current configuration

1. Open NetPro on the ES. Print or export the current plant-bus connections.
2. For each AS-OS pair, identify whether the connection type is S7 or S7 fault-tolerant. If any are S7 (not fault-tolerant), this is the primary root cause.
3. On the OS server, open WinCC Explorer → Computer → Properties → Redundancy. Confirm the redundancy state and partner.
4. On each OS client, verify OSSRV01 (and the redundant partner, if any) is listed in Computer → Properties → Server.

8.2 Upgrade the connection to S7 fault-tolerant

1. In NetPro, right-click the plant-bus connection → Properties → Connection Type.
2. If S7 fault-tolerant is not selected, delete the connection and re-insert it as fault-tolerant. Adding the partner endpoint requires both CP443-1 modules — confirm both appear in the partner list.
3. Compile NetPro (button Save and Compile) and check the diagnostic output for errors.

8.3 Download the new configuration

1. HW Config → PLC → Download to push the new SDBs to the H-CPU pair. Both CPUs receive the same SDBs.
2. On the OS server PC, open the Station Configuration Editor and confirm PC Station name matches NetPro.
3. Use Station → Download → PC Station to send the new SDBs to the OS PC's CP1613.
4. Restart the WinCC Runtime on the OS server.

8.4 Tune the WinCC redundancy parameters

Edit WinCC-Project\Computer\OSSRV01\WinCC_Server.cfg or use the GUI to set:

[Redundancy]
Activate=1
RedundancyPartner=OSSRV02
SyncCycle=10
TagStatusBad_UpdateTime=5
MaxPathDisruptionTime=30

8.5 Validate NIC teaming on OS clients

1. On each OS client, run simatic_net_diag.exe (Start → SIMATIC → SIMATIC NET → Diagnostics).
2. Confirm both CP1613 are listed as Active in the fault-tolerance group.
3. Pull cable on CP1613 #1 and verify the OS client continues to update within 5 s.

9. Verification and Acceptance Test

A formal failover acceptance test must demonstrate the following on the live plant:

1. Warm standby: Both CPUs are in RUN and H-synchronized. OS server shows green. All OS clients show green.
2. Triggered switchover: Operator presses the CPU switchover button on CPU 0 (front panel — the MAST and RES LEDs toggle). Document the time.
3. Plant-bus detection: OS server logs "H-system: CPU redundancy loss" system message; cleared within 5 s.
4. OS client recovery: All OS clients display values again within 5–15 s, ideally < 10 s. Blank-screen time must be < 30 s to meet the acceptance criterion.
5. Loss-of-CPU test: Pull power on CPU 0. Same recovery target. CPU 1 takes over.
6. CP443-1 failure: Unplug network cable on CP443-1 #1. CPU 1 takes over the plant bus. OS server recovers within 5 s; OS clients within 30 s.
7. Cable break on terminal bus: Pull cable on OS client A's CP1613 #1. The client continues to update via CP1613 #2 within 5 s.

10. Diagnostic Tools and System Messages

10.1 Key system messages on the OS server

• 0x1590 / "H-system: CPU redundancy loss" — Normal during switchover; expected to appear and clear.
• 0x0001 / "Connection failed" — Plant-bus connection broken; S7H partner unreachable.
• 0x8001 / "Redundancy partner not reachable" — OS server cannot contact redundant OS server.
• 0x000E / "Partner not found" — The CP443-1 partner IP is misconfigured or on a different subnet.

11. Common Pitfalls and Field-Proven Caveats

• Pitfall 1 — WinCC V7 redundancy license: The WinCC Redundancy option requires a separate license key. Without it, the redundancy block in Computer → Properties → Redundancy appears but is disabled at runtime. Clients do not switch over even though the configuration looks correct.
• Pitfall 2 — Subnet mismatch: CP443-1 #1 on 192.168.0.10 and CP443-1 #2 on 192.168.0.11 with the OS server on 192.168.1.20. The OS server must reach both subnets; if the routing is not configured, only one CP443-1 is reachable and failover is asymmetric.
• Pitfall 3 — CP1613 firmware: Firmware < V2.5 on CP1613 does not support S7H with H-system redundancy monitoring; symptom is intermittent detection. Upgrade firmware through SIMATIC Manager → PLC → Update Firmware.
• Pitfall 4 — PC station name mismatch: The Windows hostname of the OS server must match the PC Station name in NetPro. Mismatched names cause the Station Configuration Editor to display "not associated" and SDBs are not loaded.
• Pitfall 5 — OS client running WinCC as primary project: An OS client configured with its own project (i.e. its own tags) instead of a WinCC client that subscribes to the OS server will display a frozen picture on CPU switchover because it has no plant-bus connection at all.
• Pitfall 6 — Time sync drift: If the OS server's clock drifts more than 2 seconds from the H-CPU pair, the system message ordering can be corrupted and the client triggers an archive resync storm. Configure NTP from a plant-wide time source.

12. Frequently Asked Questions