Troubleshooting Siemens CPU 410-5H Synchronization Module Failure

Diagnose and recover a SIMATIC S7-400H / PCS 7 redundant controller pair (CPU 410-5H) where one CPU has dropped to STOP because a synchronization module has failed. The article covers LED interpretation, diagnostic buffer event IDs, the hot-swap procedure, fiber-optic inspection, firmware compatibility, and the verification sequence that confirms a clean return to redundant operation.

1. Problem Statement

The reported field symptom is a CPU 410-5H redundant pair in which the standby CPU sits in STOP mode while the partner CPU continues in RUN as master. One of the two synchronization port LEDs on the failed CPU is not blinking (off, solid, or in a stuck pattern), while the matching port on the partner CPU still shows the expected cyclic activity. The fiber-optic cable was already replaced, the hardware configuration was reloaded, and a warm restart was performed - none of which restored redundancy. The fix that finally cleared the fault was replacement of the synchronization module itself.

The CPU 410-5H (typical order number 6ES7410-5H08-0AB0, with earlier generations 6ES7410-5H04-0AB0 and 6ES7410-5H01-0AB0) is the SIMATIC S7-400H and PCS 7 controller used in high-availability process automation. Redundancy is established in hardware: two CPUs exchange their internal state through dedicated synchronization modules plugged into interface slots IF1 and IF2 of each CPU and connected by crossed fiber-optic cables. The user program has no direct access to the synchronization link; the operating system handles the handshake autonomously. A STOP condition triggered by sync-link loss is therefore a hardware fault, not a logic fault, and it is not resolved by reloading the program.

2. CPU 410-5H Redundancy Architecture

Each CPU 410-5H provides two interface slots that accept a synchronization module. The redundancy link is therefore a 2-out-of-2 structure: the two sync modules on CPU 0 are connected by crossed fiber-optic cables to the matching sync modules on CPU 1, producing two physically independent redundancy links. If one link is broken or degraded, the system continues to operate on the remaining link, but the operating system will post a maintenance event and the affected CPU will eventually drop to STOP if it cannot maintain a valid redundancy state.

The interface slots are located on the front of the CPU. The CPU itself is not hot-swappable, but the synchronization modules are designed to be hot-swappable: they can be removed and re-inserted on a running CPU without damaging the hardware, provided the current redundancy state permits it. This is the design foundation that makes the field fix possible without taking the process to a controlled shutdown.

The redundancy concept in the S7-400H distinguishes three operating modes:

• Solo - only one CPU is present, or one CPU is in STOP and the partner is also unavailable. The system runs without redundancy.
• Standby - both CPUs are present and the partner CPU is the active master; this CPU is the standby and is being kept in sync through the sync modules.
• Redundant - both CPUs are in RUN, both are kept in sync, and a switchover can occur without loss of the process.

A failed sync module forces the affected CPU out of the redundant state and, if a clean link cannot be re-established, into STOP.

3. Synchronization Module Hardware

The synchronization module is a small plug-in submodule. The current Siemens catalog designations for the CPU 410-5H synchronization modules are in the 6ES7960-1AA0x-0XA0 family (for example, 6ES7960-1AA06-0XA0 or 6ES7960-1AA08-0XA0 depending on the CPU firmware generation and the desired optical budget). Modules are paired: two identical modules per CPU, and the FO cables on both links should be of the same type and length to keep the optical budget symmetric.

The authoritative reference is the Siemens Industry Online Support entry ID 74736822, "Synchronization modules for the CPU 410-5H", which states explicitly that "you require two synchronization modules per CPU, connected in pairs by fiber-optic cable" and that "the system supports hot-swapping of synchronization modules." The full document is available at Synchronization modules for the CPU 410-5H - Siemens Support (ID 74736822). Always verify the order number of the spare module against the compatibility list published in that entry for your specific CPU firmware version.

4. Synchronization Module LED Status Indicators

Each synchronization module exposes a four-LED status display. The LED meanings defined in the Siemens support article (entry 74736822) are the authoritative field reference.

In a healthy H-system both sync modules on each CPU show a cyclic RUN blink, and the IFM2F and STOP LEDs are off. The reported field symptom - "one synch port LED is not blinking" - is consistent with a sync module that has lost its RUN state and is no longer participating in the redundancy handshake. The exact LED pattern (solid STOP, solid IFM2F, all LEDs off) is the key to distinguishing the failure from a fiber-optic break (where you would expect a maintenance/MAINT indication on the partner side rather than a hard fault on the module side).

5. Root Cause Analysis

The root cause for a single sync port losing its blinking pattern while the matching port on the partner CPU continues to operate normally is, in the vast majority of field cases, the synchronization module itself. The Siemens support article explicitly states that "synchronization of master and stand-by CPU is done by hardware and it is independent of software," and that "if you have a redundancy failure, one CPU in stop mode and error LEDs indicate an error in one of the sync modules, try to change the sync modules with a new one and test again."

The failure modes that produce this exact symptom are:

• Optical transmitter/receiver degradation on the sync module, producing a link that no longer locks even with a clean fiber-optic cable.
• Internal logic failure of the sync ASIC that prevents the module from completing the handshake with the partner module.
• Loss of an internal supply rail on the module (5 V or 3.3 V) due to component aging - electrolytic capacitors and small regulators are the most common suspects in field returns.
• ESD damage or contamination of the LC receptacle on the module side, less common in benign environments but seen in plants with high airborne contamination.

What the failure is usually not:

• A user program bug - the sync link is hardware-only and is not driven by the application.
• A bad HW Config download - the CPU will accept the new configuration and run with it; the sync link is unaffected.
• A wrong fiber-optic cable length, provided the cable is within the published optical budget. Pair mismatch (one cable 1 m, the other 10 m) is allowed for physical layout reasons but must respect the optical budget on each link.

That is why the field-tested fix is to replace the suspect sync module rather than to chase the symptom through the configuration or the program.

6. Diagnostic Buffer and Event IDs

Before any physical intervention, capture the diagnostic buffer (CPU > Diagnostic Buffer in STEP 7 / TIA Portal, or via the HMI Web server). For CPU 410-5H in an H-system, the following event classes and OBs are the most useful when investigating a sync-link failure.

For the most reliable diagnosis, read out the system state list (SZL) for the H-system status. The standard SZL IDs for an H-system include the redundancy status word and the link quality of each sync port. Capture the SZL before and after the swap to confirm that the link has been restored and that all four ports are in LINK-UP / SYNC-ACTIVE state.

7. Step-by-Step Troubleshooting Procedure

1. Verify the physical symptom. Confirm the LED pattern on both sync modules of both CPUs. Document whether IFM2F, MAINT, RUN, and STOP are lit, off, or blinking. Photograph the front of the CPUs; this is invaluable for remote escalation and for the maintenance log.
2. Connect to the master CPU (the one still in RUN). Open STEP 7 V5.x or TIA Portal in the matching project version and read the diagnostic buffer of the failed CPU. Note the timestamps of the last successful redundancy event and the first sync-link error.
3. Read the SZL for H-system status. Look for the sync-link state of each of the four sync modules (CPU 0 IF1, CPU 0 IF2, CPU 1 IF1, CPU 1 IF2). The expected states in a healthy system are LINK-UP / SYNC-ACTIVE for all four. A single LINK-DOWN on a port that has a known-good FO cable points to the module, not the cable.
4. Inspect the FO path. Unplug the cable on the failed port and inspect the LC ferrule under magnification. A scratched or contaminated ferrule can mimic a bad module. Clean with a proper LC cleaning pen and re-insert. If the cable is suspect, replace it with a known-good Siemens 6ES7960-1AA04-5xA0 cable of the same length. The reported case ruled the cable out, so move on to step 5.
5. Reload the hardware configuration (HW Config in STEP 7, or device configuration in TIA Portal) and perform a warm restart. If the standby still drops to STOP, the failure is in the sync-link hardware, not the configuration.
6. Hot-swap the suspect sync module as described in Section 8. With the master in RUN, remove the failed sync module from the standby CPU. The system tolerates the brief loss of one link. Insert the replacement module and wait for the operating system to re-acquire the link - the RUN LED should resume its cyclic blink within seconds.
7. Bring the standby back into redundancy. From the master, issue a "Switchover with synchronization" or perform the standard link-up sequence. Monitor the diagnostic buffer for a clean redundancy recovery event.
8. Verify redundancy with a controlled switchover as described in Section 11.

8. Synchronization Module Hot-Swap Procedure

Hot-swapping is supported. The Siemens support article (ID 74736822) confirms that "the system supports hot-swapping of synchronization modules." The field-tested sequence is:

1. Ensure the partner CPU is in RUN and is the master.
2. Confirm that the standby CPU has at least one healthy sync link still up. If both links are down, hot-swap is not safe; perform a controlled shutdown of the standby first.
3. Pull the failed sync module straight out of its slot. Do not rock it; the connector is keyed.
4. Inspect the slot pins for damage, contamination, or bent contacts. A flashlight and a loupe are sufficient.
5. Insert the replacement module firmly until the latch engages. The module's LEDs will run through a self-test (typically all four LEDs light briefly).
6. Wait for the RUN LED to begin its cyclic blink, indicating that the link is up and synchronization has been re-established.
7. Check the diagnostic buffer of both CPUs for a clean "synchronization restored" entry, and verify with the SZL that all four ports are LINK-UP / SYNC-ACTIVE.

9. Fiber-Optic Cable Inspection and Validation

Although the reported case ruled the FO cable out, a damaged or contaminated cable can present the same symptom as a bad module. The CPU 410-5H sync ports use LC connectors. The cable must be:

• An official Siemens 6ES7960-1AA04-5xA0 series cable (POF or PCF, depending on the order number), or a manufacturer-approved equivalent that meets the published optical budget.
• The same type on both links of a CPU. Mixing POF on one link and PCF on the other can cause one port to be marginal while the other is healthy.
• Within the maximum length published for the cable type. For POF the practical limit is around 10 m; for PCF, longer runs are supported. The exact maximum is documented in the cable's data sheet.
• Free of kinks, crushed sections, and tight bend radii below the manufacturer's minimum (typically 30 mm for POF).

To validate a cable in the field, swap it with the matching cable on the partner CPU. If the failure follows the cable, replace the cable. If the failure stays on the same port, replace the module.

10. Firmware and Configuration Considerations

The CPU 410-5H has gone through several firmware generations (V6.x, V7.x, V8.x, with V8.x being the most recent at the time of writing). Each generation has a specific list of compatible sync modules. A mismatched sync module and CPU firmware combination can produce a "module not recognized" or "module faulty" diagnostic entry, even with a brand-new module. Before installing a replacement sync module, confirm that its order number is on the compatibility list for your CPU's firmware. The Siemens support entry 74736822 is the authoritative reference for this list.

If the module is recognized but the link still does not come up, the next step is to verify that the H-system parameters in HW Config (STEP 7 V5.x) or device configuration (TIA Portal) match the physical configuration:

• Both CPUs are configured as H-CPUs, not as standalone CPUs.
• Sync module slots IF1 and IF2 are configured and assigned the correct module type.
• The monitoring time and the maximum number of tolerated link losses are appropriate for the application (typical monitoring time: 500 ms to 2000 ms, depending on the OB 1 scan time and the configured redundancy behavior).
• The S7-400H function blocks (OB 70 / OB 72 / OB 82 / OB 83 / OB 85 and the H-sync OB family) are loaded on both CPUs.
• The two CPUs are connected to the same plant bus (MPI / PROFIBUS / Industrial Ethernet) and the H-system addresses are correctly assigned.

If a configuration mismatch is suspected, perform a full HW Config download to both CPUs in STOP, then perform the warm restart from the master. A configuration problem is usually visible in the diagnostic buffer as an OB 85 or OB 82 entry that names the affected slot.

11. Verification and Commissioning

After the module swap, run the full H-system commissioning sequence before returning the plant to production.

1. Confirm both CPUs are in RUN-redundant (RUN LED on both, redundancy status word = redundant).
2. Read the SZL of both CPUs; all four sync ports should be LINK-UP / SYNC-ACTIVE.
3. Read the diagnostic buffer of both CPUs; the last entry should be a clean redundancy event with no warnings.
4. Perform a controlled switchover test. Force the master to STOP via the programming device, confirm the standby takes over within the monitoring time, then bring the original master back to RUN as the new standby.
5. Perform a second switchover in the opposite direction to confirm that the link is symmetric and that both directions recover cleanly.
6. Run the application under the new redundancy for at least one full scan cycle and verify that the process values are stable. For a process-critical plant, run through at least one operational shift before considering the repair complete.

12. Preventive Maintenance

• Keep at least one spare sync module and one spare FO cable of each length in use on site. The mean time to repair for a sync-link loss is dominated by the time to obtain a replacement, not the time to install it.
• Log the diagnostic buffer weekly. A growing count of MAINT events on a sync port is an early warning that the module is degrading and should be planned for replacement.
• Clean the LC receptacles of the sync modules and the FO connectors every time the cabinet is opened. Use a proper LC cleaning pen; do not use isopropyl alcohol on the connector end-face without a lint-free wipe, and never touch the ferrule end-face with bare skin.
• Respect bend radius and tensile load on the FO cables. The most common field cause of FO link loss is mechanical damage to the cable, not electrical failure of the module.
• Verify the firmware compatibility of the spare modules against the CPU firmware before ordering. The compatibility list changes with each CPU firmware release and is published in the Siemens support entry 74736822.
• Maintain a record of the orientation of the FO cables. The crossed-cable topology is easy to disrupt during maintenance; a clear cable labelling scheme prevents the introduction of new faults.

13. Common Pitfalls

• Chasing the symptom through the program. The sync link is hardware-only. Time spent debugging the user program is time wasted; the diagnostic buffer and the LEDs are the right starting point.
• Replacing the FO cable when the LEDs already point to the module. The cable was the first thing tried in the reported case, but a solid IFM2F or a missing RUN blink on the module side is the textbook indication to replace the module, not the cable.
• Swapping sync modules between the two CPUs. If the spare is from the same batch as the failed module, the partner module on the same CPU may be near end-of-life as well. It is better to bring in a fresh, date-coded spare.
• Hot-swapping a sync module when the partner CPU is also in STOP. If both CPUs are down, there is no master to maintain the process. Bring the master back to RUN first, then perform the swap on the standby.
• Forgetting to clear the diagnostic buffer after the repair. A diagnostic buffer full of historical sync faults makes the next incident harder to diagnose. Clear the buffer after the verification sequence is complete.

14. Troubleshooting Matrix