1. Problem Overview
SIMATIC WinCC V6.0 SP4 runtime stations deployed in multi-station OPC networks may begin to hang 1 to 2 hours after a clean start. The freeze is partial: the runtime continues to update the on-screen date/time stamp and OPC communication tags, so the WinCC process has not crashed, but user input is blocked, screen-change requests are ignored, and the Graphics Designer and Global Script editors will not launch when invoked through the Explorer. Windows shell functions (Explorer, taskbar, Start menu) remain fully responsive, which distinguishes this fault from a true operating-system lockup. In some cases the monitor goes blank while the IPC's SIMATIC and PROFINET/Ethernet LEDs continue to blink normally, confirming that the PC is alive but the WinCC subsystem is stalled.
Because the fault began years after commissioning, a fresh Windows installation and a clean WinCC 6.0 SP4 reinstall (including all hotfixes consolidated in the service pack) do not resolve the symptom. The fault is therefore not a corrupted operating system, a defective hard disk, or a faulty HMI license, but a runtime protocol-level conflict. The reproducible root cause reported in the field is the Microsoft DCOM error Event ID 10009 generated when an HP network printer port has been bound to the WinCC station's DCOM endpoint, causing the OPC server to flood the system with DCOM authentication retries every few seconds.
2. Observed Symptoms
| Layer | Symptom | Status During Hang |
|---|---|---|
| WinCC Runtime | Screen change, button clicks, tag-forced picture change | Blocked (no response) |
| WinCC Date/Time display | Internal clock object | Updating |
| OPC communication | External tag subscription to peer WinCC stations | Updating |
| Graphics Designer / Global Script editor | Launched from Explorer | Fails to open |
| Windows Explorer, taskbar, Start menu | Shell responsiveness | Fully responsive |
| SIMATIC / PROFINET LEDs on IPC | Hardware link indicators | Blinking (normal) |
| Monitor | Display panel | Sometimes off; PC still running |
| Apdiag (WinCC diagnostic tool) | Real-time runtime diagnostics | No internal error recorded |
| Event Viewer > System / Application | DCOM Event ID 10009 | Logged repeatedly |
The fact that Apdiag reports a clean runtime while the Event Viewer is full of DCOM events is a critical diagnostic clue. Apdiag inspects the WinCC message queues, internal tag state, and connection tables; it does not audit Windows-level DCOM and RPC traffic. A DCOM storm that consumes handles, sockets, or non-paged pool memory will be invisible to Apdiag but will eventually starve the WinCC process of resources, producing the partial-freeze pattern shown above.
3. Diagnostic Procedure
- Open Start > Configuration > Administrative Tools > Event Viewer on the affected WinCC IPC. The Event Viewer runs independently of the WinCC runtime and is accessible even while the runtime is frozen, because it is hosted by the Windows shell.
- Navigate to Windows Logs > System and Windows Logs > Application. Filter by source DCOM and Event ID 10009.
- Look for the message body:
DCOM was unable to communicate with the computer <COMPUTERNAME> using any of the configured protocols; the error was from the RPC server and the HRESULT was 0x800706BA. - Note the target computer name referenced in each 10009 entry. In multi-station WinCC OPC networks, expect to see the printer-bound peer machines repeating every 10 seconds or faster.
- Cross-check with Apdiag (WinCC installation folder, typically
Siemens\WinCC\bin). Apdiag will show clean internal state, confirming the fault is at the Windows DCOM layer, not the WinCC engine. - Open Devices and Printers in the Windows Control Panel, right-click the affected HP printer, and choose Printer Properties > Ports. Inspect the Configure Port entries. A port of the form
\\<WINCC_PC>\...that points at the WinCC IPC indicates the printer has been redirected to use the WinCC station's DCOM endpoint, which is the fault pattern. - Export the Event Viewer log (Action > Save Filtered Log File As...) to a
.evtxfile for change-control records.
4. Root Cause Analysis: DCOM Event ID 10009
Event ID 10009 is raised by the Microsoft Distributed COM service (the RPCSS / DCOM launch surrogate) when a client process attempts to instantiate or call a COM object on a remote computer over a protocol the server has not negotiated. The HRESULT 0x800706BA (RPC_S_SERVER_UNAVAILABLE) confirms that the remote endpoint is reachable on the network but the RPC service on the target is not responding within the configured timeout. In WinCC multi-station OPC networks, the OPC proxy/stub components negotiate DCOM on TCP port 135 plus an ephemeral high port; the WinCC OPC server registers itself as a COM application in the registry at HKEY_CLASSES_ROOT\AppID\<OPC CLSID>\LocalService, and DCOM must reach the remote computer using the credentials configured under Component Services > Computers > My Computer > DCOM Config.
When an HP network printer driver is configured to use a Win32-standard TCP port that has been redirected to a network share on the WinCC IPC, the spooler service (spoolsv.exe) issues DCOM calls into the target station on a polling cadence. If the target's DCOM configuration, firewall, or RPC endpoint mapper rejects the call, the spooler logs 10009 and retries. Two compounding effects occur:
- The DCOM subsystem queues a retry for each call. A printer poll at 10-second intervals produces 6 retries per minute, 360 per hour, 8 640 per day per affected printer.
- Each failed call may leave a non-paged pool allocation, a Winsock handle, or an LPC port reservation that is only released when the call ultimately times out. On a long-running IPC, the kernel object table fills and WinCC's user-mode process cannot allocate the GDI or USER handles it needs to repaint or to launch child processes (Graphics Designer, Global Script editor).
That is why the runtime appears to "hang" while the date/time and tag services keep ticking. The WinCC process is alive, threads are scheduled, and the OPC publish loop continues to update internal tag values, but the message pump is starved of the resources it needs to dispatch window messages such as button clicks, picture-window repaints, and child-process creation.
5. WinCC OPC Multi-Station Architecture
In the reported configuration, one WinCC station (Computer1) acts as both OPC DA client and server, talking to six peer WinCC stations over the factory LAN. Each peer station runs its own OPC server instance, and Computer1 aggregates their tag data into its internal tag database using the OPC channel of WinCC.
| Station | Role | DCOM Dependency |
|---|---|---|
| Computer1 (failing station) | OPC client + server, HMI runtime | Must allow inbound DCOM on TCP 135 + ephemeral |
| Peer WinCC #1 through #6 | OPC servers | Outbound DCOM permitted; printer redirection forbidden |
| Print server / HP printer | Spooler service | Must not be redirected to a WinCC IPC |
Two of the six peer WinCC machines had an HP printer installed whose Ports tab in Printer Properties pointed at Computer1 by means of a UNC path. Each print job (and the driver's constant status poll) generated a DCOM negotiation with Computer1, which after a Windows update or an OPC topology change began to fail with 10009. The pattern is consistent with the documented behavior of Microsoft DCOM in mixed-credential and Kerberos-delegation environments where the local security authority rejects the caller's identity.
6. The Printer Port Conflict Mechanism
HP and most other vendors ship a "Standard TCP/IP Port" monitor that allows the Windows spooler to talk to a network-attached printer. When the port is misconfigured to point at a UNC path such as \\Computer1\HPUniversal, the spooler on the peer station attempts to enumerate the remote printer's print queues using the IRemoteWinspool COM interface, which is implemented by the print spooler on Computer1 and exposed over DCOM. The call uses the credentials of the user who is currently logged in interactively on the peer station. If Computer1's DCOM defaults have been hardened (a common hardening step for OPC servers), the anonymous call from the spooler is rejected. The spooler logs the rejection as Event ID 10009, then re-issues the call on its polling cycle.
WinCC 6.0 SP4 inherits the V6.0 SP4 service-pack consolidation, which the Siemens support entry Delivery Release: SIMATIC WinCC V6.0 SP4 (ID 22450372) documents as the cumulative set of hotfixes for the V6.0 line. None of those hotfixes change the Windows DCOM plumbing, so the fix must be applied at the operating-system or workstation level, not by upgrading WinCC.
7. Step-by-Step Resolution
- Stop the WinCC Runtime on the affected IPC: open the WinCC Explorer and click the Stop Runtime button, or run
net stop "CCAgent"from an elevated command prompt. - Stop the Print Spooler service on every peer WinCC machine whose printer was redirected to Computer1:
net stop spooler. This immediately stops the DCOM storm and lets the non-paged pool drain. - On each affected peer station, open Control Panel > Devices and Printers, right-click the HP printer, and select Printer Properties > Ports. Remove the entry that points at
\\Computer1\...using the Delete Port button. Confirm the dialog. - If the printer must remain available, recreate a port using a vendor-specific monitor (HP Standard TCP/IP Port, SNMP-Enabled) that points at the printer's actual IP address, not at the WinCC IPC. Re-add the printer using the new port.
- Re-apply the correct DCOM hardening on Computer1 using Component Services > Computers > My Computer > DCOM Config. Set the Default Authentication Level to Connect and the Default Impersonation Level to Identify as recommended for WinCC OPC topologies. Verify the WinCC-specific DCOM entries in
HKEY_CLASSES_ROOT\AppIDare intact. - Open the Windows Firewall with Advanced Security and confirm inbound rules permit
ole32.dll(RPCSS),spoolsv.exeon a dedicated subnet, and the Windows Management Instrumentation (WMI) rules. Avoid blanket allow rules; scope them to the OPC subnet. - Start the spooler on each peer:
net start spooler. Send a test page to confirm the new port works. - Clear the System and Application event logs to get a clean baseline, then start the WinCC Runtime on Computer1.
- Monitor Event ID 10009 for at least 4 hours. With the printer port removed, the count should be zero or near-zero.
dcomcnfg > Default Properties > Enable Distributed COM on this computer) breaks every OPC subscription in the network.8. Verification and Post-Fix Validation
| Check | Method | Pass Criterion |
|---|---|---|
| DCOM 10009 cadence | Event Viewer filter, watch 1 hour | 0 to 1 entry per hour (noise only) |
| WinCC runtime stability | Run for full shift without restart | No screen-change freeze |
| Graphics Designer launches | Right-click picture > Open in Graphics Designer | Editor opens in < 5 s |
| Global Script editor | WinCC Explorer > Global Script > C-Editor | Editor opens in < 5 s |
| OPC tag continuity | Apdiag internal tag trace | No gap in subscription timestamps |
| Non-paged pool | Performance Monitor > Memory > Pool Nonpaged Bytes | Stable, not climbing |
| Print job round-trip | Send test page from peer station | Prints in < 30 s; no 10009 entry |
For permanent monitoring, configure a Windows Scheduled Task that runs wevtutil qe System "/q:*[System[(EventID=10009)]]" /c:1 /rd:true /f:text every 15 minutes and writes the count to a log file; trigger an alert when the count rises above 5 per hour.
9. Preventive Hardening
- Printer redirection policy. Add a Group Policy Object (GPO) under Computer Configuration > Policies > Windows Settings > Security Settings > Local Policies > Security Options that disables Print spooler > Allow print spooler to accept client connections on WinCC IPCs. WinCC does not need to accept inbound print jobs; the OPC server is a server of COM, not print.
-
DCOM scope. In
dcomcnfg, restrict the Default Protocols to TCP/IP only. Remove Named Pipes, which often mask DCOM errors by retrying over different transports. - Service-pack discipline. Keep WinCC 6.0 SP4 patched to the latest hotfix level documented under SIMATIC WinCC V6.0 SP4 Delivery Release (ID 22450372). SP4 consolidates the hotfixes that fixed the OPC DA 3.0 callback path and the global-script compiler crashes; staying current prevents the new DCOM storm from triggering latent WinCC bugs.
- Network separation. Place the OPC subnet and the print subnet on different VLANs. The Siemens WinCC V6.x Communication Manual recommends a dedicated plant-bus segment for OPC traffic; the same segment should never carry Windows file and print traffic.
- Event-log automation. Attach a scheduled task to the Event Viewer subscription for Event ID 10009. Page the on-call engineer on the first occurrence of three events within 60 seconds from the same source computer; that cadence is the incipient DCOM storm.
10. Troubleshooting Matrix
| Symptom Pattern | Likely Cause | First Diagnostic Step | Fix Path |
|---|---|---|---|
| Hang after 1-2 h, date/time updates, Windows OK, Apdiag clean | DCOM 10009 storm (most often printer) | Event Viewer filter on 10009 | Remove redirected printer port, harden DCOM |
| Hang on screen change only, no DCOM events | Picture cache overflow or C-script deadlock | Apdiag deadlock trace, GSC Diagnostics | Refactor cycle-action scripts, raise tag trigger threshold |
| Hang at startup, no runtime | Corrupt project database | Check \WinCC\Diagnose folder |
Restore from backup, run CCRebuildCC.db equivalent for V6 |
| Hang during alarm logging only | SQL Server Express write stall | SQL errorlog, disk queue length | Archive, defragment, extend storage |
| Random hang, IPC display off, LEDs normal | GPU driver / power management | Device Manager > Display adapters > Properties | Disable C-states in BIOS, lock GPU power profile |
| Hang correlated with antivirus scan | Real-time scanner locking WinCC DLLs | AV log timestamps vs. hang timestamps | Add WinCC and OPC exclusions |
| Hang on dual-monitor only | Graphics driver regression | Safe-mode single monitor test | Roll back driver, lock primary monitor |
11. Apdiag and Companion Tools
Apdiag is delivered with WinCC 6.x and exposes the following signal groups relevant to this fault:
- Runtime > Process State: confirms whether the WinCC process is alive. A "running" entry here while the UI is frozen is the diagnostic signature of a Win32 message-pump stall.
- Tags > OPC > Subscription: shows the timestamp of the last received tag update from each OPC subscription. Continuous updating confirms the OPC layer is healthy.
- Connections > Channel Diagnosis: lists the per-channel state, including TCP socket status, retry counts, and timeouts.
- Scripts > C-Action / VBS-Action: shows the queued and running script invocations. A growing queue with no completion events indicates a script thread is blocked.
Pair Apdiag with Process Explorer (Sysinternals) and watch the Handles and USER Objects columns for the CCExplorer.exe process. A monotonic climb in USER objects across hours, with the screen frozen, is the DCOM storm's resource signature. Compare it to a known-good peer station to confirm the deviation.
12. Related Field Notes
Several points are worth recording in the site log when you close this fault:
- The DCOM storm is invisible to Apdiag by design; the diagnostic toolchain must include the Windows Event Viewer. Document the path to Event Viewer > Windows Logs > System > Filter Current Log > Event ID 10009 in the operator quick-reference card.
- The fault does not reproduce on a freshly imaged IPC until a printer is installed. The reproduction recipe is: install WinCC 6.0 SP4, install the HP Universal Printing driver, redirect its port to the WinCC IPC, restart the spooler, wait 60-90 minutes. The hang will then occur without any further user action.
- Several Windows cumulative updates have re-tightened DCOM defaults since the V6.0 SP4 era. After every Patch Tuesday, retest a non-production WinCC IPC for this fault before rolling the patches into production. The V6.0 SP4 base is conservative on DCOM; cumulative updates since 2018 have made it stricter.
- Graphics Designer and Global Script launch as child processes of the WinCC Explorer. When the parent process cannot allocate the necessary USER handles, child-process creation fails silently. That is why the user can open Windows Explorer (no USER object contention) but cannot open Graphics Designer (a child process of the affected process).
- The monitor blanking reported in the field is a separate symptom most often caused by the GPU driver entering a low-power state while the Win32 message pump is blocked. The fix for the blanking is the same as the fix for the hang; restoring the message pump restores the GPU keep-alive.
What is WinCC 6.0 SP4 Event ID 10009 and why does it freeze the runtime?
Event ID 10009 is a Microsoft DCOM error logged when a client cannot reach a remote COM server. When an HP printer is redirected to a WinCC IPC, the spooler generates hundreds of failed DCOM calls per hour, exhausting non-paged pool and USER handles. WinCC's process stays alive and updates tags, but its message pump can no longer dispatch button clicks or launch editors, producing the 1-2 hour hang. Reference: Microsoft DCOM documentation.
How do I find the source of the 10009 storm on a WinCC 6.0 SP4 station?
Open Start > Administrative Tools > Event Viewer > Windows Logs > System, filter by source DCOM and Event ID 10009. The target computer name in the message body is the IPC being polled. In the reported case, two peer WinCC machines were polling Computer1 every 10 seconds because of redirected HP printer ports.
Why does Apdiag show no error while the runtime is hanging?
Apdiag inspects WinCC's internal queues, tag state, and channel diagnostics. It does not audit Windows-level DCOM traffic or kernel handle tables. A DCOM storm therefore looks clean to Apdiag even while it starves the WinCC process. Always pair Apdiag with the Windows Event Viewer when investigating hangs.
What is the correct fix for the printer port that causes the DCOM storm?
Delete the redirected port in Printer Properties > Ports, then recreate a vendor-specific monitor port (HP Standard TCP/IP Port) that points directly at the printer's IP address, not at a WinCC IPC. After removing the redirection, monitor Event ID 10009 for one hour; the count should drop to zero.
Is upgrading WinCC 6.0 SP4 to a newer service pack a valid fix?
No. The DCOM storm is a Windows spooler-to-DCOM interaction, not a WinCC defect. The V6.0 SP4 base code, including the consolidated hotfixes documented under SIMATIC WinCC V6.0 SP4 Delivery Release (ID 22450372), does not change the DCOM plumbing. The fix must be applied at the operating-system or workstation level by removing the printer redirection and hardening DCOM defaults.