Good news!

The workflow troubleshooting tool that you might remember from over a decade ago is back and better than ever. Some good folks with the System Center product group have recently listened to this engineer’s crying and pleading. It has finally been rebooted with a few improvements.

If my memory serves me correctly (I’m sure folks will be quick to correct me) the Workflow Analyzer was first introduced in the System Center Operations Manager 2007 R2 Authoring Resource Kit which included a number of helpful tools for authoring, testing, and troubleshooting management packs.

The last time I remember this tool working correctly was back in the days of SCOM 2012. Changes in the subsequent versions of Operations Manager had caused the tool to crash (documented with workarounds by Michel Kamp) and ultimately be left behind.

What is a trace?
This tool allows you to initiate a trace of a specific workflow running for a specific instance of an object. A trace is basically a way to tell the workflow to reveal extra information about what it is doing. The ability is already baked into the code by the developers, we simply need to turn it on temporarily to capture verbose details. In essence, it allows us to spy on the agent.

Steps

Initiate trace from mgmt server, observe on agent machine

1) Launch the tool on the mgmt server to select:
– a healthservice/agent
– a workflow
– a specific object instance (for which the workflow is running)
This will initiate the trace instructions on the mgmt server (in the Ops database) which will get sent to the agent.

2) Also Launch the tool on the agent machine:
Select: Connect to existing Workflow Analysis
The tool will begin to collect/reveal activity for the designated instance and workflow.

Note: Assuming you have installed the tool to the default location on both the mgmt server and agent machine:
C:\Program Files\Microsoft System Center\Management Pack Tools
The installer simply extracts the files. You can copy them to any location and launch the tool.

Example

• Launch the tool on a management server.
• Select agent/HealthService from dropdown list (devdb01).
• Start
• (be patient, this might take a minute)

• Select the workflow
• Trace
• (be patient, this might take a minute)

• Select specific object instance
• Start

The tool should have created a special override in a custom management pack that will tell the agent to trigger the agent tracing.

The special MP should be delivered to the agent within a few minutes.

You will not see meaningful output on the management server when tracing on a remote agent.

To view the trace activity, you must also run the tool on the agent machine.

• Launch the tool on the agent machine
• Connect to an existing Workflow Analysis

Note: the IntervalSeconds for my target workflow has been decreased to reduce the amount of time that I need to perform the trace procedure.

Eventually we see data appear:

This is what the property bag items look like when formatted neatly. Double click to open the entry:

In this example we can see the property bag values being used in an expression filter for cookdown in the monitor type module:

Optionally, you can Export the trace data to a .csv file on the local machine:

The default delimiter is a comma. Update the delimiter as you see fit.

Close/exit the trace when done on the agent machine and on the mgmt server. Make sure the temporary trace management pack has been removed. If it still exists, remove it manually.

Download

https://www.microsoft.com/en-us/download/details.aspx?id=102671

Too complicated? … or unable to log into agent machine?

No problem, use the Agent Trace task Trace Workflow Agent Task.
Want to retrieve trace data effortlessly from remote agent machines? Try the File Transport management.

Note:

2021.04.05: Bugs/issues at the time of this writing. PG folks are aware of these minor issues and will likely address them in a future update.

1. Workflow selection screen. The window will freeze if you select “Refresh” after the view has been filtered.
2. During active agent machine trace, on mgmt server TraceWorkflow display window, will show ambiguous GUID messages. These can be ignored for now.
   Example:
   Unknown( 69): GUID=178c26ea-6c39-3d5f-3457-f40c4e8f6fb5 (No Format Information found).