Visualizing Azure DevOps Dependencies with PlantUML
Azure DevOps is a tool used to organize work into a backlog, track work items as they progress from being brand new to being done, and store lots of associated details. Think of it as a highly structured to-do list for teams. Among those details are each work item’s relationships to other work items. Visualising the dependencies between items could give me some insight into how the work is structured and what work items might be held up by other work items.
What’s a Dependency?
Features, User Stories, and Tasks are all types of work items within DevOps. For example, in Azure DevOps, Features often include several User Stories. A single User Story may include a number of Tasks. A Feature and User Story have a Parent-Child relationship. For this discussion, we will assume that a Feature cannot be considered complete until all of its User Stories, its children, are complete.
In other words, Features are dependent upon their child User Stories. In the same way, User Stories are dependent upon their child tasks. If we were to visualize these dependencies, they would form a tree; the Feature at the base of the tree branching into User Stories, and Tasks forming the leaves. It’s a beautiful tree of dependencies.
These parent-child relationships are automatic within DevOps, but there are other ways to indicate dependencies. The method I use is to specify Predecessor-Successor links. A work item’s predecessor is something that must come before, or precede, it. Its converse is the successor, something that comes after, or succeeds, that work item. Using these links, we can identify dependencies among work items that do not have a parent-child relationship.
Dependencies in DevOps
While it’s easy to specify these links among work items, DevOps does not, out of the box, provide a way to visualize these links. Being able to see them all at once could provide insight into how your work is structured and, least of all, help you identify any incorrect links. Nor is it possible to extract these links using DevOps’ built-in query tool. The fields simply aren’t available.
This leaves the DevOps API as the only way to programmatically extract these links. I used the API to traverse the Parent-Child and Predecessor-Successor links from a given starting point. Along the way, I constructed dot code that can be fed into GraphViz to render a dependency graph. I used curl to fetch API responses, flattened them with jq, and processed them with Awk to generate dot code.
I needed only the DevOps work items endpoint with the $expand=relations parameter.
I also needed to specify my project name and create a Personal Access Token in DevOps.
Using these pieces of information,
I created bash functions that let me hit the work item endpoint.
vscurl() {
local username="email@example.tld"
local token="xxxyyyzzz"
curl -sk -u${username}:${token} $1
}
If you are just getting started, you may want to remove the -s option so that you will see error messages from curl.
Then I created a function that returns the details for a given work item.
vsworkitem() {
local organization="myorg"
local project="My%20Agile%20Project"
local url="https://dev.azure.com/${organization}/${project}/_apis/wit/workitems/${1}?api-version=6.0&\$expand=relations"
vscurl "${url}"
}
Now I can invoke vsworkitem() like this:
vsworkitem 12345
This returned the raw JSON for that work item. To parse JSON into a flat format that I can use with Awk, I used jq.
jq’s author describes it as “sed for JSON data.” Like sed, jq is a powerful tool with a mind-bending syntax. It’s also not unlike learning to use a calculator that uses postfix operators or reverse-Polish notation (RPN). I think RPN and sed are still easier to grok than jq.
To output a tab-separated list of the links for a given work item, I used jq to progressively flatten the JSON structure, then output to tab-separated values.
It looks like this:
jq -r '{id: .id, assigned: .fields["System.AssignedTo"].displayName, title: .fields["System.Title"], state: .fields["System.State"], relations: .relations[]} | {id: .id, assigned: .assigned, title: .title, state: .state, url: .relations.url, name: .relations.attributes.name} | "\(.id)\t\(.title)\t\(.assigned)\t\(.state)\t\(.url)\t\(.name)"'
There’s a lot going on here. I’m not going to explain it because I still don’t fully understand it. In the end, I got the key information I needed for this work item.
- ID
- Title
- Assigned To
- State
- Relation URL (link to the relative)
- Relation Name (type of link (e.g. Child, Successor))
On Windows, jq outputs DOS-style line endings,
so I ran its output through sed -e 's/\r//'
or awk would only see a single record.
To make this easy to invoke, I created a bash script in my path that looks like this:
#!/bin/bash
vscurl() {
local username="email@example.tld"
local token="xxxyyyzzz"
curl -sk -u${username}:${token} $1
}
vsworkitem() {
local organization="myorg"
local project="My%20Agile%20Project"
local url="https://dev.azure.com/${organization}/${project}/_apis/wit/workitems/${1}?api-version=6.0&\$expand=relations"
vscurl "${url}"
}
vsworkitem "$1" | jq -r '{id: .id, assigned: .fields["System.AssignedTo"].displayName, title: .fields["System.Title"], state: .fields["System.State"], relations: .relations[]} | {id: .id, assigned: .assigned, title: .title, state: .state, url: .relations.url, name: .relations.attributes.name} | "\(.id)\t\(.title)\t\(.assigned)\t\(.state)\t\(.url)\t\(.name)"' | sed -e 's/\r//'
Walking the Tree
Now I had a tool I could use at the command line to fetch work item details that could then be fed to another tool. This tool, which I will call vsrel, accepts a work item ID as an argument and returns all of its relatives. I built a tool using Awk that would use vsrel repeatedly to build a picture of all the dependencies under a given work item.
I did this by building a queue of work items that I needed to fetch. My program pulls IDs from the queue one at a time, fetching the information for that work item. As I fetched work items, I added linked items to the queue.
There was a problem, though. If there were circular references in the data, my program would keep looping around forever! I didn’t have time for that, so I also kept a list of work items that I had already visited. If a queued item is already in the visited list, my program skips over it.
The rest of the program builds a collection of nodes and edges that are used to create dot code that can be fed into GraphViz.
Here’s the full Awk code with comments.
#!/usr/bin/awk -f
# join array elements into a string
function join(array, start, end, sep, result, i) {
if (sep == "")
sep = " "
else if (sep == SUBSEP) # magic value
sep = ""
result = array[start]
for (i = start + 1; i <= end; i++)
result = result sep array[i]
return result
}
function wrap(text, lines, lineidx, words, limit, newtext) {
limit = 25 #looks nice
newtext = text
if (length(text) > limit) {
split("",lines)
lineidx = 1
split(text,words," ")
for (w in words) {
if (lines[lineidx] == "") {
lines[lineidx] = words[w]
} else {
if (length(lines[lineidx]) + length(words[w]) < limit) {
lines[lineidx] = lines[lineidx] " " words[w]
} else {
lines[lineidx] = lines[lineidx] "\\n"
lineidx++
lines[lineidx] = words[w]
}
}
}
newtext = join(lines, 1, length(lines), " ")
}
return newtext
}
function dequeue(id) {
qidx--
if (qidx <= 0)
id = 0
else
id = queue[qidx]
if (verbose) print "dequeue [" qidx "] " id >"/dev/stderr"
return id
}
function enqueue(id) {
queue[qidx] = id
if (verbose) print "queue [" qidx "] " id >"/dev/stderr"
qidx++
}
BEGIN{
FS=OFS="\t"
# We're going to have a to-visit queue (LIFO),
# and a hash of visited items.
split("",queue)
split("",visited)
qidx = 1
# Hashes for work item details.
split("",nodes)
split("",titles)
split("",edges)
# Pre-populate the queue with work items given on the command line.
for (idx=1;ARGV[idx] != ""; idx++) {
enqueue(ARGV[idx])
}
# Field positions for the output of vsrel
id = 1
title = 2
assigned = 3
state = 4
url = 5
type = 6
# We need to go through a bunch of work items,
# finding their children and predecessors.
currentid = dequeue()
while (currentid != 0) {
cmd = "vsrel " currentid
lineidx = 0
while ((cmd | getline) > 0) {
lineidx++
relid = $url
sub(/.*\//,"",relid)
if ($state != "Removed") {
# We want to build a dict of work items and their titles as we go.
# This will be output at the end as a list of UML nodes.
if (lineidx == 1) {
nodes[currentid] = $state
# Try to get as much title on there as possible.
gsub(/"/,"",$title)
$title = wrap($title)
if ($assigned == "null") $assigned = "Unassigned"
titles[currentid] = currentid " - " $assigned "\\n" $title
}
# We only care about child and predecessor links
if ($type == "Child" || $type == "Predecessor") {
enqueue(relid)
# Build a list of edges.
# This tells us what items are dependent on other items.
# If A depends on B being done first, we say A -> B
edges[currentid,relid] = 1
# Keep track of Child links
if ($type == "Child") xref[relid] = currentid
# If we are adding a Predecessor link, and there's already a Child link,
# drop the parent-child edge since too many links add visual noise
if ($type == "Predecessor") delete edges[xref[relid],relid]
}
} else {
# This was Removed. We should bump it from the edges
delete edges[xref[relid],relid]
}
}
close(tmp)
# Add this item to the visited list and dequeue the next item.
visited[currentid] = ""
# Make sure the item wasn't already visited.
do {
currentid = dequeue()
} while ((currentid in visited) && currentid != 0)
}
# When the queue is empty, we are done.
# Output dot code for GraphViz/PlantUML/PlantText to render into an image.
# Color the nodes to match, more or less, their state in DevOps.
colors["New"] = "gray"
colors["Active"] = "blue"
colors["Closed"] = "darkgreen"
colors["Done"] = "darkgreen"
colors["Blocked"] = "red"
colors["QA Ready"] = "aqua"
colors["QA Fail"] = "darkorange"
colors["QA Pass"] = "aquamarine"
colors["Move to Prod"] = "purple"
# output dot for PlantUML
print "@startuml"
print "digraph G {"
print " rankdir=LR"
print " node [penwidth=2;shape=rectangle]"
for (idx in nodes) {
print " " idx " [label=\"" titles[idx] "\"; color=\"" colors[nodes[idx]] "\"]"
}
print ""
for (idx in edges) {
split(idx,parts,SUBSEP)
# reversing the order makes the chart easier to read for non-technical folks
print " " parts[2] " -> " parts[1]
}
print "}"
print "@enduml"
}
This generated dot code that looks something like this:
@startuml
digraph G {
rankdir=LR
node [penwidth=2;shape=rectangle]
72997 [label="72997 - Xander Harris\nMake a snack and gear up\nfor battle"; color="darkgreen"]
72998 [label="72998 - Xander Harris\nHost inspirational\npajama party"; color="darkgreen"]
72999 [label="72999 - Rupert Giles\nDefeat The Master and\nsave the day"; color="blue"]
73000 [label="73000 - Xander Harris\nTrap The Master in the\nold abandoned\nwarehouse"; color="darkgreen"]
73031 [label="73031 - Xander Harris\nOverhear the baddies\nplotting against you"; color="darkgreen"]
73033 [label="73033 - Buffy Summers\nShare the plan with the\nscooby gang"; color="blue"]
73214 [label="73214 - Buffy Summers\nStab The Master with a\nwooden stake"; color="darkgreen"]
73215 [label="73215 - Buffy Summers\nGrind bones into dust"; color="darkgreen"]
73216 [label="73216 - Buffy Summers\nBury bone dust in haunted\n cemetary"; color="darkgreen"]
74903 [label="74903 - Rupert Giles\nDevelop a contingency plan"; color="blue"]
75495 [label="75495 - Rupert Giles\nCross-reference ancient\ntexts"; color="gray"]
72998 -> 73214
72998 -> 73215
72998 -> 73216
74903 -> 72999
75495 -> 72999
73033 -> 75495
73000 -> 72999
73214 -> 72999
73031 -> 73033
72997 -> 72999
73215 -> 72999
73033 -> 74903
73216 -> 72999
}
@enduml
Here’s the output from GraphViz.
I think this is an interesting way to visualize a backlog, highlighting the dependencies among work items. It can be difficult to visualize this by drilling into the individual work items within DevOps. Right now, I build these dependency graphs for a single feature at a time. It’s certainly possible to generate graphs with lots of nodes.
