Legend

Legend runs affiliate websites in gaming and sports. SquadIntel was the internal analytics tool its teams used to grow those websites. C-suite, product managers, content editors and SEO specialists used it to see traffic, how visitors engage with the website, how the website ranks on search engines, whether they click out to partners, and more.

Role

Lead designer and co-PM

Team

Engineering manager, 3 engineers (frontend, backend, full-stack)

When

2020 to 2026

Context

SquadIntel was my main project at Legend. From late summer 2025 I led a redesign of the whole product, which we called Delta and released in May 2026. I was lead designer and co-PM, and for much of the redesign there was no other PM, so I made the product calls as well as the design ones. In my last six months there I also shipped to production myself.

Problem

Teams at Legend changed their websites every day. They edited a toplist (a ranked list of partner links), rewrote a title, redesigned a component, updated keywords. Each change taught them something about what grew the website. Annotations were how SquadIntel captured that.

Most analytics tools let people pin a note to a chart, like “Discovery feature release” on 10 April. SquadIntel’s annotations went further. People logged what kind of change it was, a description, how big an impact they expected, which pages it touched, and other metadata. SquadIntel then checked in after 7, 14 and 28 days and compared the metrics before and after the change. If the change worked, the team repeated it on similar pages. If it hurt, they reverted it.

annotations created a month

300+

of them never concluded

~85%

That loop rarely closed. Teams logged annotations because they were told to, then never came back to see how the change performed, let alone act on it. Wins (beneficial results) weren’t repeated and losses weren’t tracked. The answers people needed were already there, so the question was what kept them from going back for them.

I ran several rounds of research with people across the company, not always existing users and not always about SquadIntel, to learn about their pains more broadly. The problems ran through the whole life of an annotation: logging the change, waiting for results, reading them, concluding, and learning from it. No single fix would move the conclusion rate, so the redesign worked on every step. With a small team, each step also meant choosing what to build and what to drop, so the sections below cover the ideas I set aside as well as the ones that shipped.

1. Logging: annotations pointed at the wrong pages

Whenever an annotation was logged, a user had to fill in which pages it touched. Adding one or two URLs was straightforward (a simple copy/paste into the field) but anything more than 10 URLs was a pain. Those were the cases that mattered even more: a change across many pages is a bigger change, and more likely to shift metrics.

A wildcard (the % symbol) could be used to represent any string within the URL (e.g. /tents/%) but people were hesitant to use it without seeing the actual resulting pages. In other cases, the wildcard would simply not be enough to cover the entire range of pages the users wanted to select. So annotations were often attached to the wrong set of pages. Some users duplicated an old annotation just to reuse its list of URLs, carrying the wrong pages into the new one.

For changes spread across a website I added conditions: URL contains, doesn’t contain, starts with or ends with, combined with AND and OR, and mixed with page category and subcategory. A live count showed how many pages matched the criteria before the user submitted the annotation. In my interviews, people were openly relieved.

Annotation filters could be added to include only certain traffic in the annotation results. In Delta, I decided to remove them from the form, reasoning that a change can have effects outside the area being watched and filters would hide them. In practice almost everyone only cared about organic search and their share of the market. Without filters they got results full of noise, and set the same filters by hand every time.

So I brought filters back, with presets: saved sets of filters we’d already built for the dashboards, because people kept reapplying the same ones there too. One pick replaced several filters, and a starred preset became the default for every new annotation.

Delta’s form also asked for more than the old one. The more we knew about each change, the more we could learn about which kinds of change worked and which didn’t. Metadata also made annotations filterable, which investigations and auto-subscribe (further down) rely on. But every field adds work to logging, and people already logged only because they had to. So a field had to help people learn or filter, and I turned down the requests that didn’t:

• Readiness fields. Lead PMs asked for fields like who had the idea and whether the designs were ready. Annotations start once a change is live, so these checks would come too late. The real problem was that work reached production without a clear expected outcome. That needed fixing before the build, likely in team processes or our project tracking tool. Tracking it in SquadIntel as well would have created a second source of truth.
• Initiative tags. People asked for tags like “July algorithm update” to group related work. Tags describe the kind of change, not the project it belongs to. What they wanted was to see how a Jira initiative was performing, so I prototyped pulling initiatives from Jira as a filter instead. Unfortunately it didn’t make it to production during my time there, due to other priorities.
• Custom tags. Letting anyone create tags would have filled the list with near-duplicates. People could ask us to add one instead but such cases were rare.

One field did earn its place: criticality, meaning how big an impact a change could have. It started as a “major change” flag people could filter by. It became its own field, minor, moderate or major, because most annotations were small content changes, and that mattered for filtering, which auto-subscribe (below) relies on.

2. Waiting: results got lost in the noise

The 7, 14 and 28 day check-ins went out by email, as SquadIntel had no in-app notifications. Lead PMs and executives wanted to stay on top of everything and ended up with around ten emails a day. With that much noise, a check-in no longer prompted anyone to open the results, and those were the moments an annotation was meant to be read. The complaints came down to two things: changes they didn’t care about, and new pages with too little traffic to read anything into.

Emails were reduced but not enough. Most of what was left was about new pages with little traffic. So I added a visit threshold: the email only went out if the pages had a set number of visits in the last 7 days. Traffic varies too much between markets for one number, so I offered the options 50, 100, 500 or a custom value. As a result, users told us their inboxes became much more manageable and relevant.

of PMs, managers and senior stakeholders adopted auto-subscribe

90%

3. Reading the result

In the old UI the annotation report was a rather basic table of metrics with a before and after value and trend, supplemented with a multi-segment breakdown of the data in the “after” period. It wasn’t easy to find what caused a shift. The data was there, but people had to go through every segment type by hand, with nothing to guide them to what had moved. I rebuilt it to expose all segments at once: a small table for each segment type (e.g. device, country, traffic source), with tabs to switch between before and after, or between what trended up and down.

I also designed a much bigger comparison mode, a free-for-all for advanced users: any page against any page, any period against any period, with year-on-year comparisons for seasonal sports websites. It would also have covered new page launches. A new page has no “before” to compare against, so people could compare it with similar pages instead and see how similar launches had performed. People asked for this in my research. We delayed the build, and I finally made the call to scrap it altogether, as we’d seen that the new report did the job pretty well.

The harder problem was knowing whether a shift was legitimate. Annotations aren’t variant experiments. A change goes to every visitor, so at best annotations show correlation. People kept bringing up this struggle: is this a real shift, or just normal variation?

We ran an experiment to let the numbers decide. When creating an annotation, people would say how much they expected a metric to move, and that would decide whether it was a win. This didn’t work. People didn’t know what shift to expect, and when something moved they still couldn’t tell if it was real.

So I prototyped a significance signal that conveyed “this movement is likely worth looking into”, and a snooze for results that weren’t significant yet. Both depended on data work from other teams that we couldn’t get at the time, so neither shipped during my time there.

Instead we shipped Reviewers. Content editors created the most annotations, and they could see the SEO data in SquadIntel, but reading it well takes an SEO specialist’s skills. So I modelled annotations on GitHub’s pull requests: whoever opens one adds reviewers to check their code. Each reviewer reads the data and gives their own verdict before the author concludes.

It was a drastically different direction. It added work for people, but it brought everyone up to date on a change and put the right expertise on the interpretation problem. Not every annotation got reviewers. The ones that did tended to be significant work that needed deep interpretation. The ones without were mostly logs, like “new page launched”, with no “before” data to compare.

Overlapping changes made judging harder still. Teams often shipped several changes to the same pages within days. I designed a timeline showing which annotations overlap, and a notice suggesting the best time for the next change on a page. I received pushback on the notice: it would make people afraid to ship. Therefore I dropped it in favor of the timeline view. It aimed to inform rather than block users from creating new annotations.

4. Concluding

In the old UI, annotations were a plain list. A list works, but it carried no idea of a workflow: it read more like an inbox. The new annotation experience shipped as a board, with reviewers and conclusions together, and I saw it as the most valuable part of the redesign.

Annotations started in Measuring. After 7 days they moved to In review while reviewers gave their verdicts, then to Ready to conclude, and stayed on the board with reminders until someone concluded them. Concluding opened one dialog. It showed the reviewers’ verdicts, asked the owner for a final interpretation, and asked for an outcome: win, loss, no impact (enough data, but nothing moved), inconclusive (not enough data to interpret). A loss also needed a next step: revert, already reverted, needs more work or nothing to do.

The first version was more involved. People moved annotations between statuses by hand, the write-up lived in a separate Analysis tab, and follow-ups were a free-form checklist. I went a different direction for all three. The annotation’s lifecycle defined where it appeared on the board. The Analysis tab was out of sight, so I suspected people would skip it, and instead I moved it into the Conclude action to reinforce the workflow. The checklist was more flexible than anyone needed, so it turned into the four choices above.

5. Learning from it

A concluded annotation only paid off when someone used what it taught. That took remembering a past result existed and going to find it, and few people did.

My first designs for this step gave the learning a place of its own: a library of concluded annotations, a hub of reports on which kinds of change paid off, and an ideas board with votes. Each one needed people to remember to visit, and that was the habit missing in the first place. Little of it shipped during my time there, partly due to other priorities.

So my later designs brought the learning to where people already were. A page’s dashboard got a short AI summary, and a button on an annotation or a page returned AI-generated recommendations, each of which could become a Jira ticket. Recommendations needed technology we didn’t have when the board shipped, and they only worked once there were concluded annotations to learn from. Together these closed the loop: idea, release, measure, learn, and back to the next idea.

How it shipped

The new annotation experience went to an opt-in beta first, and 60% of the people who tried it chose to stay on it. We fixed what the beta group ran into, released it to everyone, and finally switched off annotation creation in the old UI. Other parts of Delta came in stages after launch.

Impact

Before

~15%

of annotations concluded

After

~80%

concluded, each with an interpretation and a next step

The conclusion rate flipped. Most of the annotations still open were new pages, with no “before” to compare against.

With annotations and other areas of SquadIntel doing more of the work, teams relied less on Adobe Analytics and Tableau, which came with a reduction in licensing costs.

of the addressable user base used SquadIntel every month

81%

report views a month

5,000+

What I’d do differently

• Comparisons that fit the change. Before and after only works when there is a before. The method of comparison would follow the nature of the change: a new page, for example, would be compared with similar pages when they launched. That would cover most of the annotations that stayed open.
• Learning on the page itself. An AI analysis of every conclusion, suggesting what to try next on each page or category of pages, shown on that page’s dashboard. It didn’t get designed during my time there, but it would have put the learning where people already look every day.
• An Inconclusive outcome. Concluding offers win, loss or no impact, and no impact means there was enough data but nothing moved. Some results couldn’t be settled either way. I wanted an Inconclusive state for those, so they weren’t forced into an outcome they didn’t earn. I suspected this was another reason people held back from concluding annotations.
• Investigations as their own thing. Advanced analysis would work better as a standalone tool rather than tied to one annotation, with comments built in. People could save an investigation, share it with others, and the saved investigations would feed the AI analysis too.