Part 6: When the Loops Broke

My first semi-working architecture was a monolith. One workflow that did everything: loop through companies, fetch jobs for each one, loop through jobs, evaluate each with AI, accumulate results, send the email. Elegant on paper. Fragile in practice.

n8n has opinions about how data flows. Nested loops — a loop inside a loop — don’t behave the way you’d expect if you’re coming from traditional programming. Variables set inside the inner loop couldn’t be seen outside it. Arrays that should have iterated through every item only ran once. I’d get partial results, or no results, or results from the wrong company attached to the wrong jobs.

I fought it for a while. Tried workarounds. Asked Claude for alternatives. Some worked temporarily, then broke when I added the next piece. Eventually I hit a wall I couldn’t code around. The monolith had to come apart.

So I refactored. One workflow became four: a main orchestrator, a sub-workflow that loops through companies, a sub-sub-workflow that processes jobs for each company, and a final sub-workflow that aggregates everything and sends the email. Each piece has a single responsibility. Each piece can be tested independently. Each piece fails without taking down the others.

Here’s the thing: I didn’t choose this architecture. I was forced into it. But it’s better than what I would have built if the nested loops had worked. The monolith would have been harder to debug, harder to extend, harder to explain. The four-workflow version is cleaner. More resilient. Easier to reason about.

Sometimes the wall you hit is doing you a favor.

Next up: thinking about the next user before you’re done being the first.

Categories:

Leave a Reply

Your email address will not be published. Required fields are marked *