Software Architecture is highly contextual.
That kind of context that doesn’t reside only on written documents nor glamorous diagrams. The one that you build overtime as you understand more the business, the user, and the software at hand.
It’s the accumulated set of decisions that drove you where you are, enables certain futures, and locks you down from others.
The software is the result of all those decisions, some intentional while others accidental, those that survived the real world stress and chaos, and they incomplete representation of what happened that drove it where it’s today.
I’m sure you had to work with legacy systems, and understanding the full picture by only looking at the code was impossible (and drove to the wrong conclusions). You also had to speak to people to understand the whole.
Because code lacks context. It only tells you a part of the story.
And all of a sudden, we got AI. Here is my experience so far combining architecture and AI.
Lack of architecture, AI doing more of the same
We all had that feature that our architecture doesn’t support yet; like geoqueries, time series graphs, you name it.
Sometimes it’s obvious, other’s not that much.
The team gets that kind of feature to develop. AI makes a first SQL query with some JOINTS, adds an index, and the feature works in local and staging. You test it, and all good to go.
And then production kicks us hard.
You open the monitoring system, and you realize that that query won’t perform for several reasons. You’re stressing the database a lot, affecting other queries as well.
Running that query against the main DB was the wrong decision, yet, it can be unnoticed if that wasn’t caught up early on the SDLC. This example can be detected during the PR review, but think of more subtle changes that can be overseen.
In order to make that performant, we would need to either materialize the views, or move the structure to dedicated DB that supports that kind of high performing aggregation for example.
Learning:
Adding features on top of existing capabilities works ✅
Adding features that require extending the system capabilities could be undetected by the AI. The main missing context could be:
Production system demands like traffic/load/etc,
Current system limitations like not supporting geo-queries,
Available solutions/vendors that might require budget approval.
The common AI approach in this case is to keep with existing capabilities over extending them. More of the same.
This pattern locks you to the existing architecture, because it will rely on existing as much as possible.
Good if the existing architecture is good enough,
Bad if you aim to move away from it.
A person with enough backend and database experience, would have picked this issue either on design time, or PR review time. Indeed, this is usually what happens when the surface of the change is obvious enough.
But the thing here is that before, we would have paired or debated around the solution as a team. Now, we are moving away from those collaborative spaces to be more impactful, or focused on delivery. So, the amount of small nuances like the one I shared starts to happen more often.
For most cases, those bad decisions that impact production are noticeable enough to be addressed. The problem is those that do not make enough noise and accumulate over time.
Providing Context and Asking for Solutions to the AI,
and the solutions not being context specific enough to apply.
Recently, we had to architect the capability of real-time communication, a chat. If you ask everyone about how to implement that, you will receive something in the lines of:
Let’s implement a Pub/Sub with a Redis Cluster and WebSockets. Let’s aim to use a OSS implementation that handles the retires, and other infrastructure stuff for us. We keep our stateless application to handle the different messages, and business logic.
And to be fair, that’s a good solution for most cases. AI replied the same. So, we should be able to move forward with it, shouldn’t we?
Well. No.
Those solutions have implications. Not because it’s common and accepted, it should apply to us.
Maintaining a Redis Cluster, and thousands of WebSockets connections can easily become a nightmare to handle.
But this cost isn’t obvious for the team developing the feature, because that’s handled by the infrastructure team.
That decision has implications on the other team. Now, they have to keep those things up. Plus, adding real-time adds more stress to the existing application and database.
Our traffic is modest. So our solution should be. We should aim for something that we shouldn’t manage, and perform good enough for the current traffic and expect to perform well up to a x10 - x20 in traffic increase.
So, we continued to explore more approaches. And we found that a serverless stateful application based on the Actor Model to be suitable for our use case. Avoiding all the infrastructure complexity, and focus on the pure business logic.
This solution based on our traffic is peanuts 🥜.
We moved from an accepted solution by the big tech, which has organizational consequences to maintain that solution up, to a context specific one.
That one created way more debate than the accepted one. The good kind of debate. Which I enjoyed a lot by the way.
Learnings
AI provides common industry approaches to problems, but they are generic solutions.
Those approaches are usually shared by big tech, a very different context compared to us.
It also defaults to our current capabilities, and it’s hard to explore alternatives.
AI doesn’t know how to navigate organization complexities, like:
How to propose to rely on into a new vendor, or,
How to propose a separate service on a monolithic application structure.
AI went well to PoC different approaches guided by humans, you stress your architecture to see how much can it support, in terms of performance/concurrency, as well as new product needs, and/or use cases.
AI knows that’s now, but not what’s next
We are delivering this feature, but our product lead is already thinking way ahead of this. Are we moving into that direction? Is our current solution making that harder or easier to reach?
I have been lucky enough to work with brilliant business and product people. And one single thing that I know after working with them is that there is no way to do a braindump of what they have in mind.
Adding that to the AI context, as a markdown maybe? No way it happens (yet?).
You learn about their vision though conversations, debates, and endless discussions on how they see the reality, how they are modelling it in their heads.
And then my work is to:
Create shared Models between the team members to understand what’s being considered; Event Storming is still one of the best tools I found for this.
Understand first where we are, so we can understand how far we are from there, and which steps have to be done to be closer to that.
Drive a way to model that domain in code, as well as improving the architecture to support that domain model, and the different stress that it will suffer after serving our user needs.
I find it hard to encode that as context for the AI agents. Because:
It consumes a lot of context, and it provides the wrong guidance to the agents.
Should I work on this feature or the next feature? That criteria is hard to delegate, because it also adds the dimension of certainty. Should we address that now or later?
The product vision, and the domain model at hand is something you iterate on.
The more you learn, the better you get at it. Aim to delegate that, and you lose that skill of closing the present and the future.
The domain model needs to be shared between people, otherwise each person implements their own understanding of the problem. AI makes that gap between people understanding worse. Which has two main outcomes that I can think of:
You rely on AI suggestions for the implementation;
Making the solution not a business differentiator. Remember that AI is trained on existing solutions, you should create new ones, otherwise why using yours and not the existing?
You receive the feedback very late on the SDLC, usually not on the PRs, but either in the demos with the product people, or worse, in production based on customers feedback.
Other areas I’m exploring
How the org chart is supporting the changes based on how AI can drive cross-team changes, and how that affects the Conway’s Law.
DevOps culture impact. If creating is cheap, but maintaining isn’t, how that dynamic is affected.
Please, we are all learning together. Leave a comment with your learnings so far. 🙌
A drawing I made
I’m learning about watercolor, and I wanted to introduce it as part of my newsletter. I hope you like it 😄
