5 AI tools to build to be a better Engineering Leader in 2025
5 stories, 5 tools, 5 actions to take to level up as an engineering leader in the gen AI world!
CodeRabbit - Cut code review time & bugs in half with AI-powered reviews (sponsored)
Unblock your team's productivity with intelligent code reviews. CodeRabbit is an AI code reviewer that helps engineering teams ship faster while maintaining high code quality standards.
CodeRabbit provides:
Intelligent PR summaries and guided file-change walkthroughs
Battle-tested linter integration (Biome, Ruff, PHPStan)
Proactive security issue detection
Custom review rules using AST grep
Used by 2000+ organizations and trusted for 5M+ PR reviews, CodeRabbit helps teams eliminate review bottlenecks and maintain consistent code quality. And yes, it's free for open source projects!
Join 15,000+ developers who use CodeRabbit daily to streamline their code review workflow.
Let’s get back to this week's thought!
Intro
The future of engineering leadership isn’t just about managing people. It’s very important to leverage different tools to expand your impact.
As AI continues to progress and new tools are being created almost daily, it’s important to keep finding ways to use different tools to help you. The most effective leaders in 2025 and beyond will do exactly that.
The best way to start?
Start building your own tools that’ll make you a better engineering leader. You’ll increase your overall knowledge of AI, LLMs and different tools available + you’ll build useful things to help you!
Lucky for us, we have Gilad Naor with us. In his career, he worked as an engineering leader for more than a decade for different companies including being an Engineering Manager for companies like Amazon and Meta.
Let’s get straight into it!
The times they are a-changin'
Every week there is another groundbreaking Large Language Model (LLM). It’s exhilarating. And exhausting.
As an engineering leader, you already have so much on your plate. You have to:
Coach a struggling engineer
Interview candidates for the team
Work with stakeholders to align on what to build
Handle the latest crisis, like the defect that broke prod.
And that’s just before you’ve had your morning coffee break.
How do you find the time to stay up to date on the latest AI change?
Here is a powerful tip that you can steal: Benchmark yourself against the Best.
I know that you are busy. Are you busier than Dr. Werner Vogels, AWS’s Chief Technology Officer (CTO)?
A few months ago, he went out and built Distill CLI. It’s an AI tool to capture meeting notes and action items. He uses it when he meets members of his “Office of the CTO.” This is a small team of Distinguished Engineers who work closely with the CTO.
Why did he carve time out of his busy day to build this?
Three reasons:
He had an itch to scratch.
He wanted to learn more about AI.
He wanted to learn more about Rust.
The best way to learn is to build something that you need!
In today’s post, I will show you how to build your own “Office of the CTO,” or “Office of the EM.”
I will share five management stories. For each story, you will learn how to build an AI tool to help you. And in each step, you will learn more about how to make the most out of LLMs.
Once you’re done, I’ll share three mistakes to avoid.
Let’s get started.
Five stories, five tips, five tools
1. The (nearly) botched promotion
The story: The promotion was supposed to be a no-brainer.
Out of nowhere, a senior manager from another team pushed back. His take: she didn’t help an engineer on his team.
My saving grace?
I took copious notes during our 1:1s together. In a couple of hours of work, I was able to find the relevant notes from a few months back. Turns out that the other team didn’t need her help.
Promotion saved.
The tool to build: The first persona that you should build for your Office of the EM (O-EM) is your Executive Assistant. This agent will help you prepare for upcoming meetings, search through your notes, and form connections between meetings.
The insight: LLMs are only as good as the information that they have access to. You can expose your code base and documents to the AI. However, AI cannot help you with information that is not saved anywhere.
The action: Start transcribing and summarizing all of your meetings. Including your walking 1:1s. If you want to build your own tool, OpenAI’s open-source whisper model is a great tool to use.
2. The VP review
The story: I remember the first time I had a VP-level review at Meta. My mentor’s biggest advice was to identify what the VP, and only the VP, can help us with.
The secret to communication is to understand, deeply, what the other side cares about.
This is challenging to do when the other person is a VP at a $500B corporation.
Luckily, I had access to a mentor with enough experience working with VPs to help guide my preparation.
The tool to build: Build an Empathy Bot. This AI will help you prepare for meetings with executives, cross-functional partners, and even customers.
The insight: LLMs are trained at the internet scale. When you chat with an LLM, it doesn’t know if you are an engineering manager, a car mechanic, or a 12-year-old boy. This is why so many AI responses feel like generic crap.
The action: Start your queries by telling the LLM who they should be. What their title is. What their role is. How many years of experience they have. The more information that you provide, the more the AI will hone in the most relevant data at its disposal. This dramatically changes the quality of the advice that it can share.
3. The editor
The story: This time, I knew that the deadline was set in stone. As in, the hardware will literally be pulled out of the racks set in stone.
It was a complex project that spanned multiple halves and 7 engineers. And then a key partner and subject-matter expert left the company.
I knew that I had to update all the stakeholders on the new risks as soon as possible. I knew that I had to nail the writing.
I did an okay job. Not great. This led to more back and forth, slower reaction times, and some lost trust.
The tool to build: To help you in your writing, you need your own writing editor in your Office of the EM.
The insight: Most LLM-produced text is generic. The secret is to train the LLM to write in your own voice. What most people miss is the diminishing costs of fine-tuning a model. 90-95% of training costs go into building the raw foundational model. Another 5-10% goes into post-training. The cost of fine-tuning a model? It’s just a rounding error.
The action: Use “in-context learning” to train the LLM on the fly, directly in your prompt. Ask an LLM to summarize three of your emails in one line. This then becomes the example “questions,” with the emails as the “answers.” Copy these question-answer pairs into the end of your query. This will drive the LLM to write text in your own voice.
4. The devil’s advocate
The story: We built a tool. It worked great for internal users. But did it even matter?
The insight came when we were able to prevent a journalist from writing a damaging story. Not because we were able to hide the truth. But because we were able to react much faster to their query and provide them with data that completely turned around the story.
Who would have even thought about PR risk?
The tool to build: Our tool for the Office of the EM is the Devil’s Advocate. Their job is to help us look at each and every challenge from multiple points of view.
The insight: LLMs are averaging machines. The broader the boundaries in which they average out, the less meaning is captured in their response. A tighter and narrower query leads to a higher quality answer.
The action: Do you remember the Single Responsibility Principle (SRP)? Apply it to LLMs. Write separate agents to highlight risks in: security, privacy, legal, PR, etc. Create small queries and chain the response together at the end.
5. The management coach
The story: It took me months to take action. My manager and my skip kept pushing me to take more direct steps with an engineer that was no longer meeting expectations.
I kept pushing back. I had every excuse imaginable. In hindsight, I now know that I was afraid. And I let that engineer down.
The tool to build: Finally, we will build our own coach. The tool that can help us manage ourselves.
The insight: LLMs provide us with a safe space. They create the perfect opportunity to share our concerns, fears, and challenges.
The action: Create three AI agents to help you prepare for difficult conversations. The first will brainstorm your engineer’s potential reactions and moods. The second will role-play as the engineer so that you can practice the conversation. The last will review the conversation and provide feedback.
The future
As you build these tools, you have to be aware of three critical challenges.
The 7/38/55 rule
Albert Mehrabian’s 7/38/55 rule, while often misunderstood, is directionally accurate.
7% of the emotional information in an interaction between two people is conveyed in the actual words.
38% is captured in the tone of voice.
55% is encoded in the body language.
Today, LLMs only help us with the first 7%. There is good reason to believe that this will be true for the near term.
Gathering training data for the semantics of intonation and body language has fundamental challenges.
When you have a high-stakes management challenge coming up, talk to an actual person.
The risk of averages
Large Language Models are averaging machines. They are great at it.
The tighter you make the boundaries in which they do the averaging, the higher quality output you will receive. There is a darker side to this, though.
The moment your boundaries go outside the scope of what they know, their quality goes down rapidly.
This is easy to see in generative AI models that create pictures and videos. Just try to create an image of a left-handed guitar player, or a full glass of wine.
The key insight is that this limitation exists for text-based models as well, it’s just less, well, visual.
People and problems
All of business is just three things:
People working together to
Solve a problem
Using tools.
People. Problem. Tools. This fundamental truth is not going away any time soon.
The tools that you can use are changing rapidly. The problems to solve and the challenges of influencing people are not going away.
Last words
Special thanks to
for sharing his insights on this important topic with us!Make sure to follow him on LinkedIn and also check out his FREE lightning lesson Stay Technical as an Engineering Manager With These 3 Tips.
The lightning lesson will be on April 30.
We are not over yet!
I am NOT a Fan of Heroism in the Engineering Industry
Check out my latest video. I am sharing why I am not a fan of heroics in the engineering industry. I value predictability, consistency and reliability + overall long-term good work.
New video every Sunday! Subscribe to not miss it here:
New benefits for paid subscribers → Infographics!
Paid subscribers can now get different infographics for becoming a great engineering leader.
Infographics are practical & actionable guides that contain important information about the specific topic (from an article) condensed in one page!
There are 4 infographics currently available:
How to Build Credibility in The Engineering Industry (get this one for FREE here)
How to Propose an Impactful Improvement to the Codebase and Own the Implementation
How to Build Good Relationships Inside and Outside Your Engineering Team
8 Signs That You Have a Good Manager
Paid subscribers can get them all here: 🎁 Products for paid subscribers
More infographics coming soon!
Liked this article? Make sure to 💙 click the like button.
Feedback or addition? Make sure to 💬 comment.
Know someone that would find this helpful? Make sure to 🔁 share this post.
Whenever you are ready, here is how I can help you further
Join the Cohort course Senior Engineer to Lead: Grow and thrive in the role here.
Interested in sponsoring this newsletter? Check the sponsorship options here.
Take a look at the cool swag in the Engineering Leadership Store here.
Want to work with me? You can see all the options here.
Get in touch
You can find me on LinkedIn, X, YouTube, Bluesky, Instagram or Threads.
If you wish to make a request on particular topic you would like to read, you can send me an email to info@gregorojstersek.com.
This newsletter is funded by paid subscriptions from readers like yourself.
If you aren’t already, consider becoming a paid subscriber to receive the full experience!
You are more than welcome to find whatever interests you here and try it out in your particular case. Let me know how it went! Topics are normally about all things engineering related, leadership, management, developing scalable products, building teams etc.
 | A guest post by
|
Love these ideas for AI usage, especially "the editor" and "the devil's advocate." I'm running almost all my docs through AI for critical feedback, and it helps me build out a stronger, well-thought-out doc
I’ve set up five conversations with different prompts to simulate the five AI bots you mentioned. Let’s see how they go!