I added a ‘Self and Career’ section to the newsletter. Those articles will be primarily non-technical and will go out two weeks early to paid subscribers. Thank you for your support!

Things are moving so quickly in AI it can tough to know what topics to learn, what’s important, and what to focus on. Part of this newsletter is sharing that with you along …

Read more

There’s been an interesting push recently for “tokenmaxxing”, or the idea that burning through more tokens means an engineer has been more productive. The thought process is more tokens mean more AI use, which means getting more done and saving more time.

In reality, the number of tokens used is a development velocity metric similar to measuring lines of code written: It’s not only inaccurate, but can actually be measuring the opposite: a decrease in development velocity. For more on measuring developer velocity via lines of code, read -2000 Lines of Code.

Measuring tokens used results in a manifestation of Goodhart’s Law: “When a measure becomes a target, it ceases to be a good measure”. This creates a utility gap in what the measurement actually produces: Users prioritize models that burn tokens, redundant context over precise requests, and unnecessary agentic use for small work. Instead of engineers using proper techniques to be more productive with AI tools, they measure their productivity with how high up they are on a token spending leaderboard.

My experience has shown the opposite of tokenmaxxing (which I like to call “tokenminning”) is ideal for high velocity agentic development work. I’ve found my token use to directly correlate with the time I spend on a feature. If my agent is burning tokens, it generally means more time spent by me monitoring and reviewing that agent’s progress. This is directly contrary to the primary goal of agentic development tools: Enable engineers to accomplish more in less time.

To achieve this goal, it’s key to rely less on agent reasoning and ludicrous token spend and instead focus on a structured, well-thought-out engineering process that makes it easier for the coding agent to understand your request and adhere to it.

In this article, I cover topics to help you make your agentic development faster. We’ll go over:

• My 3 tips for making agentic development use less of your time.

• The tip I think is less helpful than most make it out to be.

• A better measurement for agentic development velocity.

If you really want to get deeper into proper agentic development, don’t miss Packt’s Hands-on Spec-Driven Development Workshop coming up on May 14th.

This is a hands-on workshop that teaches you how to be more consistent when building with AI via spec-driven development. You’ll create a real application while learning how to define clear specs, guide AI reliably, and reduce rework.

This is the second cohort for this workshop after the first sold out. There are only 8 spots left. Secure your spot now and get 45% off with code LOG45.

I highly recommend Packt’s education resources so I’m excited to bring this discounted opportunity to you. The above is an affiliate link to help support the newsletter at no extra cost to you.

My top three tips

From working with agents both in and outside of work, here are my top three tips for making your agentic development workflow faster for you. The primary focus of these tips is on saving your time instead of token costs for your company.

Below are also great tips for anyone building agents. Among other things, agents are fundamentally a context engineering problem. When you’re working with AI coding tools, you’re curating that context in real time.

1. Use smarter models

Ever since Boris Cherny explained that he only uses Opus for software development at Anthropic back in January, I’ve been trying to do the same with my own agentic development.

His thesis is using a larger model and spending more per token results in far less overall token use than using a smaller model on the same task. The smaller model requires more steering and rework, resulting in tokenmaxxing and costs more overall in the long run than using the more expensive model per token.

In proper agentic development, I’ve also found this to indeed be the case. Laying out requirements and setting up proper architecture is much more easily done with a more intelligent model with reasoning capabilities. It not only improves spending as mentioned above, it also decreases the time an engineer spends correcting the model. While this might seem contrary to tokenminning philosophy, it actually supports it in the long run.

A tip I’ve frequently seen shared to get better cost-performance from agentic tools is to route models according to the task. The advice to ‘use a smaller model when performing a simple task’ is something I recommend against.

This is a very hard problem space. It’s very difficult to reliably predict which tasks a small model will be able to do on its own and which a larger model is needed for. I’ve used automated model routing for many tasks and can immediately tell when a small model has been prioritized. Even on teams that have the data necessary to make these decisions, model routing is difficult.

I recommend using larger, smarter models whenever possible as the time and cost trade-off in the end is worth it. One exception to this rule is for hobbyist projects. In that case, I recommend using whatever you want. Small models are often good enough in that environment and spending $100/mo on a subscription isn’t worth it.

See my article on that here:

2. Be precise

The biggest downfall of coding agents is that they try to do too much. I’ve seen this manifest in random changes being made to unrelated files when an agent works on a task, an agent going beyond its current work on future tasks without being provided permission to do so, an agent repeating already finished work, and agents running unnecessary CLI commands in the midst of a task.

My biggest quality of life gain has been reining in these models to ensure they aren’t going off the rails. This is a harder task than it seems, but the best way to go about this is to be very precise in your requests to the agent and the environment you set up for them to work in.

Here are the considerations I take to be more precise:

1. Do some engineering before you start coding. Understand the problem and the code that needs to be written so you can properly direct the agent in the tasks it needs to do. You should know what needs to be done before the agent starts working. If you want to better understand how to do this, check out the workshop above.

2. Scope your requests to the agent to manageable tasks. Agents tend to go off the rails when you give it too much freedom to interpret implementation details. Be explicit about what you want the agent to do and the changes that need to be made to make that happen. This can’t be done without step 1.

3. In your requests, try not to make any typos. I’ve found this bit of precision helps an agent understand a request much better.

4. Manage context based on the task. As context gets larger, the agent has a greater chance of working outside of its scope. This means using separate chats for separate tasks, as the entire chat is passed in each time a message is sent. You don’t want information from a previous task polluting the task you’re completing now.

5. Be mindful when curating what the agent has access to and continually adjust this as you work. For example, I’ve found some agents to perform poorly when using source control tooling. I’ve removed that agent’s ability to call those tools and I do it myself instead. This also applies to the skills and MCP servers the agent can access which have the potential to pollute context and hurt performance.

3. Write manual code

This one might seem preposterous to anyone else who is chronically online, but, yes, the art of manually writing code is not dead. I’ve seen many people resort to only coding with AI even to the point of prompting for tiny changes.

You’re going to save a lot of your time spent waiting for agents to understand your request if you take care of small changes yourself. It’s often wise to ‘Accept All’ to an agent’s output and make minor tweaks on your own instead of telling the agent what’s wrong and have it do it for you.

Examples of these changes include:

• Renaming a variable to be more descriptive.

• Simple readability fixes.

• Fixing or removing imports.

The only downside to manual coding is that you need to tell the agent you tweaked the code, otherwise, the next change by the agent will end up overwriting your work. If you don’t let the agent know, this can be a serious source of repeated work.

Don’t fall for the “I haven’t written a single line of code” narrative. I don’t know of a single engineer working on large-scale production services that doesn’t write some of the code by hand.

A better metric for agentic engineering velocity

The current best way to measure agentic development velocity is a metric called stickiness or code retention (but I prefer stickiness). This is a measurement of how much AI-generated code lasts without needing to be tweaked and without being overwritten soon after.

This metric is much better measured by AI coding tools to understand their efficiency, but it’s also something engineers should keep in mind when using their tooling. If their AI-generated code is sticky, it’s likely they’re working productively with the AI. If it isn’t sticky, there might be too much rework occurring, which means the above tips should be worked on.

—

Tokenmaxxing is a silly way to measure developer productivity. The most important metric you should maximize with agentic engineering tools is the amount of time you reclaim to work on other things. Tokenminning, in my opinion, is the way to do this.

What tips do you have for faster agentic development? I’d love to hear them in the comments.

If you enjoyed this article, don’t forget to subscribe to AI for Software Engineers to get more just like it in your inbox.

Subscribe now

Thanks for reading!

Always be (machine) learning,

Logan

Share

I’ve conducted enough entry-level technical interviews to identify the patterns and mistakes most candidates make. Below I detail the top three things you can do to avoid common mistakes and separate yourself as a top candidate during the interview process.

This is specifically regarding technical Leetcode-style interviews (not system design, although some of the information might apply to both). I’ll share a question I’ve asked quite a bit recently and how each of the tips below applies to it.

Each tip below also applies to a part of the technical interview evaluation that does transition to real-world software engineering skills. I know this isn’t always the case, but I feel these are especially important for anyone being evaluated for a software engineering role.

Note: While I work for Google and have access to Google interviewing resources, everything below is my personal opinion. Interviewing is a very human experience, after all.

What an Interviewer is Looking For

There’s always a lot of focus on finding the optimal solution in a technical interview and being perfect in your reasoning for how you arrived at that solution, but an interviewer is looking for much more than just that.

They don’t just want to evaluate your technical knowledge. They want to understand how you think. Understanding a candidate’s reasoning along with their knowledge of software engineering fundamentals tells them a lot about how you will perform on the job.

When you’re interviewing at this level, your interviewer will focus on three things:

Read more

I’ve seen an interesting new fad on social media recently that I like to call “vibe releasing”. This is the same as “vibe coding” but it takes it one step further and releases the code to production without properly reviewing it first.

I can’t overstate how terrible of an idea this is.

In fact, this year’s “State of AI-assisted Development” report released by Google centered around one idea: AI is an amplifier. It analyzes AI coding metrics from this past and proves that coding with AI makes proper engineering practices more, not less, important.

It shows that companies with good engineering culture and practices will see AI positively impact their development velocity and companies with bad engineering culture and practices will see the opposite. “Vibe releasing” is the definition of a bad engineering practice.

This article includes everything you should take away from Google’s report and how it applies to you.

Takeaways

If you’re just here for the takeaways, here they are:

• 2025 was the first year AI had a quantifiable positive impact on software development.

• Trust is a huge factor in AI coding tool effectiveness.

• Companies with bad engineering cultures and practices will see their development velocity slow with AI. Conversely, companies with good engineering cultures and practices will see their development velocity quicken with AI.

If you want to know the specifics and what your organization should do to ensure AI works for you instead of against you, read on.

Report methodology

First, let’s understand how the report was created and how research was conducted. When evaluating metrics, this is always the first step.

Read more

Last week I posted a role my team is hiring for on X and LinkedIn (check them out because we’re still hiring!) and I received hundreds of messages. Sorting through them made one thing clear—people suck at presenting themselves.

It’s a tough market right now and differentiation is more important than ever. Most great jobs aren’t found by cold applying to positions, but through one’s network. DMs are the new cover letter and understanding how to DM properly is paramount to optimally present oneself.

This article uses my experience on the hiring side of DMs to teach you how to DM for a job properly. I share this primarily because I think it’s important and AI for Software Engineers readers should stand out, but also because I learned a lot about what I was doing incorrectly too.

I’ve split this into five parts. In this new

sletter, I explain:

1. The current job climate and why that’s the first thing to understand.

2. How to set up a good elevator pitch.

3. Why your resume is still important.

4. The things you should avoid.

5. One final tip.

1. Understand the current job climate

The current job market is weird. Overall demand for software engineers has gone down, but demand for developers with experience in AI is at its peak. This means we have a lot of people competing for jobs, but for many roles there are very few qualified applicants.

To put things into perspective, only a few hours after posting about the open role, I had ~100 DMs on X and about half that on LinkedIn. Over the next few days, I had hundreds on both platforms. Yet, only a small group of candidates were actually a good fit.

In the current job market, you should understand that:

1. You need to stand out immediately. Your messaging should be efficient.

2. A response may be delayed. You shouldn’t expect a response in hours or even up to a day. Be smart about when you follow up.

3. You may never get a response. Responding to everyone isn’t feasible due to volume.

4. You need to be prepared. Your resume and other supporting materials should be ready to go.

Your goal is to make it easy for the person reviewing your experience. I can’t emphasize this enough: Making it easy for the person to review your work makes a huge difference in how likely you are to make it further in the hiring process.

With these things in mind, let’s figure out how we can make this process easy.

2. Developing a good elevator pitch

Update: After spending a lot of time going through DMs I realized it makes my life a ton easier when a person sends their resume immediately with their elevator pitch. Send it as a pdf via shareable link (such as Google Drive).

Your goal with your first message should be to fit enough information in a blurb readable in fifteen seconds to make the person feel they’re missing out on talent if they don’t respond to you.

There are two things to pull from that.

First, your message should be readable in fifteen seconds or less. If you send a wall of text containing your entire CV, it won’t be read. You’ll be skipped and the person hiring won’t think twice. They simply don’t have time for this. Remember, you don’t need to list all your experience, just the most relevant and recent.

Second, your message should be convincing. It should figuratively scream “I’m the person for this job.” So many people fall into the trap of just sharing what they do instead of the impact they’ve had.

Those hiring care much more about the outcome of work than the work done. When you share the impact your work had, you’re also forced to get into more detail about it. Descriptions of ‘what someone has done’ tend to be generic. I can’t tell if you’re the right person for the job based on a vague description of work. I can tell much better if I understand the outcome of that work.

I responded to so many people with ‘Tell me more about your work’ because they only sent me a generic description of what they did. In many cases, when they responded with more detail, their work was much cooler than they had originally made it seem.

Here’s a bad DM:

“Hi, I’m a software engineer with 5 years of experience in AI and machine learning. I’ve worked at Company X and Company Y building models. Let me know if you want my resume. Interested in your role.”

Here’s a good, very similar DM:

“Saw your AI engineer role—I’ve built production ML systems at Company X that boosted user engagement by 35% using PyTorch and deployed via AWS. Resume: [link]. Excited to chat about how I can contribute.”

Outside of the content, notice how the second message sent a resume link immediately. This isn’t a link to a LinkedIn profile, portfolio, or personal site. This is a link to a polished resume. The reader doesn’t have time to visit a website for everyone to look for information. You need to bring the info to them.

Interestingly, the quality of DMs on LinkedIn was much higher than X in general, but qualified candidates overwhelmingly came from X.

3. Your resume is still important!

The number one issue I see when looking over resumes is bullet points that share what someone did instead of the impact they had. I mentioned this in the previous section, but it’s by far a bigger issue on a resume than in a DM.

When you focus resume bullet points on the impact of your work, you’ll not only share what the work was, but why it was important and better communicate what you’re capable of.

A bad bullet point:

“Trained machine learning models for image recognition using TensorFlow.”

A good bullet point:

“Achieved 92% accuracy for image recognition tasks and reduced image processing time by 40% by training a machine learning model on a 50k-image dataset using TensorFlow.”

To write excellent resume bullet points, I recommend the XYZ formula: “Accomplished [X] as measured by [Y], by doing [Z].” You can find more examples of good and bad bullet points at that link.

The second issue is poor formatting and non-important information. Your resume should be a single page. Those hiring won’t have time to look at a second. The single page should contain the most relevant information first and call attention to the technologies and skills your work shows you have.

Your resume should not contain a skills section that lists your proficiency in languages and technologies. Those proficiencies should come across in your bullet points. As you share your impact, include what you used to make that impact and bold those technologies so they stand out.

Take care to draw attention to the technologies listed in the job description. This is a really easy way for a recruiter (especially one with a non-technical background) to decide you might be a fit for a role.

Some examples of this:

“Increased user retention by 25% for a platform with 1M+ daily active users, by leading development of scalable recommendation engine using Python, PyTorch, and Kubernetes.”

Or:

“Increased user retention by 25% for a platform with 1M+ daily active users, by leading development of scalable recommendation engine (Python, PyTorch, Kubernetes).”

The fourth issue I see is candidates not being honest about their experience. I’ve seen very few people outright lie about their experience but I’ve seen a lot of people embellish it. Lies and embellishments come across very quickly when having an actual conversation with someone about what they’ve done and it doesn’t look good.

This also frequently comes across when looking over someone’s resume. The experience listed in your DM should match the experience listed on your resume. I should be able to look at your resume and think “That’s what they were talking about.” I’ve seen too many resumes that seem like an entirely different candidate than was portrayed via DM. That’s a red flag to me.

The final issue I see in resumes (and very frequently when chatting with candidates) is using the term we or similar group phrasing. Your resume and candidacy are about you. You need to specifically share the impact you had.

A tip: Tailor your resume to each job listing. Other candidates are doing this and you’re competing with them. You should match your experience exactly with what the hiring team is looking for if you want the best chance of making it through to the hiring rounds.

4. Things not to do

Before we finish, here are a few things you shouldn’t do when messaging someone for a job opportunity. I experienced all of these and more this past week.

1. Don’t spam (in messages or elsewhere). Many people would follow up with a ‘gentle ping’ two hours after sending their previous message. Remember, the person you are messaging is looking for someone to work with. Be respectful because they definitely won’t want to work with someone who isn’t.

2. Don’t use AI for these messages. It’s very obvious and a very quick filter anyone can use to eliminate candidates. If you’re using AI to write, your DM will be poorly written and super generic. It only takes a minute or two to draft a worthwhile message.

3. Don’t be threatening. I don’t know if this is a result of the alpha male mentality spread around the internet, but anyone telling me I must hire them or I’ll regret it is immediately disqualified.

4. Don’t just say ‘hi’. This is a good rule of thumb for all professional communications. Time is limited and sending ‘hi’ is a waste of it. Get right to the point.

5. Don’t DM just because you want to ask a question, yet. This isn’t because it’s disrespectful, but simply due to logistics. I’m happy to answer questions via DM, but I won’t have time when filtering through job-related messages.

One final tip: Do things in public. Learn new skills, build cool things, and share what you’re doing online. This is the differentiating factor for getting a job in the current market.

If I’ve missed anything, leave it below! Thanks for reading.

Always be (machine) learning,

Logan

Share

You can find both of these below:

This list contains the top 10 skills every software engineering working in (or wanting to work in) AI should be focusing on in order of how frequently they occurred on AI-related engineering job descriptions.

My methodology for gathering my data points isn’t 100% foolproof, but I’ve been tracking job listings for close to 6 months now while focusing on companies with the following 3 qualities:

1. A high hiring bar. I know these companies put time and effort into their hiring process to get the best engineers. They’re more likely to provide more accurate and higher quality job listings.

2. Are doing interesting work in AI and ML. Most were companies at the forefront of AI including startups and large tech companies. These are companies that are innovating in the space and likely to use the most up-to-date technologies.

3. Companies that pay well. A huge reason this is interesting is so you can learn the skills that will let you increase your compensation.

Let’s jump right in!

The most-mentioned skills in job descriptions

1. Python

This one is kind of obvious, but learning Python as a software engineer has a huge return on investment. The entire ML ecosystem is built on top of Python code, and the large majority of machine learning libraries are built to work with it.

About 90% of job descriptions mentioned Python. In my opinion, this is a language all software engineers should learn anyway for interviews, as it removes a lot of the language-related hardship so you can focus on the data structures and algorithms.

2. PyTorch

PyTorch is the dominant deep learning framework across the industry. Companies hiring for AI engineering roles expect engineers to be fluent in PyTorch because it underlies most modern systems. PyTorch was included as a requirement for about 70% of roles.

It’s the easiest deep learning framework to get started with and, in my opinion, the most readable. I’d say it’s the most beginner-friendly, even though the framework you use in industry will depend on the company you work for. The other popular frameworks are also included in this list, just further down.

3. Kubernetes

Kubernetes is the infra layer that allows ML systems to run at scale. It shows up in nearly every listing that touches ML infrastructure because it allows containerized workloads to be deployed, managed, and scaled across clusters. Kubernetes was also mentioned in about 70% of roles.

Kubernetes is the standard way to train and serve models on multi-GPU clusters. Training and serving in production rarely occurs on one machine. Kubernetes handles scheduling, scaling, and fault tolerance for large workloads.

4. Cloud Platforms (AWS, GCP, Azure)

Most companies are running their training and serving on one of the three major cloud platforms. These showed up on job descriptions about 70% of the time combined. To me, having experience working with one shows that you can learn how to work with the others. Most companies seemed to share this opinion, though some asked specifically for experience in one or the other.

AWS was mentioned the most, then GCP, and then Azure. I think Azure actually has a more dominant market share, but the switch to TPUs for many companies has them using Google’s infrastructure. Understanding cloud platforms, the tools they offer, and how to use those tools is beneficial for an AI-related job search.

5. SQL

Any time you’re working in AI, you’ll be working with data. Anytime you work with data, understanding SQL is a must. There hasn’t been a single role that I’ve worked that hasn’t required SQL. It appeared in about half of job descriptions.

Being able to inspect data is a must for any engineer wanting to work in machine learning.

6. C++

C++ shows up consistently in job listings for performance engineering, inference optimization, and GPU-related work. While Python dominates at the surface, the underlying libraries and kernels are written in C++.

For an infrastructure-related role or any role focusing on ML efficiency, C++ is the go-to language. It appeared in about half of the job descriptions.

7. LLM APIs and Prompt Engineering

Companies want engineers that already have experience working with LLMs. Building systems with AI is fundamentally different from building traditional software systems because of their non-deterministic nature. Language models are quickly being integrated into every product, and it’s beneficial to understand and have experience doing that.

Many companies asked for experience with major LLM API providers (Google, OpenAI, Anthropic, etc.), but skills like prompt engineering were also in demand. This was in just under 50% of job listings, but I expect it to rise in the coming months.

8. Data Processing at Scale (Spark, Kafka, Airflow)

Similar to SQL, technologies enabling data pipeline handling at scale are central to building AI and machine learning systems. The real key to bringing value to ML (whether infra or at the product level) is understanding how to scale. These technologies help you do so.

I don’t work with these technologies currently, but my understanding is that Spark is the standard for batch jobs, Kafka for real-time streams, and Airflow for orchestrating workflows. Engineers will need to build and maintain systems processing massive datasets using these technologies. These three technologies were included in about 40% of listings combined.

9. Inference and Model Serving (Triton, TensorRT, ONNX)

Companies are very keen to find engineers that know how to serve large machine learning models efficiently. Companies are currently burning cash at the scale they’re serving models. Bringing serving costs down is a huge goal right now and probably will be for the next few years.

Tools like NVIDIA Triton, TensorRT, and ONNX are frequently listed in infra roles because they allow companies to deploy models with lower latency and lower cost. Basically, companies want engineers that know how to squeeze performance out of their compute. These were listed on about 30% of job listings.

10. JAX and TensorFlow

Both JAX and TensorFlow are ML frameworks like PyTorch. However, if I were to recommend learning one right now, it would be JAX. It seems companies are favoring it, especially for training on TPUs. TensorFlow is still relevant, but more and more companies seem to be transitioning away from it to JAX.

Even if you aren’t primarily training models, it’s worth understanding these frameworks because you’ll likely use them to a certain extent. They were included just under 30% of the time together on job listings.

Let me know if I missed anything. Thanks for reading!

Always be (machine) learning,

Logan

Share

For the past few months, I’ve been tracking the AI and software engineering job market to better understand what’s going on. I’ve even written about how the job market isn’t just bad, it’s strange.

One thing I found particularly interesting is how companies are titling job postings and just how inconsistent those titles are across the industry. This inconsistency makes it even more difficult for potential employers to find jobs they qualify for.

I’ve consolidated the research I’ve done into 5 AI-related roles engineers should know about if they want to work in AI. I’ve detailed what the job actually does and the qualifications for each for the following five roles:

• AI Research Scientist

• Research Engineer

• Machine Learning Engineer

• Software Engineer, AI

• AI Engineer

In the future, I’ll be getting more into the actual skills employers expect and how prevalent they are in job applications. Enjoy!

1. AI Research Scientist

This role is pretty cut-and-dry: AI research scientists conduct original research with the goal of advancing AI capabilities. The focus is on finding novel capabilities of AI.

The role entails a lot of experimentation and follows a research workflow (hypothesis, test, iterate, etc.). You’ll be expected to keep up with AI research and write papers to share what you’re working on.

In industry, there are very few research positions that require absolutely zero software engineering skills. Working in industry, you should know at least the basics of software engineering. You’ll work with engineers that will do most of the engineering but you need to be able to contribute where needed and communicate effectively with those engineers.

Qualifications:

• PhD (sometimes an MS) in a related field

• Deep research experience in a relevant AI field

• Papers at AI conferences

• Strong mathematical background

• An ability to independently develop new research hypotheses

2. Research Engineer

This is another established role where the responsibilities are well-defined, but it can get muddied due to the introduction of the next job in this list.

A research engineer’s focus is understanding how novel research can be applied to real-world applications. This means research engineers work very closely with research scientists to turn advancements in AI into something that can be used in the real-world.

This role includes things like implementing cutting-edge research papers from scratch, building prototypes of new AI capabilities to prove their usefulness, creating tools to support research activities, and iterating upon all of the above.

Qualifications:

• PhD or MS in a related field

• A strong understanding of ML and AI fundamentals

• Experience building prototypes and new products

• Experience with common ML frameworks and tooling

• Strong software engineering skills

3. Machine Learning Engineer

This is where the roles start getting a bit muddied but I think the industry has settled on what a machine learning engineer is.

A machine learning engineer is responsible for creating AI systems that work in production. This means all the software engineering that goes into create production software systems at scale plus being responsible for the entire model lifecycle.

At some companies, ML engineers will design, train, and evaluate models from scratch along with setting up the systems to make them work in production. In others, they’ll take the models from the research scientists and/or prototypes from research engineers and do the work needed to make those function at scale.

This role includes building and maintaining training pipelines, implementing model evals and testing, setting up model monitoring and performance tracking, handling model versioning and deployments, and everything else that goes into software systems (A/B testing, CI/CD, etc.).

As an aside, I would also include anyone working in MLOps or Infrastructure as a Machine Learning Engineer as their entire role is to understand the engineering that goes into ML. However, it seems many of those roles are included in the last two roles listed below.

Qualifications:

• BS or MS in a related field

• Deep understanding of ML algorithms and their applications

• An understanding of modeling techniques and the ML experimentation process

• Proficiency with MLOps tools

• Strong software engineering skills

4. Software Engineer, AI

This is by definition a bit more broad of a role title because both “software engineer” and “AI” have very broad definitions.

A software engineer working in AI builds AI systems. Their focus is on the application layer and they use (usually) pretrained models to solve a specific problem. Essentially, this is a lot of backend or fullstack software engineering with the complexities of AI layered on top.

This role can also be responsible for ML-related work such as data or training pipelines, but usually this role doesn’t train models and solely focuses on applications and the complexity of that application. This means understanding memory usage, cost estimates, resource optimization, data handling and integrity, evals and testing, safety, and more.

This is the title out of these five with the highest growth potential. The application layer is incredibly important and where real people find the most value from AI. Working with AI systems is more difficult than working with traditional software systems and requires great engineers to function effectively.

Qualifications:

• BS or MS in a related field (or equivalent experience)

• An understanding of AI/ML fundamentals and recent advancements

• Proficiency in cloud platforms and distributed systems

• Proficiency in common languages (C++/Python/Go are pretty common)

5. AI Engineer

This is by far the most vague job title in this space. I’ve seen it used as the same thing as an ML engineer, the same as a software engineer working in AI, and I’ve even seen it used to describe what I would consider a product engineer.

My take: The title ‘AI engineer’ is commonly used for roles that are less well-defined. I see this role listed frequently at startups where an employee needs to wear multiple hats and at larger companies with poor job descriptions. The one thing that’s pretty consistent is AI engineers work on the application of AI (jobs are often titles as ‘applied AI engineer’ because of this).

My suggestion for working as an AI engineer is to think about which of the above roles you’re most interested in and focus on the qualifications for that. Then, pay attention to the actual job description for AI engineering roles to find out more specifically what qualifications you need.

Qualifications:

• BS or MS in a related field (or equivalent experience)

• An understanding of ML frameworks and tooling

• Strong software engineering skills

• Whatever else is specific to the given job description

  ---

Hopefully this is helpful! As always, let me know if I missed something or you have an questions by leaving a comment.

Always be (machine) learning,

Logan

Share

An agentic engineer is an engineer who just does things. They’re able to identify high-impact work that needs to be done and take care of it without being told what to do or how to do it.

You might think that sounds like what every senior engineer should be able to, but there’s a difference. The key to being a senior engineer is the capability to take a system from requirements through the design and build stages on their own. An agentic engineer takes this a step forward.

An agentic engineer starts earlier in the pipeline. They identify the requirements on their own by understanding problems, recognizing needs, defining scope, and taking initiative entirely on their own. While senior engineers make progress, an agentic engineers drive outcomes.

Now you might be thinking: “Oh, I’m familiar with this. You’re talking about a 10x engineer.” I want to clarify that I’m not. In fact, articulating why I’m not talking about The 10x Engineer will put into perspective why an agentic engineer is becoming so important.

To me, a 10x engineer has always been a legend due to their in-depth understanding of technical concepts and their intuition that allows them to pick up new concepts quickly. Previously, this knowledge and intuition was the key to obtain 10x output and their ability to build systems faster than their colleagues. However, this is no longer the case.

As AI progresses, it breaks down the barrier that separated a 10x engineer from the rest of the pack. Technical concepts and fast coding are becoming more accessible to everyone. The defining factor in the age of AI is taking action. This is what defines an agentic engineer and what puts them ahead of their peers: Their ability to know what actions to take and the drive to execute them.

What you’ll see all over social media is everyone telling you to be an agentic engineer. What you don’t see anyone telling you is how. Why is this? Being an agentic engineer is difficult and is rooted outside of technical ability.

It’s something I’ve struggled with and I think about a lot, so in this article I’ll simplify the three things that set an agentic engineer apart from the rest and how you can become one.

If you missed last week’s article, I wrote about why right now is the best time to be a software engineer (contrary to everyone’s opinion). You can check it out here:

1. Understand Business Objectives

The first step that turns someone into an agentic engineer is understanding business objectives. Regardless of the size of company you work for, the problems that need to be solved align with business objectives. If you want to identify the most important problems and tackle them, you must take the time to understand the business objectives they aim to solve.

When I first started working as a software engineer, I naively thought my job was just to build software. I’ve always been confident in my knowledge of technical concepts but I quickly realized I had no experience in applying that knowledge.

This is something many engineers, researchers, and PMs in tech struggle with because it’s a learned skilled that isn’t taught in school. It also isn’t a skill that just clicks and you have it—it’s a skill highly dependent on the technology you’re working on, the company you’re working for, and the role you play in building the systems you work on. Understanding business objectives is an ongoing, frequently changing task.

The good news is that it gets easier with practice—and practicing it should always be readily accessible. Take note of the emphasis on ‘should’ here because it’s something we’ll get back to.

At most companies, leadership frequently goes through planning cycles with their team where they identify what they want to accomplish and the work needed to do so (objectives). They also identify concrete metrics that define whether or not those objectives were met (these are KPIs, which stands for ‘key performance indicators’).

The results of this planning cycle are shared throughout the organization with all roles so they can understand the most important problems to tackle and align on projects to do so.

The easiest way to know the most important problems you (as an individual) can solve to make an impact is to internalize the objectives and KPIs from these planning cycles. Coincidentally, this is also the best way to manage up and understand the work the leadership that controls your career progress wants completed. Every agentic engineer I know understands current business objectives and KPIs and solves problems accordingly.

Let me bring it back to the emphasis I put on should some paragraphs earlier. All the information I mentioned above should be made widely available to the entire organization. Some companies have well-defined processes for doing so.

One of the biggest and less-talked-about advantages of working for companies like Google, Meta, Microsoft, or other large tech companies early in a developer’s career is that these companies have these processes in place and this make it easy for junior developers to gain the skills they need to understand business objectives—a skill that pays dividends throughout their career.

Some companies don’t have these processes and it makes it much more difficult to practice this skill. If the company you work for doesn’t have these processes, you have two options:

1. Find a way to communicate up your management chain to understand business objectives. This will require frequent communication with your leadership and good communication from them.

2. Find a company that does. I know this isn’t an excellent solution, but it’s reality.

2. Be Curious

In my first software engineering job out of college, I lucked out with an incredible manager. He’s now a Partner at Microsoft1 and gave me some of the best early career advice I’ve ever received.

He simply told me to be more curious. He reminded me that one of the best things about software engineering is you can figure out how anything works as long as you have access to the source code (with the exception of machine learning systems, of course, but even then it’s helpful).

It had never crossed my mind that I could just look into anything I had access to just because I was curious about it and learn how it works simply because I can. It was such a simple concept that was absolutely mind-blowing for naive, new-grad Logan.

With so many companies using monorepos or large repositories to group microservices, this advice is even more beneficial. At Google we use one giant monorepo for all our code called google3. This means I have access to the source code for all Google products with the exception of intellectual property. I can literally dig into how any Google product works just because I want to.

Curiosity is important to be an agentic engineer because you can’t solve the problems you identify by understanding business objectives using the tips above if you don’t understanding the code currently solving those objectives. Or, more simply put, you can’t improve systems if you don’t understand how they already works.

So I’ll echo my first manager’s advice to anyone I meet: The best software engineers stay curious.

3. Just Start Building

A common phrase in the software community that I love is “you can just do things.” I’ll take this one step more and say “you can just build things.”

Too often we forget this. At large tech companies, there are processes put in place to align individuals on working within the same problem space. Usually these processes are meant to identify key stakeholders, refine plans, deduplicate work, and coordinate execution. Really that’s just a bunch of business speak for letting people know what’s going on and getting feedback.

One of these processes used at both Google and Microsoft (the two places I’ve worked) is document-driven design. This means writing a plan in a document, tagging reviews and approvers, and going through a feedback cycle and making changes as needed. It’s a great process for getting more eyes on a plan and informing others of the work you’re doing. It can also be great at delaying execution of the project by weeks or even months.

If every project requires a document and every document requires multiple reviewers, you can imagine that a considerable amount of engineering time is spent reviewing docs and giving feedback. Combining that with multiple iterations of document reviews and projects (especially those with more ambiguity) can end up in a sort of document purgatory where progress is stunted as stakeholders try to find alignment.

Sometimes this can go on for so long that you could have built the system multiple times in the amount of time the project takes to gain approval. This is especially humorous because part of the reason for document-driven design is to ensure time is well-spent and man hours aren’t wasted (i.e. the work is important enough to warrant time spent on it).

Having been stuck in this situation, I asked a more senior engineer how to avoid it and he told me to “just build the thing.” There are many times where he’s been getting approval for a project while actively prototyping it on the side and making adjustments in real-time based on feedback. In fact, every Googler has an “experimental” directory in google3 specifically for prototyping away from production code.

I bring this up because a major difference between an agentic and non-agentic engineer is agentic engineers aren’t stopped by processes. This doesn’t mean they don’t participate in them—they just don’t let the processes stop them from building.

It’s really easy to get caught up in these processes because they’re a huge part of an engineer’s job. It’s fundamental for company success to encourage employee agency by making it easy for them to build software whenever they want to. Any company that inhibits their engineers’ abilities to build software is not a company you want to work for.

These are just a few things I’ve learned over the course of my time as a software engineer. Let me know what’s helpful and what isn’t, and let me know what you think is fundamental for someone to be an ‘agentic engineer’ by leaving a comment.

I end my articles with a few resources to help you become a better engineer and some other interesting things you should know about. If you’re interested in getting these in your inbox each week don’t forget to subscribe.

Subscribe now

If you’ve found Society’s Backend beneficial, you can also become a paid subscriber for just $3/mo to support me and get even more articles and resources.

Enjoy the resources!

To help you become a better engineer

A Survey of AI Agent Protocols: A survey centered around how AI agents communicate with tools and data sources. A great read on a space I feel has been dominated by MCP without much thought for other options.

Getting things "done" in large tech companies: Another commentary on what it takes and (more importantly) what it means to get things done at large tech companies.

Artificial Intelligence Made Simple

The AI Infrastructure Phase Has Begun

Every month, the Chocolate Milk Cult reaches over a million Builders, Startup Founders, Investors, Policy Makers, Leaders, and more. If you’d like to meet other members of our community, please fill out this contact form here (I will never sell your data nor will I make intros w/o your explicit permission…

Read more

a year ago · 22 likes · 2 comments · Devansh

Quantization by Hugging Face: Quantization represents data with fewer bits, making it a useful technique for reducing memory-usage and accelerating inference especially when it comes to large language models (LLMs). This is a guide on quantizing a model to 4-bits and training it with LoRA.

Other interesting things

A post by Elon to essentially crowdsource difficult questions for Grok. As anyone working with machine learning knows, data cleanliness is incredibly important for proper model performance. I’m not sure how xAI intends to use this info, but I could see low-quality data be a major issue with this approach.

A different, more useful approach to using difficult questions with LLMs. This is something I’ve done before as well but I imagine Francois Chollet’s difficult questions are probably more difficult than mine.

Klarna has backed off from their “only hiring AI” approach very quickly. I said this was a bad idea when they announced this approach so I can’t say I was surprised to see they had reverted their decision so soon.

An interesting Note from right here on Substack. Learn about CPU architecture by playing a game! Who wouldn’t think this is awesome?

I’m glad to see more interesting technology and AI content on the Notes feed. It’s got huge potential but it’s too dominated by writing content and meaningless platitudes.

That’s all for this week! I’ll send a roundup at the end of the month with a more comprehensive overview of AI for the May. Make sure to subscribe so you don’t miss it!

Thanks for reading!

Always be (machine) learning,

Logan

This past week saw the release of many different coding models and tools. These releases brought up an interesting conversation that is more important than ever for software engineers to pay attention to.

The first party in this conversation I like to call Team “Software Engineers are Cooked”. They’ve been around and going strong for about three years now. Every time a new model comes out, demos pop up on social media showing what the model is capable of and how it can replace software engineers.

The second party was definitely the minority, but much louder than usual last week. This is Team “Don’t ruin your skills by coding with AI.” This team has popped up primarily in opposition to the above team. There is a strong voice not to let engineering skills atrophy by outsourcing engineering work to AI.

This situation is nuanced, of course. The counterargument to the first group is that only the successful demos tend to go viral. The failures of these models tend to be shared less. However, the failures are much more interesting when evaluating model viability for complex projects.

The counterargument to the second is that coding with AI has already proven to provide enormous productivity gains when done correctly. However, we’re now at the point where we can see the long-term negative impact from incorrect use. We’re also realizing that the magnitude of these negatives can be immense.

As someone who has coded with AI quite a bit at work (Google has been doing it since before it was cool) and on my personal projects, I want to provide a bit of important context about coding with AI. This article will go over the importance of never letting AI debug for you and mention three applications for which you should let AI code for you.

And, yes, this is another article similar to others you might find online about “How to code with AI.” The important distinction between this article and those is that those articles are about making your code better using AI. This article is about coding with AI in a way that will make you better.

Tedium: The secret sauce of software engineering

The first tasks to get automated with AI are anything tedious. Humans hate working on things that are time consuming, progress slowly, and can be monotonous. No one likes wasting time and no one wants to feel like they’re struggling on something that should be an easy task.

Software engineering is a field rife with tedium. Debugging, writing documentation, creating tests, code reviews, refactoring code, etc. are all tasks that can feel incredibly tedious, yet they’re essential to the job. It actually takes a great deal of effort as a software engineer not to get stuck in a rut when doing these tasks for hours on end.

Since software engineers are most familiar with new technology and software engineers deal with many tedious tasks, it makes sense for us to try and automate these tasks with AI. This is why we see so many posts online about software engineering being replaced by AI when in reality many more jobs would take the hit before we do. It’s because we’re so close to the technology and apply it to what we’re working on.

Replacing all tedious software engineering tasks with AI is a problem. Just because a task is tedious doesn’t mean it’s bad. In software engineering, the tedious tasks are often the ones we learn the most from. The tedium can be a struggle, but it’s the effort required on these tasks that helps us improve.

Just think about the hardest bug you’ve ever had to troubleshoot. In my experience, the difficult bugs require me to get very technical and really understanding what’s going on. I usually come out of these debugging sessions with a better understanding of the system as a whole and a greater knowledge of the specific topics that contributed to the error. I’ve found this knowledge to almost always be applicable to a later project.

If I had told AI, “Here’s my error message. Take a look at my code and fix it.” I never would have had that experience and I never would have learned. That’s a big issue in software engineering. Just like being a doctor or lawyer, continued education is paramount. The field moves quickly and if you aren’t constantly learning, you’ll fall behind.

I’m very specific about using debugging as an example. It seems to be the task engineers are most excited about automating. It’s also the task that’s most beneficial for an engineer to do themselves. You should never let AI debug for you. Automating this task has major long-term ramifications on you and your capabilities. This doesn’t mean you should never use AI to debug. Instead of letting it debug for you, you should let it debug with you (more on this below).

There are three applications where I’ve found it particularly helpful to have AI code for me:

1. Vibe coding

2. Working on particular tasks

3. Planning and understanding

Vibe coding

Vibe coding is a term coined by Andrej Karpathy on X that is essentially giving AI a list of system requirements and letting it take over from there. Engineers take a more hands-off approach to let the AI “do its thing”.

Vibe coding has really made waves for one specific reason: It allows people with next to zero programming knowledge to build their own applications. This is huge, there’s no doubt about it, but there’s also a very real value for those who do know how to code too.

First, vibe coding is great for prototyping. Now that AI can build things from the ground up, it can be a great tool for saying, “This is what I’m thinking of building, let me know what you come up with” and sitting back to see what it does. Waiting on a prompt generally takes a few minutes and can end with a working prototype. That speed is invaluable.

Second, vibe coding can grow a project beyond what you had anticipated. Giving the AI a bit more freedom allows it to write code uninhibited. Without strict requirements, it can be more creative. It’s great for coming up with alternate ways of satisfying the original requirements.

Vibe coding shines when the AI is given less requirements. Vibe coding fails when you try to adhere to a specific vision or requirements for a project. It can be very difficult to exactly match those specifications via prompting.

Stricter requirements are more common in industry and that’s why vibe coding works well on standalone projects but isn’t as successful when making changes to a complex and large codebase. It’s very difficult to non-specifically prompt a model to make a change while both ensuring the change is correct and it doesn’t have unintended side effects.

Working on particular tasks

While vibe coding doesn’t work for particular tasks, AI can still be used for those tasks. It just requires the engineer to have more knowledge of the code themselves. A prompt to do this would be something like: “I need X function to handle this case. I also need the tests adjusted to check for X.”

The more specific the prompt (and task) is, the better chance the AI will be able to succeed at the task without breaking functionality elsewhere in the code. Vibe coding doesn’t work in this case because it’s difficult to make a specific enough prompt without knowing the codebase.

You might ask: Why use AI for a change like this when I can do it myself?

First: speed. Just because the task is specific and the engineer knows what change needs to be made to the code doesn’t mean actually making the change itself is fast. This depends on many factors but codebase health is a big one. I’ve seen many simple changes require many lines of refactoring and additions in unhealthy codebases.

Second: AI is much better at avoiding “human errors”. Some examples of these are forgetting some references or not adjusting some tests when committing a change. These are things an engineer likely knows needs to be done but misses them for some reason or another.

A great example of a particular task that can be made much easier with AI is the task we discussed earlier: debugging. If you know what your error code is, where in the code you’re looking, and what you need that code to do, you can prompt a model to help you with it. Using prompting specific to the actual code (i.e. “this function needs…” and not just “do this”) will help you work through a problem while understanding what caused it.

Planning and understanding

AI is an excellent tool for learning while coding because you can learn in the context of the project you’re working on. Anyone can learn about system design, technical topics, etc. by asking an LLM or looking online, but learning these topics becomes more valuable (and easier) when you’re learning them in the context of the project you’re working on.

I use AI to help me plan and understand the systems I’m trying to build before I dig into them. I found it an excellent tool for planning projects, clarifying system goals, and understanding the optimal way for putting a system together. Again, you don’t want the AI to do these things for you. You want the AI to do them with you.

I’ve found this to work best when I reason with the model. I start my conversation with a long prompt with a lot of context and my reasoning behind the decisions I’ve already made and tradeoffs I’ve already considered. The prompt generally takes this form: “I want to make X change. I’m thinking about doing it X way because of X…” and runs for at least a paragraph.

I’ve found this prompt particularly helpful for a few reasons:

1. It provides complete context of what I’m trying to learn to the model. You can also provide this context with back and forth prompting, but I’ve found it less effective. Models tend to focus more on what has been said more recently and forget the context provided further back.

2. The model will explain correct and incorrect assumptions in detail. By prompting the model with the explanations it tends to reply in kind. This is exactly what I need when I’m trying to learn.

3. The model doesn’t give into my questions and suggestions nearly as easily. Most language models are trained to be agreeable. That’s particularly bad when coding with them because they tend to rate a user’s questions/suggestions highly when considering correctness. So when I ask, “Shouldn’t we do this?” the model will automatically agree that we should even if I’m wrong. The above prompt has helped mitigate this significantly.

4. The model avoids spinning its wheels. By adding explanations to the prompt, I’ve found models to spin their wheels less. I’ve gotten into situations where a model goes back and forth between two incorrect responses multiple times. Explanations and proper context seem to help with this.

It’s invaluable to have a learning tool directly integrated with your code and I greatly recommend using it, just make sure to take care to use it to learn and not to do things for you.

A few more tips

Here are just a few more tips for coding with AI that aren’t directly related to maximizing your long-term potential, but I still think are worth mentioning:

• Make small changes. Just like commits, have AI make small changes and frequently accept them.

• Make sure to review your AI-generated changes. This should be obvious, but it’s worth mentioning.

• Use Git especially if you’re new to programming and you’re vibe coding. Don’t end up like this.

• Ignore hype and test models yourself. The random people online aren’t going to know which model is best for your use case.

• Pay attention to model output, not just code changes. The model will output the code and also the “why” behind it. The “why” is more important than the change itself.

Important Takeaways

The first important takeaway from this article is tedium isn’t always bad. In software engineering it’s the tedious tasks that help us grow as engineers. This is especially true about debugging. However unpleasant it is, don’t let AI debug for you. Let AI do it with you.

A follow-on to this is to never replace your learning experiences with AI. Identify the things you’re doing that help you learn and make sure you continue those actions. Replace unhelpful tasks with AI instead. I see this pitfall a lot in writing. People try to replace their writing with AI when the act of writing is the learning experience.

Third, these lessons are most important to be intentional about now more than ever because of where AI is at. We’re seeing the long-term effects of coding with AI causing engineering skills to atrophy and the negative impact that has on a person (and their work) long-term.

AI has become performant enough at coding to do some important tasks entirely for us. However, AI isn’t good enough to solve everything for you. If you lose your skill set due to
AI, you’ll be in a very bad position when you reach a problem AI can’t solve for you.

Essentially, we’re seeing AI coding become good enough to be helpful, but still bad enough to be even more detrimental if used incorrectly. I urge you to be cognizant about how you’re using AI to code. If you’re using it correctly, you’ll surpass what you were previously capable of. If you use it incorrectly, you’ll fall far behind.

Let me know how you’re intentional about your AI use in general in the comments. I’m always up for learning new tips and tricks.

Subscribe now