The Top 10 AI Skills Software Engineers Should Focus On (In Order)
A straightforward list from my market research
This is the last article where I’ll be sharing my recent job market research. In case you missed it, I’ve also written about the five AI-related jobs every software engineer should know about and the strange reality of the current job market (why it isn’t just a bad market, but also a weird one).
You can find both of these below:
This list contains the top 10 skills every software engineering working in (or wanting to work in) AI should be focusing on in order of how frequently they occurred on AI-related engineering job descriptions.
My methodology for gathering my data points isn’t 100% foolproof, but I’ve been tracking job listings for close to 6 months now while focusing on companies with the following 3 qualities:
A high hiring bar. I know these companies put time and effort into their hiring process to get the best engineers. They’re more likely to provide more accurate and higher quality job listings.
Are doing interesting work in AI and ML. Most were companies at the forefront of AI including startups and large tech companies. These are companies that are innovating in the space and likely to use the most up-to-date technologies.
Companies that pay well. A huge reason this is interesting is so you can learn the skills that will let you increase your compensation.
Let’s jump right in!
The most-mentioned skills in job descriptions
1. Python
This one is kind of obvious, but learning Python as a software engineer has a huge return on investment. The entire ML ecosystem is built on top of Python code, and the large majority of machine learning libraries are built to work with it.
About 90% of job descriptions mentioned Python. In my opinion, this is a language all software engineers should learn anyway for interviews, as it removes a lot of the language-related hardship so you can focus on the data structures and algorithms.
2. PyTorch
PyTorch is the dominant deep learning framework across the industry. Companies hiring for AI engineering roles expect engineers to be fluent in PyTorch because it underlies most modern systems. PyTorch was included as a requirement for about 70% of roles.
It’s the easiest deep learning framework to get started with and, in my opinion, the most readable. I’d say it’s the most beginner-friendly, even though the framework you use in industry will depend on the company you work for. The other popular frameworks are also included in this list, just further down.
3. Kubernetes
Kubernetes is the infra layer that allows ML systems to run at scale. It shows up in nearly every listing that touches ML infrastructure because it allows containerized workloads to be deployed, managed, and scaled across clusters. Kubernetes was also mentioned in about 70% of roles.
Kubernetes is the standard way to train and serve models on multi-GPU clusters. Training and serving in production rarely occurs on one machine. Kubernetes handles scheduling, scaling, and fault tolerance for large workloads.
4. Cloud Platforms (AWS, GCP, Azure)
Most companies are running their training and serving on one of the three major cloud platforms. These showed up on job descriptions about 70% of the time combined. To me, having experience working with one shows that you can learn how to work with the others. Most companies seemed to share this opinion, though some asked specifically for experience in one or the other.
AWS was mentioned the most, then GCP, and then Azure. I think Azure actually has a more dominant market share, but the switch to TPUs for many companies has them using Google’s infrastructure. Understanding cloud platforms, the tools they offer, and how to use those tools is beneficial for an AI-related job search.
5. SQL
Any time you’re working in AI, you’ll be working with data. Anytime you work with data, understanding SQL is a must. There hasn’t been a single role that I’ve worked that hasn’t required SQL. It appeared in about half of job descriptions.
Being able to inspect data is a must for any engineer wanting to work in machine learning.
6. C++
C++ shows up consistently in job listings for performance engineering, inference optimization, and GPU-related work. While Python dominates at the surface, the underlying libraries and kernels are written in C++.
For an infrastructure-related role or any role focusing on ML efficiency, C++ is the go-to language. It appeared in about half of the job descriptions.
7. LLM APIs and Prompt Engineering
Companies want engineers that already have experience working with LLMs. Building systems with AI is fundamentally different from building traditional software systems because of their non-deterministic nature. Language models are quickly being integrated into every product, and it’s beneficial to understand and have experience doing that.
Many companies asked for experience with major LLM API providers (Google, OpenAI, Anthropic, etc.), but skills like prompt engineering were also in demand. This was in just under 50% of job listings, but I expect it to rise in the coming months.
8. Data Processing at Scale (Spark, Kafka, Airflow)
Similar to SQL, technologies enabling data pipeline handling at scale are central to building AI and machine learning systems. The real key to bringing value to ML (whether infra or at the product level) is understanding how to scale. These technologies help you do so.
I don’t work with these technologies currently, but my understanding is that Spark is the standard for batch jobs, Kafka for real-time streams, and Airflow for orchestrating workflows. Engineers will need to build and maintain systems processing massive datasets using these technologies. These three technologies were included in about 40% of listings combined.
9. Inference and Model Serving (Triton, TensorRT, ONNX)
Companies are very keen to find engineers that know how to serve large machine learning models efficiently. Companies are currently burning cash at the scale they’re serving models. Bringing serving costs down is a huge goal right now and probably will be for the next few years.
Tools like NVIDIA Triton, TensorRT, and ONNX are frequently listed in infra roles because they allow companies to deploy models with lower latency and lower cost. Basically, companies want engineers that know how to squeeze performance out of their compute. These were listed on about 30% of job listings.
10. JAX and TensorFlow
Both JAX and TensorFlow are ML frameworks like PyTorch. However, if I were to recommend learning one right now, it would be JAX. It seems companies are favoring it, especially for training on TPUs. TensorFlow is still relevant, but more and more companies seem to be transitioning away from it to JAX.
Even if you aren’t primarily training models, it’s worth understanding these frameworks because you’ll likely use them to a certain extent. They were included just under 30% of the time together on job listings.
Let me know if I missed anything. Thanks for reading!
Always be (machine) learning,
Logan
That definitely pulled me in—I was like, "JAX?!" It seems to have random surges of interest that then fade just as quickly.