AI Engineer vs ML Engineer vs Software Engineer: Who Do You Actually Need?

At the 47 New York AI companies we track daily, there are 141 open engineering roles titled "software engineer," 23 titled "AI engineer," and 15 titled "machine learning engineer." These are companies whose whole product is AI, and six in seven of their builder roles still carry the plain software title.

That ratio answers most of the question in the headline. Most AI products are software products with a model inside them. The work that makes them succeed or fail is mostly engineering around the model, not work on the model. The titles matter less than knowing which of three quite different jobs your project actually contains.

This guide covers what each role does, what New York AI companies are paying for each this week, and five questions that tell you which one to hire.

Key takeaways: an AI engineer builds products on existing models; an ML engineer builds and runs the models; a software engineer builds everything else, which is most of it. If you call a model through an API, you need the first, not the second. At NYC AI companies, posted base pay runs about $231,000 (software), $254,000 (AI) and $260,000 (ML) at the median, and the ML figure is propped up by seniority. Scope the work before you pick the title.

The short answer

| | AI engineer | ML engineer | Software engineer | |---|---|---|---| | Works on | Products built on existing models | The models themselves | The product and platform around everything | | Typical work | Retrieval (RAG), agents and tool use, prompts, evaluations, guardrails, cost and latency tuning | Training data, training and fine-tuning, feature pipelines, model serving, drift monitoring | APIs, data models, front ends, infrastructure, integrations, reliability | | Hire when | You are building on GPT, Claude, Gemini or an open model you do not train | You train or fine-tune your own models, or prediction quality on your data is the product | Always. It is most of the work, including in AI products | | Core question they answer | "Does the system give users correct, safe, affordable answers?" | "Is the model as accurate as it can be on our data, and does it stay that way?" | "Does the product work, scale and stay up?" | | NYC posted base, median | ~$254,000 | ~$260,000 | ~$231,000 |

Pay figures are midpoints of posted base-salary ranges on New York listings at the 47 companies in the NYC AI Hiring Index, read on September 26, 2026. More on what they do and do not show below.

What the New York job market says

We read every open role at these companies from their own job boards each morning. Here is how the engineering titles split this week.

| Title | Open roles | At how many companies | NYC postings with a pay range | Median base (midpoint) | |---|---|---|---|---| | Software engineer (no AI or ML in the title) | 141 | 36 | 86 | $231,000 | | AI engineer (AI, LLM, agents or generative AI in the title) | 23 | 14 | 12 | $254,000 | | Machine learning engineer | 15 | 8 | 9 | $260,000 | | Forward deployed engineer | 37 | — | 15 | $210,000 | | Research engineer or scientist | 19 | — | 9 | $260,000 |

Three things stand out.

The plain title dominates. Ramp, the biggest in-city employer we track, lists more than a dozen roles as "Software Engineer, [team]", covering credit, fraud, growth and data platform, and two as "Applied AI Engineer." Much of the AI work at these companies is done by people with the ordinary title. If you screen candidates by whether "AI" appears in their last job title, you will miss most of the people who have done the work.

ML engineering is small and senior. Fifteen open roles across eight companies, and 13 of the 15 are senior, staff, principal or lead. Companies hire ML engineers when they have models of their own to train and run, and when they do, they want people who have done it before. Attentive alone accounts for six of the fifteen.

The AI engineer premium is real but modest. About $23,000 over the software median, on a small sample. AI engineer roles are not more senior than software roles in this data (about half are senior or above, against about 60% for software engineers), so the gap is mostly the skill, not the level. The ML figure is harder to read: the three staff-level-and-above ML roles with a range post a median around $340,000, which pulls the whole median up.

Two cautions. Twelve and nine postings are small samples, so read those medians as a range of about ±$20,000, not a point. And these are posted base ranges at venture-backed AI companies in New York. They sit above national figures and above what an established company outside tech typically pays. For bands by level, see the NYC AI engineer salary guide, and for national context, the AI engineer and machine learning engineer salary pages.

What an AI engineer actually does

An AI engineer takes a model someone else trained and turns it into a product feature that works reliably. The role barely existed as a title before 2023; it grew out of software engineering once hosted foundation models became good enough to build on.

A typical week involves:

• Retrieval. Getting the right context in front of the model: chunking documents, building and tuning retrieval-augmented generation, deciding what the model is allowed to see.
• Tools and agents. Letting the model call functions, query systems and take actions, with limits on what it can do unsupervised. AI agents are named in postings at 37 of our 47 companies.
• Evaluations. Building the test suite that tells you whether a change made answers better or worse. This is the discipline that separates AI engineers from people who have written prompts. A system without evaluations cannot be improved deliberately.
• Guardrails and failure handling. What happens when the model is wrong, refuses, gets manipulated by text inside a document, or times out.
• Cost and latency. Every call has a price and a delay. Choosing models per task, caching and batching are part of the job.

What they usually do not do is train models. Some fine-tune, and the good ones know when fine-tuning is worth it (rarely, for a first version). But their raw material is a model behind an API.

The signal to look for: someone who can show you a system that reached real users, and explain how they knew it was working. "I built a chatbot" is not the signal. "I cut wrong answers from 14% to 4% on our evaluation set, and this is how we built the set" is.

What an ML engineer actually does

An ML engineer owns models: building them, training them, putting them into production and keeping them accurate as the world changes. It is the older discipline, and its core problems predate large language models by a decade.

A typical week involves:

• Training data. Collecting, labelling, cleaning and versioning it. Most model quality problems are data problems.
• Training and fine-tuning. Choosing architectures, running experiments, tuning, and knowing when a simpler model is good enough.
• Feature pipelines. Turning raw data into model inputs, identically in training and in production. Mismatches here cause some of the most expensive silent failures in the field.
• Serving. Running models on your own infrastructure at the speed and cost the product needs, often on GPUs.
• Monitoring and MLOps. Detecting drift, retraining, rolling back.

The classic ML engineering problems are prediction problems on your own data: fraud scoring, ranking, recommendations, forecasting, pricing. In those products the model is the product, and a two-point accuracy gain is worth real money. That is why companies like Attentive (marketing personalization) and EvolutionIQ (insurance claims) hire ML engineers, and why a company adding a support assistant usually should not.

The signal to look for: a model they trained that ran in production long enough to degrade, and what they did about it.

What a software engineer does in an AI product

Everything else, and in an AI product "everything else" is most of the code. Authentication, data models, the application's API, the interface, integrations with the customer's systems, billing, permissions, infrastructure, reliability.

It is easy to underestimate how much of an AI feature is ordinary engineering. The model call itself might be twenty lines. Around it sit the ingestion pipeline for the documents it retrieves from, the permission checks that stop it from showing one customer another customer's data, the queue that handles slow responses, the logging that makes failures debuggable and the interface that lets a user correct it. Those are software problems, and they decide whether the feature survives contact with real users.

This is also why the line between software engineer and AI engineer is blurring from the software side. A strong senior software engineer who has shipped one model-backed feature has learned most of what the AI engineer title implies. A self-described AI engineer who has only built demos has not learned the software half.

Which one do you need? Five questions

1. Are you training a model, or calling one? If you are calling a hosted or open model and building around it, you need AI engineering. If your plan depends on training or fine-tuning your own model on your own data, you need ML engineering. This one question settles most cases.

2. Is prediction quality on your own data the product? If a fraud score, a ranking or a forecast is what customers pay for, you need ML engineering, whether or not a language model is involved. If the model helps people do something else (answer questions, draft documents, process forms), accuracy still matters, but it is usually achieved through retrieval and evaluations rather than training.

3. How much of the project is integration? If most of the effort is connecting to your systems (a CRM, a document store, an ERP, a claims platform), the critical skills are software skills. Hire a software engineer with AI experience, or an AI engineer with strong software fundamentals, and weight the software half heavily in the interview.

4. Will you run models on your own infrastructure? Data residency, cost at high volume or latency requirements sometimes push companies to self-host models. That brings serving, GPU capacity and monitoring, which is ML engineering territory even if you never train anything.

5. Is this one project or a standing capability? A single well-scoped build, such as a document-processing pipeline or a support assistant, is often best done by one senior AI engineer, sometimes on contract. A standing capability, with several products on shared AI infrastructure, eventually needs both disciplines and a platform owner. Do not hire for the second when you have the first.

A rough rule for companies adding AI to an existing business: first hire, an AI engineer with real software depth; second hire, more software engineering; ML engineering when a model of your own becomes the plan, not before.

Four hiring mistakes we see

Hiring an ML engineer to build a RAG application. It is the most common mismatch. A researcher-leaning ML engineer will want to fine-tune; the project needed retrieval, evaluations and integration work. Both of you end up frustrated, and the project takes three months longer than it should.

Hiring on the title. "AI engineer" on a résumé proves nothing. The title is three years old, and anyone who has called an API can use it. Ask for the production system, the evaluation set and the failure they fixed. Our AI engineer interview questions are built around those scenarios, and how we vet AI engineers explains the eight disciplines we test.

Writing the job description before scoping the work. Companies often post "AI/ML Engineer" because they have not decided what the project is. Candidates read that as "they don't know yet", and the strongest ones pass. Decide first whether you are building on a model or building a model. Then the AI engineer job description guide helps you write it.

Skipping evaluations to hit a demo date. Whoever you hire, the first deliverable should include a way to measure whether the system is right. Without it you cannot tell a good engineer from a lucky demo, and the proof of concept proves nothing.

Where forward deployed engineers fit

One more title is worth knowing because it is the largest of the newer engineering titles in our data: the forward deployed engineer. With 37 open roles, it is about as common as the AI engineer and ML engineer titles put together at the companies we track. FDEs are software engineers who work inside a customer's environment, integrating the product and building what that customer needs on top of it.

It is not a role you hire to build your own product. But if you are buying an AI product, the vendor's FDE is often the person actually doing the AI engineering on your project, and it is worth asking to meet them before you sign.

Hiring any of the three

The practical sequence is the same whichever title you land on:

1. Scope the work first. One workflow, the data it touches, what "correct" means and who reviews the output. The AI project scoping pack walks through it.
2. Price it honestly. Use posted New York pay, not national averages, if you are hiring here. The NYC AI salary calculator works from the same posted ranges as this article.
3. Test the work, not the vocabulary. A two-hour scenario that mirrors your project tells you more than five rounds of trivia. See how to hire AI engineers.
4. Consider a senior contractor for the first build. If you are not sure yet whether you need a standing team, one experienced engineer for a scoped project will tell you. We match companies with vetted AI engineers for exactly this, and you can start with a project brief.

The title on the offer letter is the least important part of the decision. The questions that matter are whether you are building on a model or building a model, and whether the person you hire has shipped that kind of work to real users.