Neuronify

Evals (LLM evaluation)

Repeatable tests that score a model or agent’s outputs against examples with known good answers, so a change can be judged by measurement instead of by feel.

Hiring data as of November 4, 2026 · refreshed daily

companies naming it
29 of 47companies naming it
open roles naming it
45open roles naming it
of those based in NYC
30of those based in NYC
median posted base (n=22)
$203kmedian posted base (n=22)

What Evals means

An eval is a test suite for behavior that is not deterministic. You assemble representative inputs, define what a good output looks like — an exact answer, a rubric, a comparison judged by another model or by a person — and score the system against it every time the prompt, the model, or the retrieval changes.

Evals are what let a team upgrade a model or rewrite a prompt without guessing whether things got better. Teams that lack them tend to ship on the strength of a few hand-checked examples and find the regressions in production. Building the dataset is usually the expensive part, because it requires deciding, case by case, what correct means for the product.

Evals in New York job posts

29 of the 47 New York AI companies we track name Evals in at least one open role today, across 45 postings. 30 of those are based in New York; the rest are remote or in the companies’ other offices.

Engineering roles make up 56% of the postings that name it, with the rest spread across other functions.

How to read it

One of the clearest signals of a team that runs AI in production rather than in demos. It appears in engineering, research, and product postings, and its most distinctive companions in the same postings are tool use and RAG. If a posting lists evals, bring an example of a metric you defined and a decision it changed.

Go deeper: how to design a task-level comparison before choosing a model, in the Neuronify AI Field Guide.

What these roles pay

$183k25th percentile
$203kMedian
$238k75th percentile

Midpoints of 22 distinct salary ranges posted on New York roles that name Evals. Base salary only; on-target earnings are excluded, and a range repeated across one company’s postings counts once. For comparison, the median across all New York engineering postings is $216k. A term’s pay reflects the roles that name it as much as the skill itself — see the role mix below before reading a premium into it. Full bands by function and seniority are in the NYC salary calculator.

Companies naming Evals, daily

Count of companies with at least one open role naming the term. We read between 46 and 47 companies a day over this period, so a move of one or two is noise.

017.53529Sep 20Nov 4Sep 20: 25 companiesSep 21: 25 companiesSep 22: 25 companiesSep 23: 25 companiesSep 24: 25 companiesSep 25: 25 companiesSep 26: 25 companiesSep 27: 25 companiesSep 28: 25 companiesSep 29: 26 companiesSep 30: 26 companiesOct 1: 26 companiesOct 2: 27 companiesOct 3: 27 companiesOct 4: 27 companiesOct 5: 27 companiesOct 6: 27 companiesOct 7: 27 companiesOct 8: 27 companiesOct 9: 27 companiesOct 10: 27 companiesOct 11: 27 companiesOct 12: 27 companiesOct 13: 28 companiesOct 14: 28 companiesOct 15: 28 companiesOct 16: 28 companiesOct 17: 29 companiesOct 18: 29 companiesOct 19: 29 companiesOct 20: 29 companiesOct 21: 29 companiesOct 22: 29 companiesOct 23: 28 companiesOct 24: 28 companiesOct 25: 28 companiesOct 26: 28 companiesOct 27: 28 companiesOct 28: 28 companiesOct 29: 28 companiesOct 30: 28 companiesOct 31: 29 companiesNov 1: 29 companiesNov 2: 28 companiesNov 3: 28 companiesNov 4: 29 companies
View as table
Datecompanies
2026-09-2025
2026-09-2125
2026-09-2225
2026-09-2325
2026-09-2425
2026-09-2525
2026-09-2625
2026-09-2725
2026-09-2825
2026-09-2926
2026-09-3026
2026-10-0126
2026-10-0227
2026-10-0327
2026-10-0427
2026-10-0527
2026-10-0627
2026-10-0727
2026-10-0827
2026-10-0927
2026-10-1027
2026-10-1127
2026-10-1227
2026-10-1328
2026-10-1428
2026-10-1528
2026-10-1628
2026-10-1729
2026-10-1829
2026-10-1929
2026-10-2029
2026-10-2129
2026-10-2229
2026-10-2328
2026-10-2428
2026-10-2528
2026-10-2628
2026-10-2728
2026-10-2828
2026-10-2928
2026-10-3028
2026-10-3129
2026-11-0129
2026-11-0228
2026-11-0328
2026-11-0429

Who names it

Open roles naming Evals, by company.

  • ElevenLabs3
  • EvolutionIQ3
  • Normal Computing3
  • Ramp3
  • Rogo3
  • Anterior2
  • Hebbia2
  • Invisible Technologies2
  • Mirage (fka Captions)2
  • Patlytics2
  • Runway2
  • Alloy1

Also: AlphaSense, Arthur, ASAPP, Clay, Comet, EliseAI, Grow Therapy, Headway, K Health, Maven Clinic, Reality Defender, Rillet, Sixfold, Slingshot AI, SmarterDx, Tabs, Tennr.

Which roles

The same postings, by function and by seniority.

  • Engineering25
  • Research7
  • Product5
  • Other3
  • Operations2
  • GTM2
  • Mid23
  • Senior10
  • Staff/Principal7
  • Entry3
  • Manager1
  • Leadership1

Named alongside Evals

Terms that appear in the same postings far more often than chance would put them there. The number is how many of the 29 companies pair the two.

Open roles naming Evals

A sample from today’s data, New York roles first and one per company before any repeats. Links go to the employer’s own posting. For the full market, see AI jobs in NYC.

How this is counted

Every day we read the open roles on the public job boards of the New York AI companies in our coverage — the NYC AI 100 and the AI in NYC Show roster. This term is matched against each role’s description after removing the text a company repeats across its postings, so an “About us” paragraph cannot tag every role the company has open. A vendor’s own postings never count toward its own name.

The figure means named in a job posting. It does not mean used in production, and a “nice to have” counts the same as a requirement. Companies are the headline number because posting counts are dominated by whichever few employers are hiring hardest this month.

Full method, including the pay rules, is on the glossary index; the underlying series is the NYC AI Hiring Index.

Evals — common questions

What does Evals mean in an AI job posting?

Repeatable tests that score a model or agent’s outputs against examples with known good answers, so a change can be judged by measurement instead of by feel. One of the clearest signals of a team that runs AI in production rather than in demos. It appears in engineering, research, and product postings, and its most distinctive companions in the same postings are tool use and RAG. If a posting lists evals, bring an example of a metric you defined and a decision it changed.

How many New York AI companies are hiring for Evals?

As of November 4, 2026, 29 of the 47 New York AI companies we track name Evals in the description of at least one open role, across 45 postings (30 based in New York). The count refreshes daily from the companies’ own job boards.

What do roles that ask for Evals pay in New York?

Across 22 distinct posted salary ranges on New York roles naming Evals, the median midpoint is $203k base, with the middle half between $183k and $238k. The same figure for all New York engineering postings is $216k. These are employer-posted ranges required by New York’s pay transparency law, base salary only.

Which skills are asked for alongside Evals?

In the same postings, the terms most distinctively paired with Evals are Tool use, Prompt engineering, Guardrails, RAG, Machine learning. We rank pairings by how much more often they appear together than apart, so near-universal terms such as Python do not crowd out the informative ones.

Need someone who has actually shipped this?

A keyword in a posting is easy to match and hard to verify. We match companies with AI engineers and consultancies vetted on production work, and tell you when the project needs a different skill than the one you named.

Start a project briefJoin the expert network