Jev by TypeSafe AI: Hype, Tool or Threat?

 |  Writings

Series: Backends in Node.js and TypeScript

A few days ago, TypeSafe AI introduced Jev, a new model designed to make fast, structured decisions that software can use directly.

I read the announcement and had two questions: Is Jev more of a hype? Is it really useful for me as a backend engineer? And does it present a new threat of replacing me as a software engineer? Jev also reminded me of the OpenAI REST API "structured outputs" endpoints. I wondered whether Jev would offer any advantages over OpenAI.

Here is what I found.

What Does Jev Actually Do

Jev (by TypeSafe AI) belongs to a class of narrowly specialized models that provide structured outputs rather than just prose. It accepts application state as input and returns structured decisions. The possible outputs and their types are defined in advance.

For example, an application could provide information about a customer and ask Jev to classify the customer's churn risk, determine which support department should handle a request, or score a potential business opportunity. The application receives predefined values that it can use in its business logic.

Very similar to OpenAI "structured outputs", I think.

TypeSafe AI claims that Jev provides several advantages:

  • Response times of 70–500 ms.
  • Input costs of $0.042 per million tokens.
  • Guaranteed adherence to predefined output types.
  • Calibrated probabilities and confidence scores accompanying its decisions.

The company also claims substantial improvements in speed and cost compared with existing LLMs performing similar tasks.

These are the company's published results. Jev is currently in early access, and I haven't tested it myself.

Jev vs OpenAI Structured Outputs

I have already worked with OpenAI's API to implement structured classification and decision-making workflows. OpenAI introduced Structured Outputs in August 2024. With strict: true, developers can define a JSON Schema and receive responses conforming to that schema.

We can already classify incoming data, select values from predefined enums, extract structured information, and use the results in application code. ([OpenAI][2]). Jev provides essentially the same capabilities for these use cases.

Jev's underlying generation mechanism (it generates its decision outputs in parallel and provides associated probabilities - confidence scores) is interesting, but there is little reason to judge Jev's advantage based on its implementation details.

The practical question for a backend engineer is whether Jev produces decisions of comparable accuracy, faster and at a lower cost, for the same workload. They claim it has an edge in this. But with OpenAI Structured Outputs we can select the seriously cheaper model still very capable of producing structured outputs with a large focused (cached) context.

Jev: Useful in Real Backend Systems?

At one innovation analytics startup, I worked on backend architecture and LLM-powered ingestion and scoring pipelines. The system needed to process incoming information, classify it using controlled vocabularies, and produce structured results that could be used by subsequent application components.

I implemented deterministic validation gates to check model outputs before allowing the code to proceed through the workflow. Looking at Jev's capabilities, I can see where it could fit into this architecture. For example, I could use it to classify incoming records or calculate particular scores.

I had great results with GPT-4o Mini (quite cheaper than GPT Luna that TypeSafe AI compares Jev with) with latencies well within 100ms and costs like $0.15 per 1,000 requests of ~2K prompt and ~700 bytes JSON response.

Separately, Jev's confidence scores may seem to be useful for workflows where an application needs to decide whether to accept a model's decision or send the case for further processing. But these scores can be equally misleading requiring additional deterministic validation gates, thus adding the complexity to a workflow.

My conclusion is Jev deserves some A/B testing under realistic production workloads vs OpenAI structured outputs before any decision. With millions of requests it may offer an advantage. As well its confidence scores may be used in some cases.

A Tool Or an Engineer's Replacement.

This was my original concern. A usual senior backend engineer's work involves designing application architecture, domain models and service boundaries, implementing APIs, working with databases, and building distributed systems, and often, implementing business workflows that involve classification, scoring, and other decisions based on incoming data.

Jev could automate some decision-making operations. As well as OpenAI structured outputs does. But hardly presents any threat to the entire engineering process. So the answer is "a tool", that has had alternatives for already ~2 years.

My Conclusion

After examining Jev and comparing it with OpenAI's existing Structured Outputs, I see it as a potentially useful addition to my backend engineering toolkit with advantages that have yet to be proven for production systems already using OpenAI structured outputs.

For now, I am interested in seeing how Jev performs on real business workloads and whether its claimed performance improvements justify replacing existing LLM calls in production systems.