If vLLM already solved LLM serving, why did SGLang appear?
Introduction
Voice AI looks deceptively simple from the outside. Talk to a machine, get a response, move on. Over the past year, demos have multiplied, launch videos have flooded timelines, and “human-like conversations” have become table stakes in product pitches.
But once you move past the demo and into production, reality sets in quickly.
Latency spikes. Conversations feel unnatural. Costs spiral. Security concerns block deployment. And systems that worked flawlessly with ten test calls begin to fall apart under real traffic. Voice AI, more than most AI applications, exposes the gap between what looks impressive and what actually works at scale.
Most teams learn this the hard way when they move from a demo to Enterprise AI expectations.
This tension surfaced clearly in Neysa’s conversation with Akshat Mandloi, CTO and co-founder of Smallest.ai. What makes the discussion valuable is the their approach to voice AI and how their journey reveals a broader truth: weaker models don’t fail AI products, they fail because the system around the model isn’t built for reality.
This is kind of gap Neysa was built to address.
The Demo Trap: Why Voice AI Breaks Post POC
Most voice AI products don’t fail at the idea stage. They fail post the proof of concept.
Akshat describes a familiar starting point: a customer wants to automate a simple voice workflow – basic intent recognition, a few scripted responses, a clean handoff. On paper, this should be easy. The ecosystem is full of APIs, SDKs, and platforms promising plug-and-play voice intelligence. In practice, nothing worked.
The core issue was latency and system design.
Most voice platforms rely on cascaded pipelines: speech-to-text, then an intelligence layer, then text-to-speech. Each step adds a delay. Each dependency introduces fragility. The result is a conversation that technically functions but never feels real-time or natural.
At scale, teams usually standardize on AI Inference as a Service to keep deployment and scaling predictable.
This is the first production lesson teams learn is that humans are extremely sensitive to delay in voice interactions. A few hundred milliseconds can make the difference between “this feels human” and “this feels broken.”
Fixing this problem requires more than swapping APIs. It requires rethinking how models are trained, served, and orchestrated – which immediately pushes teams into deeper infrastructure decisions.
That sensitivity is why production teams obsess over high throughput inference and time-to-first-token behavior.
Owning the Stack – Not a Philosophy
One of the strongest signals from the conversation is that you cannot solve production voice AI without owning your core models.
Fine-tuning someone else’s model or wrapping an API only gets you so far. When something breaks – pronunciation errors, contextual failures, edge-case latency spikes, and you’re stuck. You can’t retrain. You can’t instrument deeply. You can’t close the loop fast enough.
Smallest.ai chose the harder path – build foundational models in-house, train them with their own data, and control the full inference pipeline.
That decision unlocked three things that matter deeply in production:
- Latency control – models could be optimized specifically for real-time speech.
- Naturalness – training and iteration focused on conversational flow, not benchmark scores.
- Security and deployment flexibility – including on-premise deployments for regulated industries.
But owning the stack creates a new problem – infrastructure becomes existential. When your models go down, your entire business goes down.
Production AI is a Systems Problem
One of the most important moments in the discussion comes when Akshat talks about the gap between POCs and production systems.
Building a demo today is trivial. Tools like Cursor, Lovable, and rapid prototyping frameworks make it possible to spin up impressive workflows in hours.
But production systems demand different guarantees:
- 99.99% uptime.
- Consistent latency at scale.
- Zero tolerance for hallucinations in regulated environments.
- Tight observability across every call, token, and failure mode.
These requirements shift the challenge from model quality to system reliability.
Voice AI magnifies this shift because it is both real-time and user-facing.
Failures aren’t silent. They happen in real time, in front of customers, often in sensitive moments like banking, healthcare, or collections.
This is where infrastructure stops being “backend plumbing” and becomes a competitive advantage.
Why Infrastructure Decisions Shape Product Velocity
As Smallest.ai scaled, infrastructure decisions began to directly affect how fast they could ship.
Iteration speed isn’t just about engineering talent. It’s about whether your systems allow you to:
- Spin up new inference configurations quickly.
- Experiment with model variants without long provisioning delays.
- Observe failures at a granular level.
- Fix and redeploy models within hours, not weeks.
Akshat describes a feedback loop where a failed call can be diagnosed and corrected within a day. That level of responsiveness is only possible when compute, networking, storage, and orchestration are designed to work together.
This is where Neysa enters the picture – not as a cloud provider in the traditional sense, but as infrastructure designed specifically for AI systems that must evolve continuously.
Why Neysa Velocis Fits Production AI Workloads
When Smallest.ai evaluated infrastructure options, hyperscalers were an obvious starting point. They always are.
But as workloads grow more specialised, teams run into familiar constraints:
- Limited configurability for inference workloads
- Difficulty optimizing cost at scale
- Rigid abstractions that slow down iteration
- Black-box performance trade-offs
For companies running foundational models, these constraints become blockers.
Neysa Velocis was designed around a different assumption: AI systems are living systems. They need room to evolve, reconfigure, and optimize continuously.
For Smallest.ai, this translated into three concrete advantages:
- Reliability – infrastructure that doesn’t become a single point of failure.
- Configurability – the ability to tailor inference setups to specific latency and cost profiles.
- Scalability without surprises – predictable behavior as usage grows.
When your entire platform depends on real-time inference, infrastructure stability is not a nice-to-have. It’s existential.
Run Smarter AI in Production
Deploy efficient AI systems optimized for speed, scale, and inference economics.
Cost, Control, and the Hidden Risk of Lock-In
Another subtle but critical theme in the conversation is cost control.
Voice AI systems are inference-heavy. As call volumes scale, costs can balloon rapidly if infrastructure isn’t optimized. Teams that lock themselves into rigid platforms often discover too late that they’ve sacrificed efficiency for convenience.
By choosing an infrastructure partner that supports configurability and observability, Smallest.ai retained the ability to:
- Optimize GPU usage
- Adjust serving strategies as models evolve
- Balance performance against cost without compromising reliability
This flexibility is especially important for startups competing in cost-sensitive markets where AI must outperform human labor without exceeding it in cost.
Infrastructure decisions made early can either preserve this optionality or eliminate it entirely.
Enterprise Adoption: Security, Observability, and Trust
For enterprises, especially in banking and healthcare, AI adoption hinges on trust. Three factors consistently drive “yes” decisions:
- Security and compliance – data protection is non-negotiable. That’s the same principle behind Secure Cloud for AI Teams, where controls need to follow models into production.
- Observability – enterprises need to see how systems behave, not just trust outputs.
- Deployment flexibility – including on-premise or hybrid models.
Because Smallest.ai owns its models and runs on infrastructure that supports these requirements, they can meet enterprises where they are – not force them into a single deployment pattern.
This is where Neysa’s AI-native infrastructure becomes a strategic enabler. Sovereign deployments, transparent observability, and predictable performance allow enterprises to move forward without compromising governance.
The Real Lesson: AI Is a Journey, Not a Switch
Perhaps the most important takeaway from the conversation is philosophical.
AI is not a magic wand. Generative systems don’t work perfectly on day one. They improve through feedback loops, iteration, and continuous learning. Teams that expect instant perfection often abandon AI just as it starts becoming useful.
The winners build systems that expect change.
They design infrastructure, models, and workflows that evolve – improving call by call, month by month. This mindset aligns closely with how Neysa approaches AI infrastructure: not as static capacity, but as a foundation for continuous iteration.
Looking Ahead: Scaling Voice AI Without Losing Control
Smallest.ai’s ambition is clear: own the enterprise voice AI stack end-to-end. That vision only works if infrastructure scales without becoming a bottleneck.
As workloads grow, the risk shifts from “can we build this?” to “can we sustain this?” This is where partnerships matter.
Infrastructure that scales predictably, supports rapid iteration, and preserves control over cost and performance becomes the difference between growth and fragility.
For Neysa, this conversation with Akshat Mandloi – reinforces a broader pattern seen across AI-native companies: the most important AI decisions are no longer about models – they’re about systems.
Conclusion
Voice AI exposes the hard truth of applied AI faster than most domains. It leaves no room for abstraction failures, hidden latency, or brittle systems.
What this conversation makes clear is that production-grade AI is built at the intersection of models, systems, and infrastructure. Neglect any one of these, and the entire stack collapses.
Neysa exists to make that intersection work.
By enabling teams like Smallest.ai to build, iterate, and scale without infrastructure becoming the limiting factor, Neysa turns AI from a promising demo into a reliable, evolving system.
Because in the end, the future of AI won’t be defined by who ships the flashiest demo – but by who builds systems that survive reality.
Loved the blog? Tune into Akshat’s Conversation on AI’s Happy Hour Podcast – by Neysa:
