DApp Store | Web3 Hub for Events & Games

Trending topics

Louround 🥂

Co-founder of @a1research__ 🀄️ & @steak_studio 🥩

Louround 🥂 reposted

VLAs are still very new and a lot of people find it difficult to understand the difference between VLAs and LLMs. Here is a deep dive into how these AI systems differ in reasoning, sensing, and action. Part 1. Let's breakdown the key distinctions and how AI agents wrapped around an LLM differ from operator agents that use VLA models: 1. Sense: How they perceive the world Agent (LLM): Processes text or structured data e.g JSON, APIs, and sometimes images. It’s like a brain working with clean, abstracted inputs. Think reading a manual or parsing a spreadsheet. Great for structured environments but limited by what’s fed to it. Operator (VLA): Sees raw, real-time pixels from cameras, plus sensor data (e.g., touch, position) and proprioception (self-awareness of movement). It’s like navigating the world with eyes and senses, thriving in dynamic, messy settings like UIs or physical spaces. 2. Act: How they interact Agent: Acts by calling functions, tools, or APIs. Imagine it as a manager sending precise instructions like “book a flight via Expedia API.” It’s deliberate but relies on pre-built tools and clear interfaces. Operator: Executes continuous, low-level actions, like moving a mouse cursor, typing, or controlling robot joints. It’s like a skilled worker directly manipulating the environment, ideal for tasks requiring real-time precision. 3. Control: How they make decisions Agent: Follows a slow, reflective loop: plan, call a tool, evaluate the result, repeat. It’s token-bound (limited by text processing) and network-bound (waiting for API responses). This makes it methodical but sluggish for real-time tasks. Operator: Operates, making stepwise decisions in a tight feedback loop. Think of it like a gamer reacting instantly to what’s on screen. This speed enables fluid interaction but demands robust real-time processing. 4. Data to Learn: What fuels their training Agent: Trained on vast text corpora, instructions, documentation, or RAG (Retrieval-Augmented Generation) datasets. It learns from books, code, or FAQs, excelling at reasoning over structured knowledge. Operator: Learns from demonstrations (e.g., videos of humans performing tasks), teleoperation logs, or reward signals. It’s like learning by watching and practicing, perfect for tasks where explicit instructions are scarce. 5. Failure Modes: Where they break Agent: Prone to hallucination (making up answers) or brittle long-horizon plans that fall apart if one step fails. It’s like a strategist who overthinks or misreads the situation. Operator: Faces covariate shift (when training data doesn’t match real-world conditions) or compounding errors in control (small mistakes snowball). It’s like a driver losing control on an unfamiliar road. 6. Infra: The tech behind them Agent: Relies on a prompt/router to decide which tools to call, a tool registry for available functions, and memory/RAG for context. It’s a modular setup, like a command center orchestrating tasks. Operator: Needs video ingestion pipelines, an action server for real-time control, a safety shield to prevent harmful actions, and a replay buffer to store experiences. It’s a high-performance system built for dynamic environments. 7. Where Each Shines: Their sweet spots Agent: Dominates in workflows with clean APIs (e.g., automating business processes), reasoning over documents (e.g., summarizing reports), or code generation. It’s your go-to for structured, high-level tasks. Operator: Excels in messy, API-less environments like navigating clunky UIs, controlling robots, or tackling game-like tasks. If it involves real-time interaction with unpredictable systems, VLA is king. 8. Mental Model: Planner + Doer Think of the LLM Agent as the planner: it breaks complex tasks into clear, logical goals. The VLA Operator is the doer, executing those goals by directly interacting with pixels or physical systems. A checker (another system or agent) monitors outcomes to ensure success. $CODEC

Dips in a bull market are meant to be bought, especially on projects with big catalysts We all know that AI is the narrative of this cycle, started by ai16z and Virtuals last year. My bet is that the market will focus on more complex and sophisticated technologies such as VLAs, and let me tell you why. LLMs (Large Language Models) mainly read and write text: they’re great at explaining, planning, and generating instructions, but they don’t by themselves control motors or interact with the physical world (as you may have experienced with chatgpt). VLAs (Vision Language Action models) differ from LLMs as they are multimodal systems that look at things (vision), understand instructions (language), and directly produce actions. It's like telling a robot to pick up a red cup and then moving its arm to do it. VLAs are trained on examples that pair images/video + instructions + real action traces (how a robot actually moved), and they must run fast and safely in real time. LLMs on their side are trained on huge text collections and focus on reasoning and language tasks. TL;DR LLMs think and speak whil VLAs see, reason, and act. As you can see, VLAs are a major addition to LLMs and will notably enable the next 0 to 1 innovation in the overall economy that will be robotics. A majority of investment funds are allocating a large part of their investments into this sector, seen as the next logical evolution in the AI industry. I already made a post a while ago on the current leader in the crypto market, @codecopenflow, which did not raise capital (fair launch) yet is shipping cutting-edge products and currently sitting at $23M FDV. For information, other crypto competitors raised $20m ( @openmind_agi) at what is probably a $200M to $300M ++ FDV while no product or community has been built and shipped yet. What makes Codec a leading project in the sector is that they tackle a crucial bottleneck in robotics and AI, which is the difficulty to have all the AI tools interact together. Let me explain. Their latest release, OPTR (operator), is a toolkit that helps build operators capable of interacting on multiple platforms such as robots, desktops, browsers, or simulations. The objective of an operator is to see, reason, and act (VLA) in both digital (computers) and physical (robots) worlds. This toolkit serves as core infrastructure for robotic teams aiming to test their product and enhance the overall process by providing a unified experience instead of separate ones for web browsers, simulations, or robots. This essentially makes the operator adaptive and autonomous regardless of its environment. So you get it, it will save big time for companies and developers who previously had to go through each step manually and where you can save time you can save money. It will also enable Codec to build their own operator projects and launch new capacities relatively fast onto the market, notably through their marketplace. TL;DR: You probably have seen videos of robots folding tissues, sorting boxes, or jumping on various elements. They have all been trained for this very specific use case, and unfortunately, one skill cannot be re-used in another environment like a human could do. OPTR from Codec solves this by making skills transferrable among environments and situations, making training and development a lot faster and cheaper for enterprises. This is why Codec is so interesting in unifying the digital world with the physical world. $CODEC, Coded.

Top

Ranking

Favorites

Trending onchain

Trending on X

Recent top fundings

Most notable