Thread

I built non-autoregressive decision models with RL a year ago

1363 points · 319 comments · nandakishor_ml

johnfn · 1789843452 · focus · HN ↗
It’s a tale as old as time — people don’t understand that marketing and branding are just as important, if not more so, than the product. Jev is exceptionally-well branded. Anyone can look at the webpage and understand it, and the implications, instantly.
OPs “marketing” is a single post on Reddit titled “ Predicting sales conversion probability from conversations using pure Reinforcement Learning”. Can you understand what that means? I can’t, and I consider myself reasonably technical. Is it obvious it has the same implications as Jev? Again, no idea. And it was just a single post on a subreddit that I don’t even browse! I see people on this thread saying “Jev is just BERT”. Sure, and Dropbox is just a ftp account mounted with curlftpfs!
I do feel bad for the author for finding something cool and being unable to brand it. But the full definition of “product” INCLUDES being able to coherently communicate it. In some sense the branding is just as much the “breakthrough” as the model.
1. XTXinverseXTY · 1789851116 · focus · HN ↗
  OP's was leaky slop from day one [0][1], as is his article [2]
  It is arrogant and entitled for the author to take credit for the concept of RL over sequence embeddings, and none of the work that went into pretraining, not to mention the egregious target leakage [1]
  [0]: Author fails to grasp the concept of virtual environments <a href="https://www.reddit.com/r/LocalLLaMA/comments/1kl0uvv/comment/ms0n31z/?utm_source=share&utm_medium=web3x&utm_name=web3xcss&utm_term=1&utm_content=share_button" rel="nofollow">https://www.reddit.com/r/LocalLLaMA/comments/1kl0uvv/comment...
  [1]: his `train.py` has `outcome` as a model input (conversation_metrics built from _parse_conversation which includes outcome): <a href="https://huggingface.co/DeepMostInnovations/sales-conversion-model-reinf-learning/blob/main/train.py#L73" rel="nofollow">https://huggingface.co/DeepMostInnovations/sales-conversion-... <a href="https://huggingface.co/DeepMostInnovations/sales-conversion-model-reinf-learning/blob/main/train.py#L170" rel="nofollow">https://huggingface.co/DeepMostInnovations/sales-conversion-...
  [2]: 100% of this post is AI-generated <a href="https://www.pangram.com/history/97e0be84-391d-46b8-9c16-2d8fcf8aebf5?ucc=X4GRCBL3tlK" rel="nofollow">https://www.pangram.com/history/97e0be84-391d-46b8-9c16-2d8f...
  1. fxwin · 1789982241 · focus · HN ↗
    > not to mention the egregious target leakage
    I was curious about this so I skimmed the paper [0]:
    > SalesRLAgent achieved 96.7% accuracy, outperforming the best commercial alternative by 23.7 percentage points and the best LLM approach by 34.7 percentage points.
    For a fuzzy natural language task like this, this magnitude of improvement should already set off alarm bells (Though i admit I'm not even sure what accuracy is even measured here, and the paper doesn't help either). Also, "best LLM" here refers to GPT-4 (at the time of upload, the public already had access to GPT-o3 and). I would have loved to contextualize the performance by looking at model size, but the paper is frustratingly devoid of detail in that regard:
    > The core of SalesRLAgent is a reinforcement learning architecture consisting of: • A state encoder network that processes Azure OpenAI embeddings and features • A policy network that estimates conversion probability based on the current state • A value network that estimates the expected cumulative reward • A meta-learning module that assesses prediction confi dence
    Also:
    > Beyond technical metrics, we evaluated SalesRLAgent in real-world sales environments through A/B testing. [...] After 90 days across 217 representatives and 12,433 con versations, we observed: • 43.2% increase in conversion rate for the test group
    This would be a pretty huge result but the fact that this is just shoved into a single paragrpah with no further discussion on methodology, baselines and setup makes me very suspicious.
    [0] <a href="https://arxiv.org/abs/2503.23303" rel="nofollow">https://arxiv.org/abs/2503.23303

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.