‹ BackHN Continuity

Thread

I built non-autoregressive decision models with RL a year ago

1363 points · 319 comments · nandakishor_ml

  1. johnfn · · focus · HN ↗
    It’s a tale as old as time — people don’t understand that marketing and branding are just as important, if not more so, than the product. Jev is exceptionally-well branded. Anyone can look at the webpage and understand it, and the implications, instantly.

    OPs “marketing” is a single post on Reddit titled “ Predicting sales conversion probability from conversations using pure Reinforcement Learning”. Can you understand what that means? I can’t, and I consider myself reasonably technical. Is it obvious it has the same implications as Jev? Again, no idea. And it was just a single post on a subreddit that I don’t even browse! I see people on this thread saying “Jev is just BERT”. Sure, and Dropbox is just a ftp account mounted with curlftpfs!

    I do feel bad for the author for finding something cool and being unable to brand it. But the full definition of “product” INCLUDES being able to coherently communicate it. In some sense the branding is just as much the “breakthrough” as the model.

    1. rvz · · focus · HN ↗
      This 100%. Engineers really lack understanding in marketing and branding.

      No one cares if you are "first". They only care if your product is known by as many people as possible and is better than all the other alternatives at solving a problem that is worth paying for.

      If you don't market, then no-one will care that you exist even if you solved a problem decades ago. Someone else will use your solution and take inspiration (and credit) off of your discovery because you didn't bother to tell anyone about it.

      This is exactly what happened here.

      1. verdverm · · focus · HN ↗
        another way to look at it, a product is the whole experience (landing, docs, sales, support, code, branding), not the implementation of an algorithm or process
      2. wavewrangler · · focus · HN ↗
        Sorry, but he did tell people about it, no? He showed his receipts. Reddit, arXiv- What I am seeing here is "it's just better marketing". When it comes to prior art, is better marketing sufficient? On one side we can say it's better marketing, but on another side, the side that should actually matter, coming from the direction of him being first, can't we say, it is just better research? Being that he was first and all, and Jev hasn't even published anything according other than what I read. What I am really trying to ask is, is marketing even relevant at this point? So if you have good marketing, you can just steal someone else's work, intentional or not?
        1. malux85 · · focus · HN ↗
          Steal is a loaded word here, and I argue that you cannot unintentionally steal someone's work. Stealing is deliberate, malicious theft. Its an action taken with consciousness of guilt. Other derivations exist but then its not stealing and therefore isnt that word.

          But that is NOT the point AT ALL. The point is, the same point that comes up on hacker news 1000 times a year - ideas alone are near worthless and execution matters.

          Execution includes marketing that gets you enough attention. Because theres 10,000 other similar ideas of varying quality and marketing that others will point go saying "I wAS tHErE fiRsT"

          All of this stems from the human bias of both (a) wishful thinking and (b)thinking people value what what we produce. These are natural human biases and are often dangerously wrong.

          Programmers always think its just the idea and a prototype that is valuable, because they can produce and idea and a prototype (people what what I have) which causes them to massively overvalue ideas and the importance of "who was first" and all of that because they are sanctifying the small thing they produce.

          It doesnt have to be malicious - the plain truth is theres 10,000 other ideas that are close enough that could be considered stealing even if they were truly independently developed, ideas are virtually worthless, get rid of your human biases that are clouding your judgment and focus on what matters if your idea is truly great : execution

          1. wavewrangler · · focus · HN ↗

            [dead]

    2. bennett_dev · · focus · HN ↗
      Especially with „ sales conversion probability“ it just doesn’t sound universal to other issues - there’s tons of unique models for specific use cases
    3. victor9000 · · focus · HN ↗
      It's also well established that an algorithm or architecture alone are not enough to produce a useful model. The same architecture can produce vastly different results depending on the training data, post-training, harness, etc.
    4. robrenaud · · focus · HN ↗
      > “Predicting sales conversion probability from conversations using pure Reinforcement Learning”. Can you understand what that means?

      I can understand it, and it wouldn't excite me at all.

      Jev has a beautiful API and is advertised as something much more general.

      1. verdverm · · focus · HN ↗
        The title doesn't reflect the content of the paper or project, which uses things like RAG and an orchestrator, so more than "pure RL"

        (the project before it was rehashed into Laya since Jev was released)

    5. tinyhouse · · focus · HN ↗
      You're right but it's not the full picture. It's much easier to market when you have a name brand behind you. Not sure the author would've done much better even if he messaged it better. It's like the difference between someone random saying something smart on Twitter and no one gives a shit and Karapthy saying the same thing and everyone talks about it. I'm not saying it in a bad way - those with clout around them earned the people's trust by doing something right. But it's not easy to get there and there are many people doing great things that get very little publicity if at all. Not to mention in this case Jev came from a startup that raised a lot of money and can spend it on good marketing.
    6. anonzzzies · · focus · HN ↗
      I think OP, as said on Reddit, wanted to get a lot of investment and ride the wave but did not know how to. This was said as such on Reddit today by them. This is indeed a shame especially it was a year earlier but indeed a lack of marketing; many people on Reddit told what to do in this case, in hindsight, it’s worth checking it out imho.
      1. verdverm · · focus · HN ↗
        this is a prior HN post from OP adding color to their underlying motivations

        <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49674396">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49674396

    7. calebkaiser · · focus · HN ↗
      This is also a really common thing in ML specifically. We joke about getting Schmidthuber&#x27;d, which is when Jurgen Schmidthuber (sometimes correctly) announces that he or one of his colleagues actually proposed your thing 37 years ago in a Japanese linguists journal.

      Statistical modeling, from simple classical stuff up to modern deep learning, just has this dynamic where the theory is rich and bottomless, but the actual components of implementation are pretty neat and compact. So for any given idea, there are probably 20,000 other people who have had the same intuition, just with subtly different application or implementation. Add in that depending on what your particular flavor of research is, you might name an almost identical implementation something completely different. And it leads to a huge amount of sour grapes whenever anyone&#x27;s idea really garners attention.

      If you listen to any podcast with a founder in the ML space who has been in it for long enough, they will invariably say at some point &quot;We actually developed xyz over a year before OpenAI&quot;

      1. hiddencost · · focus · HN ↗
        Schmidhuber rarely ever executed the idea correctly, which makes his claims particularly obnoxious.
    8. TomGarden · · focus · HN ↗
      Not to go all meta but the very post were commenting in is also good marketing and branding.

      So while the initial post was not good, the author is currently succeeding to some extent at what you&#x27;re describing

    9. XTXinverseXTY · · focus · HN ↗
      OP&#x27;s was leaky slop from day one [0][1], as is his article [2]

      It is arrogant and entitled for the author to take credit for the concept of RL over sequence embeddings, and none of the work that went into pretraining, not to mention the egregious target leakage [1]

      [0]: Author fails to grasp the concept of virtual environments <a href="https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;LocalLLaMA&#x2F;comments&#x2F;1kl0uvv&#x2F;comment&#x2F;ms0n31z&#x2F;?utm_source=share&amp;utm_medium=web3x&amp;utm_name=web3xcss&amp;utm_term=1&amp;utm_content=share_button" rel="nofollow">https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;LocalLLaMA&#x2F;comments&#x2F;1kl0uvv&#x2F;comment...

      [1]: his `train.py` has `outcome` as a model input (conversation_metrics built from _parse_conversation which includes outcome): <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-model-reinf-learning&#x2F;blob&#x2F;main&#x2F;train.py#L73" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-... <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-model-reinf-learning&#x2F;blob&#x2F;main&#x2F;train.py#L170" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-...

      [2]: 100% of this post is AI-generated <a href="https:&#x2F;&#x2F;www.pangram.com&#x2F;history&#x2F;97e0be84-391d-46b8-9c16-2d8fcf8aebf5?ucc=X4GRCBL3tlK" rel="nofollow">https:&#x2F;&#x2F;www.pangram.com&#x2F;history&#x2F;97e0be84-391d-46b8-9c16-2d8f...

      1. OceanKing · · focus · HN ↗
        For anyone else verifying, the target leakage appears to be as follows:

        1)`outcome` is part of `metrics` at <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-model-reinf-learning&#x2F;blob&#x2F;main&#x2F;train.py#L226" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-... and <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-model-reinf-learning&#x2F;blob&#x2F;main&#x2F;train.py#L272" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-...

        2) `metrics` goes into `ConversationState` at <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-model-reinf-learning&#x2F;blob&#x2F;main&#x2F;train.py#L285" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-... and <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-model-reinf-learning&#x2F;blob&#x2F;main&#x2F;train.py#L238" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-...

        3) `metrics` (including `outcome`) makes its way into `ConversationState.state_vector` at <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-model-reinf-learning&#x2F;blob&#x2F;main&#x2F;train.py#L73" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-..., and is returned from environment `step()` and `reset()` functions at <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-model-reinf-learning&#x2F;blob&#x2F;main&#x2F;train.py#L243" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-... and <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-model-reinf-learning&#x2F;blob&#x2F;main&#x2F;train.py#L290" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-...

        4) model ingests `state_vector` as input at <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-model-reinf-learning&#x2F;blob&#x2F;main&#x2F;train.py#L537" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-...

      2. fxwin · · focus · HN ↗
        &gt; not to mention the egregious target leakage

        I was curious about this so I skimmed the paper [0]:

        &gt; SalesRLAgent achieved 96.7% accuracy, outperforming the best commercial alternative by 23.7 percentage points and the best LLM approach by 34.7 percentage points.

        For a fuzzy natural language task like this, this magnitude of improvement should already set off alarm bells (Though i admit I&#x27;m not even sure what accuracy is even measured here, and the paper doesn&#x27;t help either). Also, &quot;best LLM&quot; here refers to GPT-4 (at the time of upload, the public already had access to GPT-o3 and). I would have loved to contextualize the performance by looking at model size, but the paper is frustratingly devoid of detail in that regard:

        &gt; The core of SalesRLAgent is a reinforcement learning architecture consisting of: • A state encoder network that processes Azure OpenAI embeddings and features • A policy network that estimates conversion probability based on the current state • A value network that estimates the expected cumulative reward • A meta-learning module that assesses prediction confi dence

        Also:

        &gt; Beyond technical metrics, we evaluated SalesRLAgent in real-world sales environments through A&#x2F;B testing. [...] After 90 days across 217 representatives and 12,433 con versations, we observed: • 43.2% increase in conversion rate for the test group

        This would be a pretty huge result but the fact that this is just shoved into a single paragrpah with no further discussion on methodology, baselines and setup makes me very suspicious.

        [0] <a href="https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2503.23303" rel="nofollow">https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2503.23303

    10. porridgeraisin · · focus · HN ↗
      I think their problem is more not being cited by the team at typesafe, as in general academic politeness. On the one hand you have the charitable assumption that they developed it independently. On the other hand, my opinion is that it is naive to expect companies to do that even if they took inspo from it, especially when this is a core product theme, and not just some supporting infra. They will of course market it as their own. If they ever release a technical report, they might cite it there, but there is no way their landing page and announcement tweet cites it.

      Also, the way highly empirical fields like ML work is that it could very well be the case that typesafe had to do a _lot_ of work to improve this one, and in this field it ends up different enough that they feel they are doing something entirely novel[1]. I am not endorsing that 100%, but that happens a lot even between academics. In many cases it is valid.

      [1] For example, this guys implementation seems to have atleast one serious issue, as {solution to OLS} points out in a sibling comment: <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49770027">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=49770027

    11. tootie · · focus · HN ↗
      I wrote an article detailing an idea I had back during web 1.0. I thought it was clever and maybe some people would find it interesting. Few years later a SaaS popped up selling the same idea as a service. It eventually became a pretty commonly followed pattern for a bunch of huge companies in the same domain.

      I have no idea if I was literally the first person to have this idea or if anyone who launched one of these businesses read my article. I definitely didn&#x27;t understand how much commerical value there was or even considered making a business out of it. I blame nobody but myself for missing an opportunity if there even was one.

      I did get like $200 for writing it which was nice.

    12. [deleted] · · focus · HN ↗

      [deleted]

    13. mcapodici · · focus · HN ↗
      1052 pts on HN. I&#x27;d say they are pretty good at marketing actually.

      While they may not initially done well they are certainly riding this wave.

      1. onion2k · · focus · HN ↗
        1052 pts on HN. I&#x27;d say they are pretty good at marketing actually.

        HN readers are awesome at saying something is great and upvoting it, but unless HN readers are your market it means absolutely nothing. Marketing is as much about putting your message to the right audience as it is about saying the right thing.

        This is made more complicated because a group as diverse as HN readers probably does contain some people who are in your target market, to be fair. The problem is that you&#x27;re getting a strong signal from the whole cohort rather than the bit you&#x27;re interested in, and it&#x27;s really easy to conflate that with a sign of success.

        As always with any startup activity, unless people are actually giving you their money it doesn&#x27;t count and you should consider it a vanity metric.

      2. derwiki · · focus · HN ↗
        I would have clicked sooner if it was titled “I built Jev a year ago”; instead, I waited until this had 1400 points to see what all the fuss was about
    14. d--b · · focus · HN ↗
      I feel like a HN post hitting 1000+ points is a good way to catch up, communication wise.
    15. geuis · · focus · HN ↗
      I don&#x27;t understand why people continue to use em-dashes. As far I know, the comment system on HN doesn&#x27;t offer them. And iOS and android don&#x27;t make them a default. They aren&#x27;t something normally used by humans because - is a keyboard option that&#x27;s easier to use.

      The usage of them immediately makes your commentary suspect. Either you aren&#x27;t using the standard web interface to make a comment, you&#x27;re using and odd 3rd party client, or its LLM generated.

      1. isityettime · · focus · HN ↗
        I&#x27;ve been using compose keys on desktop operating systems and the default keyboard on Android to write em-dashes for 10 years. I also use LLMs every workday, but I have never once used one to write, review, or edit prose because my voice is important to me.

        I don&#x27;t give a shit if my Hacker News comments sometimes look &quot;suspect&quot; to some people. The way I use language is deeply personal and I&#x27;m not going to let the clankers or reactionaries against them take it away from me.

        1. hbarka · · focus · HN ↗
          Hear, hear! An em-dash is a great stylistic device for structuring sentences. It saddens me that the use of it now has become some sort of scarlet letter. I’m hoping the co-opting can be reversed.
      2. swingboy · · focus · HN ↗
        On a Mac it’s literally just option+dash.
        1. kwinkunks · · focus · HN ↗
          That&#x27;s an en-dash (eg for number ranges). You need option+shift+dash for an em-dash.
      3. johnfn · · focus · HN ↗
        I just typed dash twice into iPhone. Hardly a “non-standard” web interface. Also, I’d like to think my comment was higher quality than anything an LLM could generate! At least, yet.
    16. glerk · · focus · HN ↗
      &gt; Predicting sales conversion probability from conversations

      That&#x27;s not just bad marketing, it&#x27;s an example of anti-marketing.

      Sales conversion? That makes me think of an old car&#x27;s salesman trying to scam me into buying something I don&#x27;t want. I positively don&#x27;t want to read this paper based on the title.

    17. WhitneyLand · · focus · HN ↗
      How many people actually read the full post? It builds up this amazing underdog story where all the benchmarks are taken as victories, and then only late in the post and section 6 is it finally revealed that the only way they won was to fine tune directly on the benchmark.

      This comparison doesn’t even make sense.

    18. ranger_danger · · focus · HN ↗
      There are entire youtube channels dedicated to &quot;who did it first&quot; rabbit holes regarding inventions that didn&#x27;t gain popularity until it was properly marketed by someone else, whether independently discovered or not.
    19. emykhailenko · · focus · HN ↗
      I know why Jev became popular: <a href="https:&#x2F;&#x2F;x.com&#x2F;mykhailen&#x2F;status&#x2F;2102219600192565586?s=20" rel="nofollow">https:&#x2F;&#x2F;x.com&#x2F;mykhailen&#x2F;status&#x2F;2102219600192565586?s=20
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.