‹ BackHN Continuity

Thread

I built non-autoregressive decision models with RL a year ago

1363 points · 319 comments · nandakishor_ml

  1. johnfn · · focus · HN ↗
    It’s a tale as old as time — people don’t understand that marketing and branding are just as important, if not more so, than the product. Jev is exceptionally-well branded. Anyone can look at the webpage and understand it, and the implications, instantly.

    OPs “marketing” is a single post on Reddit titled “ Predicting sales conversion probability from conversations using pure Reinforcement Learning”. Can you understand what that means? I can’t, and I consider myself reasonably technical. Is it obvious it has the same implications as Jev? Again, no idea. And it was just a single post on a subreddit that I don’t even browse! I see people on this thread saying “Jev is just BERT”. Sure, and Dropbox is just a ftp account mounted with curlftpfs!

    I do feel bad for the author for finding something cool and being unable to brand it. But the full definition of “product” INCLUDES being able to coherently communicate it. In some sense the branding is just as much the “breakthrough” as the model.

    1. XTXinverseXTY · · focus · HN ↗
      OP's was leaky slop from day one [0][1], as is his article [2]

      It is arrogant and entitled for the author to take credit for the concept of RL over sequence embeddings, and none of the work that went into pretraining, not to mention the egregious target leakage [1]

      [0]: Author fails to grasp the concept of virtual environments <a href="https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;LocalLLaMA&#x2F;comments&#x2F;1kl0uvv&#x2F;comment&#x2F;ms0n31z&#x2F;?utm_source=share&amp;utm_medium=web3x&amp;utm_name=web3xcss&amp;utm_term=1&amp;utm_content=share_button" rel="nofollow">https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;LocalLLaMA&#x2F;comments&#x2F;1kl0uvv&#x2F;comment...

      [1]: his `train.py` has `outcome` as a model input (conversation_metrics built from _parse_conversation which includes outcome): <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-model-reinf-learning&#x2F;blob&#x2F;main&#x2F;train.py#L73" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-... <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-model-reinf-learning&#x2F;blob&#x2F;main&#x2F;train.py#L170" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-...

      [2]: 100% of this post is AI-generated <a href="https:&#x2F;&#x2F;www.pangram.com&#x2F;history&#x2F;97e0be84-391d-46b8-9c16-2d8fcf8aebf5?ucc=X4GRCBL3tlK" rel="nofollow">https:&#x2F;&#x2F;www.pangram.com&#x2F;history&#x2F;97e0be84-391d-46b8-9c16-2d8f...

      1. OceanKing · · focus · HN ↗
        For anyone else verifying, the target leakage appears to be as follows:

        1)`outcome` is part of `metrics` at <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-model-reinf-learning&#x2F;blob&#x2F;main&#x2F;train.py#L226" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-... and <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-model-reinf-learning&#x2F;blob&#x2F;main&#x2F;train.py#L272" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-...

        2) `metrics` goes into `ConversationState` at <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-model-reinf-learning&#x2F;blob&#x2F;main&#x2F;train.py#L285" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-... and <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-model-reinf-learning&#x2F;blob&#x2F;main&#x2F;train.py#L238" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-...

        3) `metrics` (including `outcome`) makes its way into `ConversationState.state_vector` at <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-model-reinf-learning&#x2F;blob&#x2F;main&#x2F;train.py#L73" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-..., and is returned from environment `step()` and `reset()` functions at <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-model-reinf-learning&#x2F;blob&#x2F;main&#x2F;train.py#L243" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-... and <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-model-reinf-learning&#x2F;blob&#x2F;main&#x2F;train.py#L290" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-...

        4) model ingests `state_vector` as input at <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-model-reinf-learning&#x2F;blob&#x2F;main&#x2F;train.py#L537" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;DeepMostInnovations&#x2F;sales-conversion-...

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.