Apparently Grok 4.7 has 40% more weights than Grok 4.6, but the price ($6 output token, $2 input) is the same.
Given that the decrease in their margin and the fact they delayed the release of Grok 4.7 almost two weeks past the original date, XAI must not have been happy with the results for 4.7. And XAI also waited the day before Opus 5.5 is rumored to launch. I imagine Opus 5.5 will blow Grok 4.7 out of the water benchmark wise.
However, I have become skeptical of benchmarks. Grok 4.5 solved some issues setting up a buildroot system that Fable 5 couldn't do. I find the post cursor groks are phenomenal at frontend web development, though Claude is much better at backend ruby.
My favorite part of the new Groks has been how they speak in plain english. I simply cannot stand Claudish. Or even GPT, which doesn't have Claude's ticks but definitely likes to handwave explaining technical concepts. Still, nothing beats Claude 3.5 and 4 with explaining since it seems all models have regressed. I wonder if Grok 4.7 will also regress with English because of all the RL.
FYI a quick fix for claudish is to ask for the response to be in ASD-STE100 (Simple Technical English). Then it is far more readable. But I would agree that this is an annoyance and shouldn't require user workaround to get something readable.
I think this is more a meme than anything else, for a couple reasons:
First, after a while it's just as grating as Claudeish.
Second, my hunch is that it constricts the actual thinking of the LLM, like the same way that Newspeak does in 1984. It shrinks the range of thought that can be expressed if used as an input.
I think the real way to do it is to have another Claude entirely deal with the user as a liaison, but to keep the thinking in whatever format it came in.
Latent space reasoning, if you think about it, is exactly this to a crazy degree: why even formulate a thought as words if you can just keep it as matmuls until the user needs it? And then, if the user needs it, have it always specifically formulated for the user by another LLM rather than constrict its range of thought? Anyway, that's my take.
> my hunch is that it constricts the actual thinking of the LLM
I've found that prompting any constraint on output (length, style, vocab, even simple formatting) not only places additional cognitive load on the model, which burns some of whatever cognitive budget is available, it will also often skew the output in other subtle and completely unrelated ways.
Since I found this artifact interesting, I did some pretty extensive experiments a couple months ago. The increased load is real, although it may not be apparent if you're not near any cognitive boundaries. The subtle skew, however, seems nearly ever-present regardless of load.
When I noticed web chat LLMs wouldn't number section headings correctly and consistently, I began experimenting with modifying the user prompt over a period of weeks. My usage at that time was research and learning not coding. In long, detailed sessions with branching sub-topics and deep follow-ups, I found it helpful if the LLM would number each major section and letter each sub-section in replies. Simple markdown formatting that most web chat LLMs do sometimes, but not consistently nor uniformly.
While extensive, my tests were just following my curiousity, not controlled, exhaustive or well-documented. I identified about a dozen prior sessions of varying length and complexity to test and downloaded them with a browser add-on. I then removed all other user prompt instructions except for the formatting instruction. A test would typically involve changing the wording of the formatting instruction ranging from brutally simple to detailed and complete, then starting a new session, seeding one of the test sessions and continuing it. To get a feel for baseline inter-session variation, I also tried running the exact same prompt/session multiple times back-to-back, at different times and on different days of the week.
Once I identified a promising prompt candidate, I'd make it the formatting instruction in my regular, daily-use prompt for a few days. I quickly got a feel for how seemingly minor user prompt variations impact response quality, compliance and tone across fresh sessions as well as those in various states of context rot, drift, decay and cliff (<--my ni cknames for the distinct flavors of session degradation than technical terms of art).
My overall conclusion was that every instruction, no matter how minor or unrelated it seems, has some, real impact on the model's cog load, attentional focus and/or attentional weight budget. Both how these impacts manifest and what causes more or less impact is often extremely counteriintuitive. To more fully understand this, I eventually, got to the point of testing null case variants, such as the entire user prompt being one sentence completely unrelated to text formatting or the session topic, like: "Don't reference the cartoon character SnagglePuss" (in a deep dive on ancient Sumerian clay tokens). Similarly, a simple one sentence prompt requesting something the model already always does naturally also has a cost (eg "Capitalize proper nouns"). As others have observed, heavy emphasis, absolute prohibitions or emotional weight in prompts also tend to have outsized impact in both skew (impacting unrelated output tone/style) and in accelerating session degradation. "Avoid referencing SnagglePuss when you can" would have equal compliance but fewer downside impacts than "NEVER reference the cartoon character SnagglePuss" in sessions starting to degrade.
There were also surprises, such as when I was scanning transcripts of an older, longer session and noticed the LLM was doing number formatting almost perfectly. On looking at the active user prompt at the time (I keep a log of every user prompt change I make for every model), it didn't even reference formatting at all. More experimentation showed it a result of the LLM gradually mirroring my consistent use of formatting structure in my prompts over a long session (in which I never mentioned anything about formatting). Unfortunately, that mirrored trait doesn't persist to new sessions and reaching that point requires a substantial number of rounds burning quite a bit of context window.
moojacob · · focus · HN ↗
Given that the decrease in their margin and the fact they delayed the release of Grok 4.7 almost two weeks past the original date, XAI must not have been happy with the results for 4.7. And XAI also waited the day before Opus 5.5 is rumored to launch. I imagine Opus 5.5 will blow Grok 4.7 out of the water benchmark wise.
However, I have become skeptical of benchmarks. Grok 4.5 solved some issues setting up a buildroot system that Fable 5 couldn't do. I find the post cursor groks are phenomenal at frontend web development, though Claude is much better at backend ruby.
My favorite part of the new Groks has been how they speak in plain english. I simply cannot stand Claudish. Or even GPT, which doesn't have Claude's ticks but definitely likes to handwave explaining technical concepts. Still, nothing beats Claude 3.5 and 4 with explaining since it seems all models have regressed. I wonder if Grok 4.7 will also regress with English because of all the RL.
smashers1114 · · focus · HN ↗
a2dam · · focus · HN ↗
First, after a while it's just as grating as Claudeish. Second, my hunch is that it constricts the actual thinking of the LLM, like the same way that Newspeak does in 1984. It shrinks the range of thought that can be expressed if used as an input.
I think the real way to do it is to have another Claude entirely deal with the user as a liaison, but to keep the thinking in whatever format it came in.
Latent space reasoning, if you think about it, is exactly this to a crazy degree: why even formulate a thought as words if you can just keep it as matmuls until the user needs it? And then, if the user needs it, have it always specifically formulated for the user by another LLM rather than constrict its range of thought? Anyway, that's my take.
mrandish · · focus · HN ↗
I've found that prompting any constraint on output (length, style, vocab, even simple formatting) not only places additional cognitive load on the model, which burns some of whatever cognitive budget is available, it will also often skew the output in other subtle and completely unrelated ways.
Since I found this artifact interesting, I did some pretty extensive experiments a couple months ago. The increased load is real, although it may not be apparent if you're not near any cognitive boundaries. The subtle skew, however, seems nearly ever-present regardless of load.
evulhotdog · · focus · HN ↗
mrandish · · focus · HN ↗
While extensive, my tests were just following my curiousity, not controlled, exhaustive or well-documented. I identified about a dozen prior sessions of varying length and complexity to test and downloaded them with a browser add-on. I then removed all other user prompt instructions except for the formatting instruction. A test would typically involve changing the wording of the formatting instruction ranging from brutally simple to detailed and complete, then starting a new session, seeding one of the test sessions and continuing it. To get a feel for baseline inter-session variation, I also tried running the exact same prompt/session multiple times back-to-back, at different times and on different days of the week.
Once I identified a promising prompt candidate, I'd make it the formatting instruction in my regular, daily-use prompt for a few days. I quickly got a feel for how seemingly minor user prompt variations impact response quality, compliance and tone across fresh sessions as well as those in various states of context rot, drift, decay and cliff (<--my ni cknames for the distinct flavors of session degradation than technical terms of art).
My overall conclusion was that every instruction, no matter how minor or unrelated it seems, has some, real impact on the model's cog load, attentional focus and/or attentional weight budget. Both how these impacts manifest and what causes more or less impact is often extremely counteriintuitive. To more fully understand this, I eventually, got to the point of testing null case variants, such as the entire user prompt being one sentence completely unrelated to text formatting or the session topic, like: "Don't reference the cartoon character SnagglePuss" (in a deep dive on ancient Sumerian clay tokens). Similarly, a simple one sentence prompt requesting something the model already always does naturally also has a cost (eg "Capitalize proper nouns"). As others have observed, heavy emphasis, absolute prohibitions or emotional weight in prompts also tend to have outsized impact in both skew (impacting unrelated output tone/style) and in accelerating session degradation. "Avoid referencing SnagglePuss when you can" would have equal compliance but fewer downside impacts than "NEVER reference the cartoon character SnagglePuss" in sessions starting to degrade.
There were also surprises, such as when I was scanning transcripts of an older, longer session and noticed the LLM was doing number formatting almost perfectly. On looking at the active user prompt at the time (I keep a log of every user prompt change I make for every model), it didn't even reference formatting at all. More experimentation showed it a result of the LLM gradually mirroring my consistent use of formatting structure in my prompts over a long session (in which I never mentioned anything about formatting). Unfortunately, that mirrored trait doesn't persist to new sessions and reaching that point requires a substantial number of rounds burning quite a bit of context window.