The site doesn't make clear whether the models are looking at the image while they work on it, but the code includes a "look" tool they can call: "Every painter sees its looks at its provider's best image resolution."
If you ask for an SVG, they'll do it without seeing it. At least they would up until the last time I tried it. And, sometimes, it'll be recognizably the thing you asked for.
So, they can sketch without looking at it, and models without vision capability can also do it. But, I've been tinkering with claude-paint this evening, and you can see it improving over time as it draws, looks at it, draws some more. It's impressive, and it somehow mostly avoids the uncanny valley at least some of the time, which direct image models consistently still fail to do, IMHO (though lots of people on facebook are falling for it, so it obviously isn't uncanny to them).
Agent harnesses I think will usually look at it. Chat is much less inclined to do tool calls. Though it seems like both OpenAI and Anthropic are trying to close this gap by merging an agentic product into chat.
(project creator here) thanks yes maybe i should clarify it. they don't (currently) have reference images but but they very much use their vision capabilities indeed that's a big differentiatior imo (but not the only one)
dcre · · focus · HN ↗
<a href="https://github.com/aliceisjustplaying/claude-paint/blob/f0382c8c58450af83707bb1d11eb8caeea10d4cb/harness/painter/painter.ts#L8-L11" rel="nofollow">https://github.com/aliceisjustplaying/claude-paint/blob/f038...
It would have freaked me out if they could do this well without that, especially the dumber models.
SwellJoe · · focus · HN ↗
So, they can sketch without looking at it, and models without vision capability can also do it. But, I've been tinkering with claude-paint this evening, and you can see it improving over time as it draws, looks at it, draws some more. It's impressive, and it somehow mostly avoids the uncanny valley at least some of the time, which direct image models consistently still fail to do, IMHO (though lots of people on facebook are falling for it, so it obviously isn't uncanny to them).
geraneum · · focus · HN ↗
Isn’t it the same as the pelican?
dcre · · focus · HN ↗
aliceisjustplay · · focus · HN ↗