> Home Assistant already knows which bins (trash cans, for the international readers) are due for collection. [...] What it can’t tell me is whether I’ve actually done anything about it.
My first gut reaction to this was: Have you tried using a vision-language model?
I used to have an elaborate Home Assistant setup with tons of zigbee sensors here and there telling me if front door is not closed properly, if the rice cooker is left on (via current monitoring), if the garage door is left open (with a magnetic contact sensor), if window is left open, if the soil is dry, etc. I had to keep changing batteries of all these things and manage flaky Zigbee connections in the past.
More recently, I've found the absolute most reliable, ~zero-maintainence solution to be an LLM. Home Assistant already has an LLM plugin, you can wire it up to a local LLM if you want, and you can just feed it an image and ask "Is this garage door open or closed" and I found that to be much better for my sanity than changing batteries every 6 months and dealing with flaky sensors sending me false alerts.
I even found it useful for "Did the cat vomit on the floor" (if so notify me and pause the cleaning robot schedule) and "Do any plants look like they need attention"
For TFA's use case, if you already have a security camera facing front, you can just feed it that image and ask "Are the garbage bins out"
Also, if you're worried about cameras sending home images to the cloud, there are non-cloud options. (I use Unifi cameras and Home Assistant gets a RTSP stream over LAN.)
> how often do you have to change your cameras' batteries, or are you accepting ugly wiring for powering from mains
Meh (a) I want security and cat-monitoring cameras anyway (b) I'm willing to take the effort to install cameras neatly since they're capable of serving as 100 different sensors
> constantly sending images of one's private space to a cloud LLM is Orwellian
I agree. I have a local server with a GPU. The security side of things records to a local NVR. Nothing goes to the cloud.
> IP cameras are flaky, too
haven't had nearly as many problems here
> are you polling your LLM minutewise "Did the cat vomit on the floor"?
Kind of, but it only re-polls if the camera detected motion, and then every 15 minutes as a catch-all.
qurren · · focus · HN ↗
My first gut reaction to this was: Have you tried using a vision-language model?
I used to have an elaborate Home Assistant setup with tons of zigbee sensors here and there telling me if front door is not closed properly, if the rice cooker is left on (via current monitoring), if the garage door is left open (with a magnetic contact sensor), if window is left open, if the soil is dry, etc. I had to keep changing batteries of all these things and manage flaky Zigbee connections in the past.
More recently, I've found the absolute most reliable, ~zero-maintainence solution to be an LLM. Home Assistant already has an LLM plugin, you can wire it up to a local LLM if you want, and you can just feed it an image and ask "Is this garage door open or closed" and I found that to be much better for my sanity than changing batteries every 6 months and dealing with flaky sensors sending me false alerts.
I even found it useful for "Did the cat vomit on the floor" (if so notify me and pause the cleaning robot schedule) and "Do any plants look like they need attention"
For TFA's use case, if you already have a security camera facing front, you can just feed it that image and ask "Are the garbage bins out"
Also, if you're worried about cameras sending home images to the cloud, there are non-cloud options. (I use Unifi cameras and Home Assistant gets a RTSP stream over LAN.)
Eduard · · focus · HN ↗
- how often do you have to change your cameras' batteries, or are you accepting ugly wiring for powering from mains?
- how many cameras do you have installed to cover your home?
- IP cameras are flaky, too
- constantly sending images of one's private space to a cloud LLM is Orwellian
- are you polling your LLM minutewise "Did the cat vomit on the floor"?
qurren · · focus · HN ↗
Meh (a) I want security and cat-monitoring cameras anyway (b) I'm willing to take the effort to install cameras neatly since they're capable of serving as 100 different sensors
> constantly sending images of one's private space to a cloud LLM is Orwellian
I agree. I have a local server with a GPU. The security side of things records to a local NVR. Nothing goes to the cloud.
> IP cameras are flaky, too
haven't had nearly as many problems here
> are you polling your LLM minutewise "Did the cat vomit on the floor"?
Kind of, but it only re-polls if the camera detected motion, and then every 15 minutes as a catch-all.