Build & Defend Newsletter 17

One email a week - something from which I hope you'll get real value. We talk about things we can build, and how to defend them. That can apply to cybersecurity, physical buildings, digital products, and .... just about anything. It gives me a lot of latitude in what I can write about, but the two concepts are important for progress - as individuals, and as society.

Today's topic is: Hermes Agents

If you’ve been reading Build & Defend for a while, you already know I’m interested in AI agents.

Not in the “look, a chatbot wrote a poem” sense. I mean agents as working infrastructure: software you can message, instruct, schedule, connect to tools, and use as a layer between your intentions and your systems.

I’ve been experimenting with this for a while now. OpenClaw was part of that. Local models were part of that. Morning briefings, command-line workflows, notes, scripts, research, publishing, server checks, and all the small connective tissue between tools have been part of it too.

So this is not a “wow, AI agents are coming” post - you got that earlier in the newsletter history. Sometimes against your will and more than once.

Agents are already here.

The more interesting question is what happens after the initial euphoria phase.

That phase is fun. You connect a model, wire up a chat interface, ask it to do something useful, and for a moment it feels like the future arrived early.

Then the normal infrastructure questions show up.

Where does it run?

What can it access?

What does it remember? Can I make it forget certain things only? (Yes, you'll ask that question eventually....)

What happens when it fails?

Can it use the tools I actually use?

Can I move it, back it up, and recover it? (See last issue.)

That is where Hermes Agent has become interesting to me.

A lot of AI agent discussion still sounds like people reviewing magic tricks, or really smart pets.

“Look what it did.” That’s fine at first. I like a clever pot-bellied pig as much as anyone.

But if an agent is going to become part of your day, the question is not whether it can do something impressive once. The question is whether it can do boring useful things repeatedly, inside boundaries you understand and control.

That is a much higher bar.

An agent that helps you once is a tool. An agent that runs every morning, remembers your preferences, checks systems, drafts useful work, uses your files, searches past conversations, and can be reached from your phone starts to become infrastructure. Especially when it builds other infrastructure in a meaningful way.

Not critical infrastructure on day one. Not something you blindly trust with your bank account and root passwords.

But infrastructure all the same. And infrastructure has different rules.

Why Hermes caught my attention:

Okay, it all comes back to that one guy on Youtube. (Alex Finn) He started talking about it. So I decided to experiment.

Of the five agents in my household (yes....) only one of them is still OpenClaw, and I'm keeping that one around just because.

So:

Hermes Agent is an open-source (excellent) agent framework ("harness") from Nous Research.

Heres' how I set it up (the steps are similar to how I did it with OpenClaw in an earlier newsletter):

It's actually pretty easy. You can get links for those things below, but essentially on Mac OS I virtualised another instance of Mac OS (installed lume), then I installed the actual Mac OS in that VM, then I installed "Home Brew" (a command line package manager). I installed Tailscale (an outside-in way of connecting all my machines to each other, with identities), then I installed Hermes Agent. That was the one line command that's available on their website. (I won't give you that here since you'll need to ensure that you're getting the correct one from that site. It ends in "install.sh | bash".) I use my ChatGPT subscription for its “brain”.

So why did I do it?

OpenClaw is very good - don't get me wrong. The issue is that it's less mature, less polished. I'm sure that Pete Steinberg is enjoying burning through millions of tokens now that he's at OpenAI, but frankly, none of that really seems to be translating to the agent software itself. It's an open source project (excellent) with a lot of issues (not excellent) that often impede bot operations. It's like going from Gentoo Linux to Ubuntu. Both are great, and both are essentially Linux, but the latter requires MUCH less effort than the former. The former is absolutely the best way to start - you learn an absolute ton compiling from scratch - but it's not how you want to spend all your days once you possess the knowledge. So I've moved mostly to Hermes.

The serious part: permissions and boundaries

This is where the “Defend” side comes in. (I wrote about permission surfaces last week; it still applies and will always apply.)

The more useful an agent becomes, the more careful you have to be.

An agent with no tools is mostly a conversation partner.

An agent with terminal access, files, messaging, browser access, scheduled jobs, and credentials is software with reach.

That doesn’t mean you should panic. It means you should design the setup like it matters. That's what I did - isolated the agent in its own environment. It can't do anything unless it's allowed, and to do that, you have to intentionally give it specific things.

Some basic questions for that environment:

This is also where Hermes is easier to defend than OpenClaw - these questions were mostly answered in advance. Hermes has a better way for storing secrets, by default, then OpenClaw.

But it is the part that determines whether agents become useful infrastructure or just another risky automation mess.

The goal is not to make the agent powerless, but useful in a controlled fashion.

Build it like you may depend on it later

My view on agents has become more practical over time.

At first, the fun part is capability. What can it do?

Then the interesting part becomes continuity. Can I keep using this without rebuilding it every week?

Agents should be things you route intent through.

That sounds abstract, so here’s the plain version:

I don’t want to manually connect every tool in my life every time I need something done.

I don’t want to copy from a PDF into a browser chatbot, then paste the answer into a note, then manually turn that into a task, then remember to follow up next Tuesday.

I want to say what I’m trying to do and have a controlled, inspectable system help move the work forward.

Not perfectly. Maybe not autonomously in every case. But enough to reduce the drag.

That is why this is so interesting to me.

The agent is not the whole answer. The workflow around the agent is where the value lives. That'll be a topic for another newsletter.

Wait - you have five agents?

Well, in the household, yes. My son has one, my wife has one, I have two on my Mac Studio, and one on my laptop. I've migrated them all to Hermes except the laptop one (StarQ) which is going to stay as OpenClaw as I'll want to be familiar with both still over time. But why stack them?

Well, they're all sorting themselves into different functions, seemingly based on personality. (It's weird, and it probably says as much about me as anything.) Argent, my first Hermes agent, seems responsible and is acting more like my Executive Assistant. Talos, as you all know, is a bit more like a first-year employee who has already screwed up once, so I don't trust him as much as I used to, because he lost me work and he doesn't follow conversations as well as he probably should. (Again, I'm presuming he can read a Telegram chat history, which he can't. First bot to be able to do that will be so much more useful.) StarQ is dependable, but mostly a content-generator and he's very good at doing that. So that's what he does.

Over time, more and more people will have more and more agents, and software will start to have agents, and functions will have agents, and your bank account will have an agent, and there will be an agent for your airline, and one in your plane's seat, and several in your car (though you might only talk to one) and hopefully none of them will have been designed by "FromLove" (Johny Ive's company which has just released the Ferrari Luce which is just terrible looking) but they will all be there. If you start by accumulating things you need done, and assigning an agent or two to do them, you'll be ahead of the game. You'll be limited by only your imagination, and your hardware budget. And Apple's shipping schedules, which are getting longer and longer. #sigh

Stacked too many OCs and want to move to Hermes? I've included my Hermes-agent-written-and-edited-by-me guide below in the Links section.

Lume
https://github.com/trycua/lume

Tailscale
https://tailscale.com/

Homebrew
https://brew.sh/

Hermes Agent GitHub
https://github.com/NousResearch/hermes-agent

Hermes Agent docs
https://hermes-agent.nousresearch.com/docs/

ChatGPT
https://www.openai.com
https://www.chatgpt.com

Mastering the Command Line for AI
https://amzn.to/4abq9pk

Alex Finn on Youtube
https://www.youtube.com/@AlexFinnOfficial

Already have an Open Claw you want to migrate to Hermes? Check out my guide:
https://thomasquinlan.gumroad.com/l/openclawtohermes

I've got a couple of topics in mind for future newsletters - one's on workflows, and the other is on piracy. (Did I spell "privacy" wrong? Maybe…. maybe not....) Which do you want to see first? Let me know!

Thanks for reading this newsletter! Feel free to respond any time.

Thomas

Was this forwarded to you? Subscribe at builddefend.fyi.

Had enough? [Unsubscribe] - no hard feelings.*

* Well, a little bit.