Build & Defend Newsletter 9

One email a week - something from which I hope you'll get real value. We talk about things we can build, and how to defend them. That can apply to cybersecurity, physical buildings, digital products, and .... just about anything. It gives me a lot of latitude in what I can write about, but the two concepts are important for progress - as individuals, and as society.

Today's topic is: building your agent!

I've been getting a lot of requests to talk about how I've built my agents. Here's an actual step-by-step guide to do just that.

🔨 BUILD: Your AI

Your agent needs somewhere to run, and it needs software as it's "harness" (framework / installation) that will allow it to have a memory and functionality. It needs a "brain" (the actual AI model (typically a GPT like ChatGPT or Claude). It also needs an environment. As you'll see later, you generally want that environment to be isolated. That can be a virtual machine, container, or separate machine. I'm writing this from the perspective of Mac OS because that's what I run, but the steps should be broadly transferrable to Linux, and then semi-transferable to Windows.

First, set up a VM. I use Lume on Mac OS, which allows you to virtualise Mac OS INSIDE Mac OS. It's free, downloadable, and easy to use. You connect to the resultant VM w/ VNC, which is already present in Mac OS. (It does this automatically when the OS is built.)

Open a Terminal on the Mac, and install Lume:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/lume/scripts/install.sh)"

(As with anything, you should make sure you understand what these commands are doing. This goes to the Github for Cua, and installs Lume directly. However, you should also check that this is secure and I'm pointing you to the right place, and the software there is what you expect!)

Once you have it installed, you want to build your VM:

lume create myaibot --memory 16GB --disk-size 250GB --display 1600x900 --os macos --ipsw latest

When you create the VM, you can change the disk size, memory, and display size using flags to the VM. You can see the ones above I used to change to the RAM (16GB), disk size (250GB), and display (1600x900). To run it once created, run:

lume run mayaibot

I run this command using "screen" in the Terminal. That way I can always reconnect to the terminal session if I need to, and I can see the lume output, and I don't have to worry about redirecting output so that it's not blocking a terminal session that I'm using.

Once that's done, your machine should start up a VNC session for you right away. You can then connect to it like any other Mac. From the welcome screen you create a user, and then connect with that user account. Once you have that account, use the VM like a normal Mac.

Then install OpenClaw:

curl -fsSL https://openclaw.ai/install.sh | bash

Again, making sure that you're connecting to the correct thing and that you expect what will happen will actually happen. I could change on character in that line (I haven't) and do all sorts of malicious things! OpenClaw will download and it will walk you through the onboarding process.

That process is a bit extensive, so I won't go through it here, but you will need to connect it to some kind of GPT. You can do that via online (ChatGPT, Claude), or locally. I've not yet set one up with local, but for for an online service, you need to open an API key from your provider, and then paste that into the onboarding once you get to that point. It will also ask you what skills you want to install; only install the ones you'll immediately need; if there are some about which you're unsure, ignore them - you can always install them later.

It will ask you if you want to install the gateway service; this is what runs the OpenClaw, so do that. You can set up Telegram or WhatsApp (outside the scope of this, but straightforward - you can even ask the bot) and by default you can connect to it with a web browser inside the VM. It will tell you in the Terminal at what address the bot is listening once you finish the onboarding.

So now we've got an environment (Mac OS on lume), a harness (OpenClaw) and an AI (ChatGPT, Claude, etc.) which you picked.

Congratulations, you now have an AI agent!

You can - and should - leave open the VNC session in which the bot is running. The terminal session from which you started the bot in the VM will give you information as to what the bot is doing and the command it's trying to run. You can chat with it in the VM or using WhatsApp / Telegram, the latter of which is the easier and recommended method.

Anything to which you want the bot to have access you can give it in the VM. I've given it calendar access for my Apple Calendar and I can see it interact with the calendar in that VM when it schedules things for me. I've also given it access to my email and its own email account, and I recommend the latter more than the former. That said, if it screws up my email I'm not overly worried, and so far it hasn't sent anything I didn't ask it to send. 😉 I haven't given it any other kind of access to anything else yet.

However, without giving it access, you'll still want to be able to have it do things for you. It will create files, etc., and you'll need access to those. There are four ways you can do this:

I use Evernote for my notes, and you can have notes emailed into your account. The bot can email me files that show up in Evernote. This also gives me a permanent record.
You can have it output the file information directly into Whats App or Telegram (this is expensive when it comes to token use though (tokens are basically "half a word" and Open AI, Anthropic, etc. basically charge you by token use)).
You can use standard file transfer tools within Mac OS. I created a read-only SMB file share if I want it to see files (it's one directory it can't alter into which I place things I want it to see) and for anything it creates locally in the VM I use SCP (secure copy) to transfer those files to my host machine. It does not have any credentials on my machine and can't alter or connect to anything on the host.
It can run a Python webserver (the 'python3 -m http.server’) command in any directory, and you can connect to the IP of the VM and it will list the files that you can then download through a browser.

I'm most comfortable with the command line, so I usually just SCP for those things that aren't going into Evernote.

That should give you a start as to running an agent - now you have to defend it!

🛡️ DEFEND: Your AI

Defence in this instance refers to several things, but I break them down thusly:

Defending yourself generally
Defending yourself against the agent
Defending the agent against itself
Defending the agent against the world

Some of those might sound silly, but they're necessary.

The first one I hope is obvious; presume that on the internet, you need to be careful. Remember what I said above about me changing something in the URL and being able to do malicious things? You need to be careful where you click. This includes links like the ones above, in emails, and on the web. If you're not sure about something, stop, check, and don't rush. Consult people you know in tech who might be able to help if you're not sure. Make sure you read URLs - if they look funny or go places you weren't expecting, don't be afraid to stop what you're doing.

Defending yourself against the agent I covered above. The biggest thing you've already done is to isolate it to its own environment and give it minimal access to things. It should largely be a one-way push when it comes to information - it gives you things based on what you're trying to do, and you give it the bare minimum that it needs to do what it needs to do. As mentioned, I'm largely okay with it having calendar and email access, but I don't give it file access. It can't write to my hard drive. (It can write to the VM hard drive obviously.) For my wife and son, I bought separate machines (Mac Minis) onto which I installed their bots, but there isn't anything on those machines other than the bots.

Defending the agent against itself does sound a bit silly, but it can make mistakes, and it can hallucinate. You have to constantly remind it of things. One of the best ways to defend against this is to upgrade its memory. You can do that by installing something like "lossless claw" or "memory palace". For lossless claw:

openclaw plugins install @martian-engineering/lossless-claw

As mentioned, make sure you understand what that is doing before running it! (I harp on that point because it's an important one.)

For memory palace:

git clone https://github.com/jeffpierce/memory-palace.git

cd memory-palace

pip install -e .

Then you run the setup wizard as in the documentation and it will walk you through the rest.

Defending the agent against the world basically means making sure no one else can connect to it, and also that the software that goes into it is free from 'poisoning'. When you install things, those installers almost always pull in other packages and softwares, and recently "supply chain" attacks are gaining popularity as malware authors poison the 'downstream' software that is pulled in. Your easiest defence against this is to ensure that you're not using the most recent versions of things - you want to set an age (typically in things like homebrew and npm) where you're waiting at least 7 days to get the downstream software. For NPM, configure it like this (by editing the .npmrc file):

minimumReleaseAge: 10080

(That’s seven days in seconds.) This ensures that when people run attacks, you're giving the cybersecurity community time to detect that and fix things. Ironically, as quick as things go with respect to AI, there's rarely an instance where you'll need software younger than seven days, and if you do, isolation as set up helps mitigate things. (A lot of the supply chain attacks go for passwords, crypto wallets, etc. and if you're running the AI in a machine that doesn't have those, well, they don't get them.)

💰 STACK: Productivity

Once you have your AI in place, there's lots you can do. They're VERY good at anything text-based. You can of course chat with them as you normally would any other AI. You can build websites (insanely quickly) in multiple languages. You can have it help you write that novel you've always wanted to write. You can have it write the novel entirely and then edit it. It can help you with pictures - depending on the AI. You can have it create prompts that you can feed into other AIs. You can have it run as your secretary and schedule things for you. It can write emails for you. Your imagination is basically the limit. One way to get enhanced productivity is to have one AI (your bot) check things another AI (Claude Code) has done. They will optimise against each other and make suggestions and you can use those suggestions you feel are most worth it.