The Truth About ChatGPT Agent
Plus 6 case studies
Welcome to The School of Knowledge and this week’s paid essay. Each week, I send an essay to help you navigate your personal or professional transition, from those who have tried, failed and succeeded—those with skin in the game. If you want support on how to implement the mental models, frameworks, and systems, take part in Q&As and have access to our private chat, consider becoming a paid member.
OpenAI have recently released ChatGPT 'Agent Mode,' an autonomous operator with deep research capabilities that can 'do things for you.' Depending on which side of the fence you sit on—grossed out and tired by the increasing awareness of machine-produced content—or excited by the prospect of being more efficient, ChatGPT's Agent Mode (and its current limitations) should offer solace for those concerned about acquiring what the £20 pro plan can offer—gained time.
Currently, at least in the UK, I have 40 uses of Agent Mode per month. At first this appeared to be a limitation, but it's made me realise something profound: you can't sit there and use it all day, every day. When you have infinite opportunities, mediocre results lose their appeal. Warren Buffett knew better. If investors are limited to only 20 investment decisions in their lifetime, suddenly every decision demands excellence. The same can be said about Agent Mode.
People are confusing AI with 'doing the thinking' on the part of the human, but it should be considered a representation of how well you are at communicating. If you're getting lousy outcomes, is it because AI is slop, or is it because you haven't been able to communicate your desired properly? AI exposes bad thinking.
The same logic applies to ChatGPT's Agent Mode. It can't all of a sudden make you 10X more efficient if it doesn't understand what you mean to begin with. Prompting is clarity of thought translated into text.
Over the last week I've tested various ways to use Agent Mode as a business owner and writer on Substack. Some saved hours. Some were flops—but they all reminded me that I needed to better my thinking to achieve the outcome I desired.
Here are 6 personal case studies from using Agent Mode this week:
Limitations, Frustrations and When Not to Use Agent Mode
Booking Simple Reservations
I severely procrastinate at mundane jobs I have no interest in doing, no matter how easy they are. A symptom of ADHD, which I have. So, the thought of having a little helper do all the things I don't like doing sounded like music to my ears. Here's the prompt I used:
Find me a reservation for 2 people at the restaurant 'Sparrows' in Manchester, UK between 18:00 and 20:00 on a Friday or Saturday. Search for August and September.
It failed spectacularly. Sparrows is undergoing work in August, which Agent Mode recognised, but it also claimed it was closed for all of September, which it wasn't. I also wanted to see how fast Agent Mode was, so decided to manually reserve a table myself. It took about three times longer than me to not find me a table.
Emphatically, Humans 1 - 0 Agent Mode.
E-commerce Shopping
Next I needed to buy some work jeans. Here's my prompt:
I need a pair of work jeans and prefer the colour blue. They need to be hard wearing because of my job in construction, and I currently wear Levi's which I find comfortable and fit well. I'm a size 32/34 and prefer tapered jeans. Please search the web, including Amazon for a sturdy pair of jeans I can use for work under £50. They must be well reviewed.
I chose Amazon because I'd seen Agent Mode autonomously buy things for people, but I ran into significant problems. Amazon blocked Agent Mode. Whether this is due to privacy restraints in the UK or something on my end I haven't yet worked out, but it did provide a report with links to jeans within the scope of what I asked for. I'm giving it the nod here for being intuitive.
Humans 1 - 1 Agent Mode.
Booking Flights, Hotels and Tickets
Next up I asked Agent Mode to organise a day trip to Dublin for me and the two other company owners. Here's the prompt:
Find me flights from Manchester to Dublin, Ireland on the 22nd November this year for 3 people. The flight should be in the morning. Also, find us accommodation and make sure the hotel states 3 separate beds. Use booking.com or Airbnb to find a room or apartment. Keep flights and hotel/apartment under £350. Also try to find 3 tickets for the Ireland vs South Africa rugby game.
The flights it found were £14.99 and were great times. To our dismay we realised they were for a date in August! This was easy enough to change, but a massive red flag that you should always have the human sign off before purchasing anything. It did however find a suitable apartment and tickets to the game, although they were sold out when we tried to purchase them. A great effort but I wouldn't trust Agent Mode with my wallet just yet.
Humans 2 - 1 Agent Mode.
The rest of this essay is for members only. You’ve seen where Agent Mode falls down. Here’s where it shines — with the exact prompts, templates, and QA checklists I used to save a full day this week. Copy them, run them, and keep the human in the loop where it matters.