How to create and automate with AI with chat interface and nothing else - part II

AI in its current state is a wonderful tool for automation, but like with any other new tool, you have to know how to use it safely and efficiently.

You can use AI to automate your everyday tasks, with no coding, no deep knowledge of tech magic behind, with a simple access to one of the main LLMs through their chat interfaces. However, using it without basic understanding of its limitations might lead to repeating mistakes many of us have already made, and wasting your time unnecessarily.

There are certain AI limitations you have to be aware of:

• context window limitation,

• bias to comply and flatter,

• lack of strict rules to follow.

Let’s dive in, explore what it means, and how to overcome the challenges it brings. In the previous article, we covered the context window limitation. Today, we will cover:

Bias to comply and flatter

The LLMs are trained on user data, including the feedback on whether users liked it or not. Disliked answers get diminished in the learning process, liked answers have a higher value in the data set. Turns out, we don’t like that much to be disagreed with. We like when someone tells us: “That’s an excellent idea, Bob! I wish I had thought of pouring soup in my shoe before you did!” Therefore, the current state of AI reflects not only our knowledge, but also our biases, sense of self-worth and tendency to accept opinions that confirm what we already think ourselves. Results? AI does exactly that. It will compliment you on the very basic ideas like they are brand new, it will follow an inefficient way if you suggest it, because it doesn’t want to hurt your feelings.

How to handle it?

Be your own critique and teach AI to do the same. Present your ideas before you prompt the AI to follow your instructions, offer them for the LLM to judge as it’s your fiercest opponent, let it suggest the workflow, and judge yourself what suits you the most.

Example: You want to use AI to test your board-game system. You think you will copy-paste the rules in the chat, and then you will go through them with AI, tweaking them, as you go, based on its feedback. However, this workflow means the rules will get partially lost eventually, as the conversation becomes too long. The AI will struggle to keep on track with all the changes. The AI won’t tell you any of this. It will comply, compliment the work you have done, and try its best to fill in the gaps in the rules, assuming instead of knowing. Instead of this, create an agent prompted to unapologetically dig out problems in your game system. Upload the rules as one file, and search for potential problems in the various sections one by one, using the branch conversation method described above. As your first prompt, you will want to use something like “I want you to be a fearless tester of my new board-game. Don’t go soft on me, point out any flaws, misconceptions and gaps you find in the rules. First, tell me what you think about it as a whole, then we will discuss each issue separately. These are the rules, tell me all the pros and cons for using them in a boardgame.”

We will cover the remaining topic in an upcoming article. To make sure you don’t miss it, you can subscribe for receiving our newsletter and receive a free PDF with advice on AI automation here.