A Demo of TinyTroupe agents working with Ollama and Qwen2.5

When I found the TinyTroupe repository from Microsoft, it was exciting to think of the potential use cases that could stem from the ability to have multiple simulated people interacting together. My first thought in regards to this was how it could be used to bring NPCs in games to life, allowing genuine interaction and even memory of that interaction on the part of the NPCs. Being myself, I also thought about how funny it would be to make two of them argue over something that I found amusing.

When I first tested TinyTroupe, I was quickly brought back down to reality about implementing one of the funny scenarios floating around in my head. I don't say this because the repository didn't work well, but rather, because the only option to use it was with the OpenAI API, and due to the amount of "interacting" these simulated people were doing, the API costs could quickly get out of hand if you were not careful with it.

Before deciding to go ahead and get it working with a local LLM, I did some poking around the codebase, and I was extremely impressed with the way that these simulated people, dubbed "Tinyperson", were created. They each had very detailed characteristics, ages, jobs, routines, and even relationships. The potential of this to simulate actual people was more potent than I had first expected it to be.

Once I saw how detailed these tiny people could be, I got to work implementing Ollama so that I could have these tiny people engage in amusing banter without having to worry about a large bill from OAI after the fact. It took a bit of time and experimenting to get the repo working with Ollama, with specific regards to simplifying the tinyperson.mustache prompt so that a simpler local model could handle it and return the responses in the correctly structured JSON format needed to work. I also made a very simple test script that would place two tinypeople inside a chatroom and give one of them a prompt which they would use to base the conversation on.

In the initial testing, I was facing a continued "cognitive_state" keyerror, which I saw others attempting Ollama integration experience as well. I lowered the temperature from 0.7 to 0.1 which helped a lot in response format consistency. I also found that I had to send the entire prompt to the model as a single string, as when I was trying to send it in a more structured format, the model was returning incorrectly formatted responses. I believe there is a way around this but I have not tested further since my goal of a simple local fork was functioning well enough.

I decided to use Qwen2.5-32B for running TinyTroupe locally, and I have been very satisfied with its ability to handle the interactions contained in my simple test script. I have to admit that I would like it to be a bit less censored, but I was able to have two of the tinypeople converse about a rumor one of them was spreading, which I found a bit funny. I am going to attempt to use it with a variant of the Dolphin Mixtral model, which could end up being highly entertaining, assuming the model returns the correctly formatted responses.

Overall, this repo is a fascinating look into the power that LLMs have to roleplay and simulate human behavior and actions in a social context. Though you may not be able to closely mimic someone using a local model, the abilities and complexity of prompts and personas that large models like GPT-4o can handle, allows extremely in-depth simulations and can provide real-world business use cases and insights as demonstrated in the original repo's examples.

To view the video demonstration of this, see here: Run AI Simulated People Locally with Ollama! (Qwen2.5 & TinyTroupe)

For the GitHub Repo with these changes, see here: TinyTroupeOllama Github Repository