Markov-Chain Chatbot

The problem

I wanted a Discord bot that talked like the channel it lived in. The generation approach was deliberately old: an n-gram Markov chain trained on message history, no neural model involved. The interesting problems were not in the model. They were in running the thing inside an event loop without stalling it.

Approach

discord.py is built on asyncio, and everything runs on one loop. A blocking call in a message handler does not slow down one response, it stops the entire bot from receiving events. Every piece of the design followed from that constraint.

Training data came from channel history. The bot backfilled past messages and then learned incrementally from new ones. Both paths tokenize a message and update transition counts keyed by the preceding n-gram.

Storage was SQLite, accessed asynchronously. SQLite is a good fit for this workload: the state is a large table of transition counts with a read-heavy access pattern and a single writer, which is exactly what it handles well. It is a poor fit for concurrent writes, which is exactly what an event loop produces. Writes went through a queue. Handlers pushed work onto it and returned immediately, and a single consumer task drained the queue and batched updates into transactions. That gave me serialized writes without any handler ever waiting on the database, and batching cut the per-message transaction overhead, which dominated at the start.

Generation walked the chain from a seed state, sampling the next token by transition weight, with length caps and a stop condition to avoid the loops a Markov chain will happily fall into when a high-probability cycle exists.

What I found

The queue was the whole design. My first version awaited each write inline. It worked in a quiet channel and fell apart under any real traffic, with the bot visibly lagging behind the conversation. Moving to a producer-consumer split fixed the latency and, unexpectedly, made the failure behavior clearer: when the consumer fell behind, the queue depth showed it directly instead of the symptom appearing as vague slowness.

Model quality was what an n-gram model gets you. Low order produced word salad. Higher order produced fluent output that was often just a verbatim message from the training data, since with limited history most high-order states have exactly one successor. There is no setting that escapes this tradeoff, only a choice of which failure you prefer.

What I’d do differently

I did not think carefully enough about what the bot was allowed to learn from or repeat. A model trained on channel history will reproduce fragments of it, including things people would not want resurfaced. That deserved an explicit policy and a way to exclude content, and I built neither.

I would also separate the model from the transport. Chain logic and Discord handling were entangled enough that testing generation meant working around the client. The project is archived, but the queue pattern is the part I have reused since.