discollama/README.md

43 lines
1.4 KiB
Markdown
Raw Normal View History

2023-07-28 20:00:40 -07:00
# discollama
2023-10-09 16:59:22 -07:00
`discollama` is a Discord bot powered by a local large language model backed by [Ollama](https://github.com/jmorganca/ollama).
2023-07-30 14:29:29 -07:00
2023-07-28 20:00:40 -07:00
## Dependencies
2023-10-31 15:51:42 -07:00
- Docker and Docker Compose
2023-07-28 20:00:40 -07:00
## Run `discollama.py`
```
2023-10-31 15:51:42 -07:00
DISCORD_TOKEN=xxxxx docker compose up
2023-07-28 20:00:40 -07:00
```
2023-10-31 15:51:42 -07:00
> Note: You must setup a [Discord Bot](https://discord.com/developers/applications) and set environment variable `DISCORD_TOKEN` before `discollama.py` can access Discord.
2023-07-30 14:29:29 -07:00
2023-10-31 15:51:42 -07:00
`discollama.py` requires an [Ollama](https://github.com/jmorganca/ollama) server. Follow the steps in [jmorganca/ollama](https://github.com/jmorganca/ollama) repository to setup Ollama.
2023-07-30 14:29:29 -07:00
2023-10-31 15:51:42 -07:00
By default, it uses `127.0.0.1:11434` which can be overwritten with `OLLAMA_HOST`.
2023-07-30 14:29:29 -07:00
2023-10-31 15:51:42 -07:00
> Note: Deploying this on Linux requires updating network configurations and `OLLAMA_HOST`.
2023-07-30 14:29:29 -07:00
2023-10-31 15:51:42 -07:00
## Customize `discollama.py`
2023-07-30 14:29:29 -07:00
2023-10-31 15:51:42 -07:00
The default LLM is `mike/discollama`. A custom personality can be added by changing the `SYSTEM` instruction in the Modelfile and running `ollama create`:
2023-07-30 14:29:29 -07:00
```
2023-10-31 15:51:42 -07:00
ollama create mymodel -f Modelfile
2023-07-30 14:29:29 -07:00
```
2023-10-31 15:51:42 -07:00
This can be changed in `compose.yaml`:
2023-07-30 14:29:29 -07:00
```
2023-10-31 15:51:42 -07:00
environment:
- OLLAMA_MODEL=mymodel
2023-07-30 14:29:29 -07:00
```
2023-07-28 20:00:40 -07:00
2023-10-31 15:51:42 -07:00
See [jmorganca/ollama](https://github.com/jmorganca/ollama/blob/main/docs/modelfile.md) for more details.
2023-08-07 08:37:39 -07:00
## Activating the Bot
Discord users can interact with the bot by mentioning it in a message to start a new conversation or in a reply to a previous response to continue an ongoing conversation.