cdnutter
/
discollama
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Update README.md

pull/6/head
Michael Yang 2023-10-31 15:51:42 -07:00
parent e6ebff0883
commit 177526c93f
3 changed files with 16 additions and 40 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
Dockerfile
View File

@ -1,4 +1,4 @@
FROM python:3.11-alpine FROM python:3.11.6-alpine
RUN apk add --no-cache build-base libffi-dev RUN apk add --no-cache build-base libffi-dev
RUN pip install poetry RUN pip install poetry

50
README.md
View File

@ -4,62 +4,38 @@
## Dependencies ## Dependencies
- Redis - Docker and Docker Compose
```
python3 -m pip install poetry
poetry install
```
## Run `discollama.py` ## Run `discollama.py`
``` ```
poetry run python discollama.py DISCORD_TOKEN=xxxxx docker compose up
``` ```
_Note: You must setup a [Discord Bot](https://discord.com/developers/applications) and set environment variable `DISCORD_TOKEN` before `discollama.py` can access Discord._ > Note: You must setup a [Discord Bot](https://discord.com/developers/applications) and set environment variable `DISCORD_TOKEN` before `discollama.py` can access Discord.
`discollama.py` requires an [Ollama](https://github.com/jmorganca/ollama) server. Follow the steps in the [Ollama](https://github.com/jmorganca/ollama) repository to setup Ollama. `discollama.py` requires an [Ollama](https://github.com/jmorganca/ollama) server. Follow the steps in [jmorganca/ollama](https://github.com/jmorganca/ollama) repository to setup Ollama.
By default, it uses `127.0.0.1:11434` but this can be configured with command line parameters `--ollama-host` and `--ollama-port`. By default, it uses `127.0.0.1:11434` which can be overwritten with `OLLAMA_HOST`.
> Note: Deploying this on Linux requires updating network configurations and `OLLAMA_HOST`.
## Customize `discollama.py` ## Customize `discollama.py`
The default LLM is `llama2`. A custom personality can be added by changing the `SYSTEM` instruction in the Modelfile and running `ollama create`: The default LLM is `mike/discollama`. A custom personality can be added by changing the `SYSTEM` instruction in the Modelfile and running `ollama create`:
``` ```
ollama create discollama -f Modelfile ollama create mymodel -f Modelfile
``` ```
This is set in `discollama.py` through `--ollama-model`: This can be changed in `compose.yaml`:
``` ```
poetry run python discollama.py --ollama-model discollama environment:
- OLLAMA_MODEL=mymodel
``` ```
Additional LLM parameters can be set in the same Modelfile through `PARAMETER` instructions: See [jmorganca/ollama](https://github.com/jmorganca/ollama/blob/main/docs/modelfile.md) for more details.
```
FROM llama2
PARAMETER temperature 2
PARAMETER stop [INST]
PARAMETER stop [/INST]
PARAMETER stop <<SYS>>
PARAMETER stop <</SYS>>
```
If customizing the system prompt is not enough, you can configure the full prompt template:
```
FROM llama2
TEMPLATE """[INST] {{ if .First }}<<SYS>>{{ .System }}<</SYS>>
{{ end }} Tweet: 'I hate it when my phone battery dies.' [/INST] Sentiment: Negative </s><s>[INST] Tweet: 'My day has been 👍' [/INST] Sentiment: Positive </s><s> [INST] Tweet: 'This is the link to the article' [/INST] Sentiment: Neutral </s><s> [INST] Tweet: '{{ .Prompt }}' [/INST] Sentiment: """
```
This model replies with the sentiment of the the input prompt: positive, negative, or neutral.
## Activating the Bot ## Activating the Bot

4
compose.yaml
View File

@ -8,9 +8,9 @@ services:
command: ['--redis-host', 'redis'] command: ['--redis-host', 'redis']
environment: environment:
- DISCORD_TOKEN - DISCORD_TOKEN
- OLLAMA_HOST=host.docker.internal - OLLAMA_HOST
- OLLAMA_PORT=11434 - OLLAMA_PORT=11434
- OLLAMA_MODEL=discollama - OLLAMA_MODEL=mike/discollama
redis: redis:
image: redis/redis-stack-server:latest image: redis/redis-stack-server:latest