rihal-llm-suite / README.md

Rihal LLM Suite

Last updated: 4/16/2026GitHubSiraaj

Rihal LLM Suite

Rihal LLM suite contains the following services:

  1. doc-connector: DocumentProcessorActivity temporal activity to parse and extract content from documents.
  2. web-connector: ScrapingActivity temporal activity to parse and extract content from websites URLs.
  3. rag: Contains the following activities:
    1. EmbeddingToQdrantActivity temporal activity to chunk the content and embed it to qdrant.
    2. ContextRetrievalActivity temporal activity to retrieve the context from qdrant.
  4. llm: POST /generate endpoint to generate the LLM response to user query.
  5. ocr: POST /ocr endpoint to extract text from image.
  6. web: react web application for rihal-llm-suite.
  7. server: Go server for rihal-llm-suite.
  8. postgres: Postgres database for rihal-llm-suite.
  9. qdrant: Qdrant vector database for rihal-llm-suite.
  10. keycloak: Keycloak for rihal-llm-suite.

Running the project

  1. Clone the repository
  2. Run make run in the root directory of the project.
  3. To stop the project, run make stop in the root directory of the project.
  4. To clean the project, run make rm in the root directory of the project.

Users

  1. User: admin Password: 123
  2. User: user Password: 123

Troubleshooting

  1. If you are not working from the office, you need to update the OLLAMA_BASE_URL in the .env file to the correct URL.
OLLAMA_BASE_URL="https://ollama-api.rihal.solutions/"
OLLAMA_MODEL=llama3.1:70b
WHISPER_POD_URL="whisper-api.rihal.solutions"
  1. To run Bedrock, you need to have the following environment variables in the .env file.
BEDROCK_OLLAMA_MODEL_ID=us.meta.llama3-1-70b-instruct-v1:0
BEDROCK_EMBEDDING_MODEL_ID=amazon.titan-embed-text-v2:0
AWS_REGION=us-east-1
AWS_ACCESS_KEY_ID=<your AWS_ACCESS_KEY_ID>
AWS_SECRET_ACCESS_KEY=<your AWS_SECRET_ACCESS_KEY>

You can get the AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY from your AWS account.

Note: You have to request access to the us.meta.llama3-1-70b-instruct-v1:0 and amazon.titan-embed-text-v2:0 models from bedrock dashboard.

CI

If you don't have black configured in your IDE, you can run the following commands to format/lint the code.

Formatting

find . -path "./web" -prune -o -path "./rihal-llm" -prune -o -name "*.py" -print0 | xargs -0 black

Linting

find . -path "./web" -prune -o -path "./rihal-llm" -prune -o -name "*.py" -print0 | xargs -0 flake8 --ignore=E203,W503,E501