llm-twin-course

This project is a training course designed to learn the architecture and application of a LLM and RAG system ready to manufacture from end to end. It consists of four Python microprocessors that include data collection, feature extraction, model training, and inference stages. Throughout the course, MLOps and LLMOps best practices are used to process, train and distribute data.

LL

decodingai-magazine · GitOps & Infrastructure as Code

STARS4.4K
FORKS732
COMMITS0
CONTRIBUTORS0
VERSION—
UPDATED6 mo ago
PythonLLMRAGLLMOpsMIT

Common commercial products in this category: HCP Terraform, Spacelift, env0

What can it do?
  • You collect digital data on different platforms with ETL pipelines and transfer them to the NoSQL database.
  • You do feature extraction by clearing data through flow pipelines and uploading it to the vector database.
  • With LoRA or QLoRA methods, you train a language model and then version it on the Hugging Face model recorder.
Heads up
  • • No published release
User reviews
  • • NeutralGitHub Issue

    Where should I start with this code at the Course1

    “Thank you for your selfless sharing! I hope this message finds you well. I am new to this project and very excited about contributing to it. I have gone through the readme but find myself a bit lost on, where to begin with running course1. I apologize if this information is…”

  • • NeutralGitHub Issue

    Getting issue with docker logs

    “{"statusCode": 200, "body": "Link processed successfully"}% (llm-twin) goutamgupta@goutamguptas-MacBook-Pro llm-twin-course % docker logs llm-twin-bytewax 2024-08-07 10:04:59 [info ] Couldn't access the collection. Creating a new one... cls=data_flow.stream_output…”

GitHub

My note

Alternative pages

More in: GitOps & Infrastructure as Code