I meant create a mini model first but I guess training an existing one can be good too
Yeah, that’s where LoRA comes in. That would be a way easier thing to work on.
That’s a fine tuning technique right anyways maybe I’ll start these after I finish the current projects
1 Like
Yeah, it’s a fine-tuning technique. It stands for Low-Rank Adaptation. The idea is instead of changing the entire model, you take a small portion of it and fine-tune that, and then you parse the rest of the model through it.