Fine-Tuning LLMs on Large Proprietary Codebases
discuss.huggingface.coI’m currently fine-tuning a large language model (LLM) on a proprietary codebase. The fine-tuning process itself has completed without technical issues, but the performance of the resulting model is very poor—its responses are largely irrelevant, even when asked questions that are directly taken from the training dataset. The objective of this fine-tuning effort is to enable the model to assist with tasks related to the private codebase, such as code generation, code explanation, and guidance o...