Parth Asawa, Ayushi Batwara, Jason Ding, Darren Teh
Paper is available here. Public version of the finetuning-final Google Colab here.
The task we had was fine-tuning GPT-2 on Sherlock Holmes’ dialogue with the goal of generating output text in response to prompts in the style of the fictional character Sherlock Holmes. Holmes' dialogue is known to be distinct from traditional mainstream English in its punctuation, aloofness, elevated tone, terminology, complex sentences, parallel structure, and more. Among the methods we tested — including LoRA, LoRA with context, prompt tuning, and others — we discovered that employing LoRA with context yielded the best results. This conclusion is supported by the validation loss and the quality of text generated by the model, as outlined in the report. We additionally designed and explored custom evaluation and training loss function methods for style transfer.
