|
|
@@ -43,7 +43,7 @@
|
|
|
"source": [
|
|
|
"Suppose we have the following data entry:\n",
|
|
|
"\n",
|
|
|
- "```\n",
|
|
|
+ "```json\n",
|
|
|
"{\n",
|
|
|
" \"instruction\": \"Identify the correct spelling of the following word.\",\n",
|
|
|
" \"input\": \"Ocassion\",\n",
|
|
|
@@ -195,24 +195,24 @@
|
|
|
"metadata": {},
|
|
|
"source": [
|
|
|
"```python\n",
|
|
|
- " for i, entry in tqdm(enumerate(test_data), total=len(test_data)):\n",
|
|
|
+ "for i, entry in tqdm(enumerate(test_data), total=len(test_data)):\n",
|
|
|
"\n",
|
|
|
- " input_text = format_input(entry)\n",
|
|
|
- " tokenizer=tokenizer\n",
|
|
|
+ " input_text = format_input(entry)\n",
|
|
|
+ " tokenizer=tokenizer\n",
|
|
|
"\n",
|
|
|
- " token_ids = generate(\n",
|
|
|
- " model=model,\n",
|
|
|
- " idx=text_to_token_ids(input_text, tokenizer).to(device),\n",
|
|
|
- " max_new_tokens=256,\n",
|
|
|
- " context_size=BASE_CONFIG[\"context_length\"],\n",
|
|
|
- " eos_id=50256\n",
|
|
|
- " )\n",
|
|
|
- " generated_text = token_ids_to_text(token_ids, tokenizer)\n",
|
|
|
+ " token_ids = generate(\n",
|
|
|
+ " model=model,\n",
|
|
|
+ " idx=text_to_token_ids(input_text, tokenizer).to(device),\n",
|
|
|
+ " max_new_tokens=256,\n",
|
|
|
+ " context_size=BASE_CONFIG[\"context_length\"],\n",
|
|
|
+ " eos_id=50256\n",
|
|
|
+ " )\n",
|
|
|
+ " generated_text = token_ids_to_text(token_ids, tokenizer)\n",
|
|
|
"\n",
|
|
|
- " # New: Adjust ###Response -> <|assistant|>\n",
|
|
|
- " response_text = generated_text[len(input_text):].replace(\"<|assistant|>:\", \"\").strip()\n",
|
|
|
+ " # New: Adjust ###Response -> <|assistant|>\n",
|
|
|
+ " response_text = generated_text[len(input_text):].replace(\"<|assistant|>:\", \"\").strip()\n",
|
|
|
"\n",
|
|
|
- " test_data[i][\"model_response\"] = response_text\n",
|
|
|
+ " test_data[i][\"model_response\"] = response_text\n",
|
|
|
"```"
|
|
|
]
|
|
|
},
|
|
|
@@ -229,7 +229,7 @@
|
|
|
"id": "dd8158e9-cc70-4e0f-88b0-73c3e1d8c030",
|
|
|
"metadata": {},
|
|
|
"source": [
|
|
|
- "```python\n",
|
|
|
+ "```bash\n",
|
|
|
"python exercise_experiments.py --exercise_solution phi3_prompt\n",
|
|
|
"```\n",
|
|
|
"\n",
|
|
|
@@ -273,7 +273,7 @@
|
|
|
"\n",
|
|
|
"Let's take a look at some of the responses to make sure they have been formatted correctly:\n",
|
|
|
"\n",
|
|
|
- "```\n",
|
|
|
+ "```json\n",
|
|
|
" {\n",
|
|
|
" \"instruction\": \"Rewrite the sentence using a simile.\",\n",
|
|
|
" \"input\": \"The car is very fast.\",\n",
|
|
|
@@ -296,7 +296,7 @@
|
|
|
"\n",
|
|
|
"We can evaluate the performance using the Ollama Llama 3 method, which is for your convenience, also implemented in the `python exercise_experiments.py` script, which we can run as follows:\n",
|
|
|
"\n",
|
|
|
- "```python\n",
|
|
|
+ "```bash\n",
|
|
|
"python ollama_evaluate.py --file_path instruction-data-with-response-phi3-prompt.json\n",
|
|
|
"```\n",
|
|
|
"\n",
|
|
|
@@ -309,7 +309,7 @@
|
|
|
"Average score: 48.87\n",
|
|
|
"```\n",
|
|
|
"\n",
|
|
|
- "The score is close to 50, which is in the same ballpark as the score we previously achieved with the Alpaca-style prompts.\n"
|
|
|
+ "The score is close to 50, which is in the same ballpark as the score we previously achieved with the Alpaca-style prompts."
|
|
|
]
|
|
|
},
|
|
|
{
|
|
|
@@ -802,7 +802,7 @@
|
|
|
"id": "be9ab66f-5819-4b01-9a03-c45aa3b7c5b8",
|
|
|
"metadata": {},
|
|
|
"source": [
|
|
|
- "```\n",
|
|
|
+ "```json\n",
|
|
|
"[\n",
|
|
|
" {\n",
|
|
|
" \"instruction\": \"Edit the following sentence to increase readability: \\\"He made a huge effort and was so successful.\\\"\",\n",
|
|
|
@@ -832,7 +832,7 @@
|
|
|
"source": [
|
|
|
"Finally, we can evaluate the finetuned LLM using the [ollama_evaluate.py](ollama_evaluate.py) utility function:\n",
|
|
|
"\n",
|
|
|
- "```\n",
|
|
|
+ "```bash\n",
|
|
|
"python ollama_evaluate.py --file_path instruction-data-with-response-alpaca52k.json\n",
|
|
|
"```\n",
|
|
|
"\n",
|
|
|
@@ -867,7 +867,7 @@
|
|
|
"To instruction finetune the model using LoRA, use the relevant classes and functions from appendix E:\n",
|
|
|
"\n",
|
|
|
"```python\n",
|
|
|
- " from appendix_E import LoRALayer, LinearWithLoRA, replace_linear_with_lora\n",
|
|
|
+ "from appendix_E import LoRALayer, LinearWithLoRA, replace_linear_with_lora\n",
|
|
|
"```"
|
|
|
]
|
|
|
},
|
|
|
@@ -961,7 +961,7 @@
|
|
|
"\n",
|
|
|
"We can evaluate the performance using the Ollama Llama 3 method, which is for your convenience, also implemented in the `python exercise_experiments.py` script, which we can run as follows:\n",
|
|
|
"\n",
|
|
|
- "```python\n",
|
|
|
+ "```bash\n",
|
|
|
"python ollama_evaluate.py --file_path instruction-data-with-response-lora.json\n",
|
|
|
"```\n",
|
|
|
"\n",
|
|
|
@@ -994,7 +994,7 @@
|
|
|
"name": "python",
|
|
|
"nbconvert_exporter": "python",
|
|
|
"pygments_lexer": "ipython3",
|
|
|
- "version": "3.11.4"
|
|
|
+ "version": "3.10.11"
|
|
|
}
|
|
|
},
|
|
|
"nbformat": 4,
|
Daniel Kleine