Do Prompt-Based Models Really Understand the Meaning of Their Prompts?
This repository accompanies our paper “Do Prompt-Based Models Really Understand the Meaning of Their Prompts?” Usage To replicate our results in Section 4, run: python3 prompt_tune.py –save-dir ../runs/prompt_tuned_sec4/ –prompt-path ../data/binary_NLI_prompts.csv –experiment-name sec4 –few-shots 3,5,10,20,30,50,100,250 –production –seeds 1 Add –fully-train if you want to train on the entire training set in addition to few-shot settings. To replicate Section 5, run: python3 prompt_tune.py –save-dir ../runs/prompt_tuned_sec5/ –prompt-path ../data/binary_NLI_prompts_permuted.csv
Read more