Automatic Bi-modal Question Title Generation for Stack Overflow with Prompt Learning

Paper

📜 Automatic Bi-modal Question Title Generation for Stack Overflow with Prompt Learning

@article{yang2024automatic,
  title={Automatic bi-modal question title generation for Stack Overflow with prompt learning},
  author={Yang, Shaoyu and Chen, Xiang and Liu, Ke and Yang, Guang and Yu, Chi},
  journal={Empirical Software Engineering},
  volume={29},
  number={3},
  pages={63},
  year={2024},
  publisher={Springer}
}

Introduction

The framework of our proposed approach SOTitle+

Corpus & Model

We publish our dataset and trained model on Zenodo.

Experimental Replication Tutorials

In ./model_model, We shared the script to replicate the experimental data in our paper

Replication step

1. Clone the repo

   git clone https://github.com/shaoyuyoung/SOTitlePlus.git

2. Mkdir a data catalogue in the root directory and download the datasets in data catalogue.
3. Make sure your version of python is python3.9 (Due to compatibility issue with the nlg-eval library, we do not support python3.10 or later)
4. Install the dependencies according to the requirements file

   pip install requirements.txt

5. Training and evaluating the model (fine-tuning and prompt-tuning in this phase)

   python model_code/main.py 

6. Calculating the metrics

   python results/metrics.py

   If you have any questions on replication, please feel free to report in the issue🤗

Results

In ./results, run metrics.py to calculate ROUGE, METEOR, BLEU and CIDEr

SOQTG of ChatGPT

If you want to use ChatGPT to generate Stack Overflow question titles, we share scripts in ./run_ChatGPT_API.

You need to put your APIKEY and design your own prompt. We keep the original prompts from our experiment in the script.

Tool and Demo

We developed a browser plugin based on SOTitle and integrated it into the Chrome browser. Instruction for use:

1. Download and install the plugin from the SOTitlePlusPlugin folder.
2. Enter this website: https://stackoverflow.com/questions/ask
3. After you provide the problem description and code snippet, press Ctrl + Q to generate candidate titles.

We provide a demo video on youtube:https://www.youtube.com/watch?v=_KgUISAT74M