Wrong result.

[zengqg]

When I run
python main.py --train_pg

the reward or Avg.reward is negative. What' s wrong with it?

[huangjicun]

The calculation of reward is the score of the training model minus the score of the AI.
The model is weak at the beginning of the training, so the score will be less than the AI and cause the reward to be negative.
When the training model becomes stronger by training, reward and Avg.reward will become positive.

[JasonYao81000]

An episode ends when one of the players reach 21 points in pong.
Our reward is defined as:
Reward= N_win - N_lose
N_win is the number of wining rounds in one episode.
N_lose is the number of losing rounds in one episode.

When you run python main.py --train_pg, you will start to train a policy gradient model with "noob" weights.
So you will get N_win = 0 and N_lose = 21 at the begining.
Thus, the reward is -21, which is negative.