Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Identify equivalent states in MCTS #975

Open
GuoHongkai-125 opened this issue Aug 20, 2024 · 0 comments
Open

Identify equivalent states in MCTS #975

GuoHongkai-125 opened this issue Aug 20, 2024 · 0 comments

Comments

@GuoHongkai-125
Copy link

I read the paper Adversarial Policies Beat Superhuman Go AIs (T. Wang et al, 2023) and test b28-s6797 recently. b28 successfully handled the cyclic groups in that paper, but still failed to evaluated static state containing large cyclic groups across more than half of 19*19 board after >100k visits. I know that these states barely appear in real games. However, I think that the patch against cyclic group problem may be only some data augmentation. It didn't make KataGo aware of liberties or other concepts that are simple for human players.

test.zip

According to the "最佳选点" visualization in Katrain, I think that KataGo's main flaw is viewing equivalent moves as different ones. It searches very broad space with shallow depth when many equivalent moves exist. In the example of d.png in zip file, it distributes visits on several dozen of moves with several k visits. Human players understand the concept of liberties by merging equivalent states and compressing the state space to only one integer, but KataGo appears not in its searching pattern. By the way, the board state with same stone positions and different orders can be merged in MCTS, but I haven't find an elegant implementation.

I think that identifying equivalent states may reduce searching cost and enhance robustness in large group battles, since the essence of many battles is liberties. But equivalent moves may have different P and V because of the random noises in NN training, thus I don't know how to implement the identification now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant