Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot reproduce result for coco #13

Open
KaiWU5 opened this issue Oct 26, 2022 · 4 comments
Open

Cannot reproduce result for coco #13

KaiWU5 opened this issue Oct 26, 2022 · 4 comments

Comments

@KaiWU5
Copy link

KaiWU5 commented Oct 26, 2022

Following readme construction, I only got AP 0.065 after training on Seed 1 Percent 1 Coco standard while 25.81 ± 0.28 (+16.76) is claimed in the paper. Here's my setting:

Dataset:
percent 1 seed 1 Coco standard

Pretrained baseline:
I use the train_gpu2.sh modification and only got AP 0.0030 less than 1%. I trained twice and AP 0.003 is repeatable.

Training:
I use train_gpu8.sh without modification and only got AP 0.065 less than 10%

My guess is that the pretrained baseline performs low. Could you share your model and log of pretrained baseline and semi-supevised training to help use reproduce results?

@AceCoooool
Copy link
Collaborator

We use the provided scripts to run all the experiments. Could you provide the training log to help us find the problem?

@KaiWU5
Copy link
Author

KaiWU5 commented Nov 29, 2022

Thanks for reply. The log is below. From the log, I used the same config you provided with changing the data path and pre-trained path to my location.
20221026_105431.log

@AceCoooool
Copy link
Collaborator

We use mmdet==2.10.0 in our experiments, and there may some difference in later version. We will find this out and update our code soon.

@wangshaishaiya
Copy link

Following readme construction, I only got AP 0.065 after training on Seed 1 Percent 1 Coco standard while 25.81 ± 0.28 (+16.76) is claimed in the paper. Here's my setting:

Dataset: percent 1 seed 1 Coco standard

Pretrained baseline: I use the train_gpu2.sh modification and only got AP 0.0030 less than 1%. I trained twice and AP 0.003 is repeatable.

Training: I use train_gpu8.sh without modification and only got AP 0.065 less than 10%

My guess is that the pretrained baseline performs low. Could you share your model and log of pretrained baseline and semi-supevised training to help use reproduce results?

你好,我也遇见了这个问题,尝试多次,确定是环境的问题,重新配置环境就可以啦,
Ubuntu==20.2.4 cuda==11.0 Python==3.6.9 torch=1.7.0 torchvision==0.8.1 mmdet==2.10.0 mmcv-full==1.2.7 seaborn==0.11.0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants