Nations

For counting the number of ground errors ComplEx makes when training on Nations:

$ ./bin/adv-cli.py --train data/nations/nations_ternary.tsv --lr 0.1 --model ComplEx --similarity dot --margin 1 --embedding-size 100 --nb-epochs 1000 --clauses data/nations/clauses/commonbloc2_transitivity.pl --adv-lr 0.1 --adv-ground-samples 1000 --adv-ground-tol 0.1 --adv-init-ground --predicate-norm 1 --adv-weight 0 2>&1 | grep Sum
INFO:adv-cli.py:Epoch: 1	Sum of Zero-One Errors: 1
INFO:adv-cli.py:Epoch: 2	Sum of Zero-One Errors: 6
INFO:adv-cli.py:Epoch: 3	Sum of Zero-One Errors: 23
INFO:adv-cli.py:Epoch: 4	Sum of Zero-One Errors: 48
INFO:adv-cli.py:Epoch: 5	Sum of Zero-One Errors: 43
INFO:adv-cli.py:Epoch: 6	Sum of Zero-One Errors: 45
INFO:adv-cli.py:Epoch: 7	Sum of Zero-One Errors: 83
INFO:adv-cli.py:Epoch: 8	Sum of Zero-One Errors: 87
INFO:adv-cli.py:Epoch: 9	Sum of Zero-One Errors: 110
INFO:adv-cli.py:Epoch: 10	Sum of Zero-One Errors: 105
INFO:adv-cli.py:Epoch: 11	Sum of Zero-One Errors: 119
INFO:adv-cli.py:Epoch: 12	Sum of Zero-One Errors: 102
INFO:adv-cli.py:Epoch: 13	Sum of Zero-One Errors: 90
INFO:adv-cli.py:Epoch: 14	Sum of Zero-One Errors: 84
INFO:adv-cli.py:Epoch: 15	Sum of Zero-One Errors: 71
INFO:adv-cli.py:Epoch: 16	Sum of Zero-One Errors: 93
INFO:adv-cli.py:Epoch: 17	Sum of Zero-One Errors: 150
INFO:adv-cli.py:Epoch: 18	Sum of Zero-One Errors: 146
INFO:adv-cli.py:Epoch: 19	Sum of Zero-One Errors: 169
INFO:adv-cli.py:Epoch: 20	Sum of Zero-One Errors: 167
INFO:adv-cli.py:Epoch: 21	Sum of Zero-One Errors: 167
INFO:adv-cli.py:Epoch: 22	Sum of Zero-One Errors: 148
INFO:adv-cli.py:Epoch: 23	Sum of Zero-One Errors: 142
INFO:adv-cli.py:Epoch: 24	Sum of Zero-One Errors: 139
INFO:adv-cli.py:Epoch: 25	Sum of Zero-One Errors: 119
INFO:adv-cli.py:Epoch: 26	Sum of Zero-One Errors: 107
INFO:adv-cli.py:Epoch: 27	Sum of Zero-One Errors: 104
INFO:adv-cli.py:Epoch: 28	Sum of Zero-One Errors: 103
INFO:adv-cli.py:Epoch: 29	Sum of Zero-One Errors: 96
INFO:adv-cli.py:Epoch: 30	Sum of Zero-One Errors: 90

Results obtained using the method in [1] (NAACL):

$ ./bin/adv-cli.py --train data/nations/nations_ternary.tsv --lr 0.1 --model ComplEx --similarity dot --margin 1 --embedding-size 100 --nb-epochs 1000 --clauses data/nations/clauses/commonbloc2_transitivity.pl --adv-lr 0.1 --adv-ground-samples 1000 --adv-ground-tol 0.1 --adv-init-ground --predicate-norm 1 --adv-weight 1000 --adversary-epochs 0 2>&1 | grep Sum
INFO:adv-cli.py:Epoch: 1	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 2	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 3	Sum of Zero-One Errors: 2
INFO:adv-cli.py:Epoch: 4	Sum of Zero-One Errors: 10
INFO:adv-cli.py:Epoch: 5	Sum of Zero-One Errors: 19
INFO:adv-cli.py:Epoch: 6	Sum of Zero-One Errors: 4
INFO:adv-cli.py:Epoch: 7	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 8	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 9	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 10	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 11	Sum of Zero-One Errors: 29
INFO:adv-cli.py:Epoch: 12	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 13	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 14	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 15	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 16	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 17	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 18	Sum of Zero-One Errors: 1
INFO:adv-cli.py:Epoch: 19	Sum of Zero-One Errors: 10
INFO:adv-cli.py:Epoch: 20	Sum of Zero-One Errors: 15
INFO:adv-cli.py:Epoch: 21	Sum of Zero-One Errors: 31
INFO:adv-cli.py:Epoch: 22	Sum of Zero-One Errors: 38
INFO:adv-cli.py:Epoch: 23	Sum of Zero-One Errors: 5
INFO:adv-cli.py:Epoch: 24	Sum of Zero-One Errors: 4
INFO:adv-cli.py:Epoch: 25	Sum of Zero-One Errors: 4
INFO:adv-cli.py:Epoch: 26	Sum of Zero-One Errors: 2
INFO:adv-cli.py:Epoch: 27	Sum of Zero-One Errors: 3
INFO:adv-cli.py:Epoch: 28	Sum of Zero-One Errors: 3
INFO:adv-cli.py:Epoch: 29	Sum of Zero-One Errors: 4
INFO:adv-cli.py:Epoch: 30	Sum of Zero-One Errors: 4

Results obtained using adversarial training for fine-tuning the violating embeddings:

$ ./bin/adv-cli.py --train data/nations/nations_ternary.tsv --lr 0.1 --model ComplEx --similarity dot --margin 1 --embedding-size 100 --nb-epochs 1000 --clauses data/nations/clauses/commonbloc2_transitivity.pl --adv-lr 0.1 --adv-ground-samples 1000 --adv-ground-tol 0.1 --adv-init-ground --predicate-norm 1 --adv-weight 1000 --adversary-epochs 10 2>&1 | grep Sum
INFO:adv-cli.py:Epoch: 1	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 2	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 3	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 4	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 5	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 6	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 7	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 8	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 9	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 10	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 11	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 12	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 13	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 14	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 15	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 16	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 17	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 18	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 19	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 20	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 21	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 22	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 23	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 24	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 25	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 26	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 27	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 28	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 29	Sum of Zero-One Errors: 0
INFO:adv-cli.py:Epoch: 30	Sum of Zero-One Errors: 0

[1] https://rockt.github.io/pdf/rocktaschel2015injecting.pdf