skip-gram meaning

skip-gram meaning

by Jean-Cédric Chappelier -
Number of replies: 0

I just checked: "skip-gram" only refers to the context, not to the negative samples:

  • it's the context itself which is named a "skip-gram" as opposed to a usual n-gram, because the context is made of two n-grams, on each side; pretty much like a (2n+1)-gram on which you "skipped" the word in the middle ;

  • I personally associated the word a bit further to the method, to negative sampling, understanding it as those n-grams ((2n+1)-grams, actually) you have to "skip" because they are wrong. I went too far, it seems... ;-)
Anyway, CBoW is also making use of "skipgrams", whatever they mean and thus the terminology is still confusing...