Package edu.berkeley.nlp.lm
Class ContextEncodedProbBackoffLm<W>
- java.lang.Object
-
- edu.berkeley.nlp.lm.AbstractNgramLanguageModel<W>
-
- edu.berkeley.nlp.lm.AbstractContextEncodedNgramLanguageModel<W>
-
- edu.berkeley.nlp.lm.ContextEncodedProbBackoffLm<W>
-
- Type Parameters:
W
-
- All Implemented Interfaces:
ContextEncodedNgramLanguageModel<W>
,NgramLanguageModel<W>
,java.io.Serializable
public class ContextEncodedProbBackoffLm<W> extends AbstractContextEncodedNgramLanguageModel<W> implements ContextEncodedNgramLanguageModel<W>, java.io.Serializable
Language model implementation which uses Kneser-Ney style backoff computation.- Author:
- adampauls
- See Also:
- Serialized Form
-
-
Nested Class Summary
-
Nested classes/interfaces inherited from interface edu.berkeley.nlp.lm.ContextEncodedNgramLanguageModel
ContextEncodedNgramLanguageModel.DefaultImplementations, ContextEncodedNgramLanguageModel.LmContextInfo
-
Nested classes/interfaces inherited from interface edu.berkeley.nlp.lm.NgramLanguageModel
NgramLanguageModel.StaticMethods
-
-
Field Summary
-
Fields inherited from class edu.berkeley.nlp.lm.AbstractNgramLanguageModel
lmOrder, oovWordLogProb
-
-
Constructor Summary
Constructors Constructor Description ContextEncodedProbBackoffLm(int lmOrder, WordIndexer<W> wordIndexer, ContextEncodedNgramMap<ProbBackoffPair> map, ConfigOptions opts)
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description float
getLogProb(long contextOffset, int contextOrder, int word, ContextEncodedNgramLanguageModel.LmContextInfo outputContext)
Get the score for an n-gram, and also get the context offset of the n-gram's suffix.int[]
getNgramForOffset(long contextOffset, int contextOrder, int word)
Gets the n-gram referred to by a context-encoding.NgramMap<ProbBackoffPair>
getNgramMap()
ContextEncodedNgramLanguageModel.LmContextInfo
getOffsetForNgram(int[] ngram, int startPos, int endPos)
Gets the offset which refers to an n-gram.-
Methods inherited from class edu.berkeley.nlp.lm.AbstractContextEncodedNgramLanguageModel
getLogProb, scoreSentence
-
Methods inherited from class edu.berkeley.nlp.lm.AbstractNgramLanguageModel
getLmOrder, getWordIndexer, setOovWordLogProb
-
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface edu.berkeley.nlp.lm.NgramLanguageModel
getLmOrder, getLogProb, getWordIndexer, scoreSentence, setOovWordLogProb
-
-
-
-
Constructor Detail
-
ContextEncodedProbBackoffLm
public ContextEncodedProbBackoffLm(int lmOrder, WordIndexer<W> wordIndexer, ContextEncodedNgramMap<ProbBackoffPair> map, ConfigOptions opts)
-
-
Method Detail
-
getLogProb
public float getLogProb(long contextOffset, int contextOrder, int word, ContextEncodedNgramLanguageModel.LmContextInfo outputContext)
Description copied from interface:ContextEncodedNgramLanguageModel
Get the score for an n-gram, and also get the context offset of the n-gram's suffix.- Specified by:
getLogProb
in interfaceContextEncodedNgramLanguageModel<W>
- Specified by:
getLogProb
in classAbstractContextEncodedNgramLanguageModel<W>
- Parameters:
contextOffset
- Offset of context (prefix) of an n-gramcontextOrder
- The (0-based) length ofcontext
(i.e.order == 0
iffcontext
refers to a unigram).word
- Last word of the n-gramoutputContext
- Offset of the suffix of the input n-gram. If the parameter isnull
it will be ignored. This can be passed to future queries for efficient access.- Returns:
-
getOffsetForNgram
public ContextEncodedNgramLanguageModel.LmContextInfo getOffsetForNgram(int[] ngram, int startPos, int endPos)
Description copied from interface:ContextEncodedNgramLanguageModel
Gets the offset which refers to an n-gram. If the n-gram is not in the model, then it returns the shortest suffix of the n-gram which is. This operation is not necessarily fast.- Specified by:
getOffsetForNgram
in interfaceContextEncodedNgramLanguageModel<W>
- Specified by:
getOffsetForNgram
in classAbstractContextEncodedNgramLanguageModel<W>
-
getNgramForOffset
public int[] getNgramForOffset(long contextOffset, int contextOrder, int word)
Description copied from interface:ContextEncodedNgramLanguageModel
Gets the n-gram referred to by a context-encoding. This operation is not necessarily fast.- Specified by:
getNgramForOffset
in interfaceContextEncodedNgramLanguageModel<W>
- Specified by:
getNgramForOffset
in classAbstractContextEncodedNgramLanguageModel<W>
-
getNgramMap
public NgramMap<ProbBackoffPair> getNgramMap()
-
-