## org.apache.lucene.analysis.ja
Class Token

    java.lang.Object
      org.apache.lucene.analysis.ja.Token

    public class Token

extends Object

Analyzed token with morphological data from its dictionary.

Constructor Summary
`Token(int wordId, char[] surfaceForm, int offset, int length, JapaneseTokenizer.Type type, int position, Dictionary dictionary)`

Method Summary
`String` `getBaseForm()`
`String` `getInflectionForm()`
`String` `getInflectionType()`
`int` `getLength()`
`int` `getOffset()`
`String` `getPartOfSpeech()`
`int` `getPosition()` Get index of this token in input text
`int` `getPositionLength()` Get the length (in tokens) of this token.
`String` `getPronunciation()`
`String` `getReading()`
`char[]` `getSurfaceForm()`
`String` `getSurfaceFormString()`
`boolean` `isKnown()` Returns true if this token is known word
`boolean` `isUnknown()` Returns true if this token is unknown word
`boolean` `isUser()` Returns true if this token is defined in user dictionary
`void` `setPositionLength(int positionLength)` Set the position length (in tokens) of this token.
`String` `toString()`

Methods inherited from class java.lang.Object
`clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

Constructor Detail

### Token

    public Token(int wordId,
                 char[] surfaceForm,
                 int offset,
                 int length,
                 JapaneseTokenizer.Type type,
                 int position,
                 Dictionary dictionary)

Method Detail

### toString

    public String toString()

  Overrides:
    `toString` in class `Object`

### getSurfaceForm

    public char[] getSurfaceForm()

  Returns:
    surfaceForm

### getOffset

    public int getOffset()

  Returns:
    offset into surfaceForm

### getLength

    public int getLength()

  Returns:
    length of surfaceForm

### getSurfaceFormString

    public String getSurfaceFormString()

  Returns:
    surfaceForm as a String

### getReading

    public String getReading()

  Returns:
    reading. null if token doesn't have reading.

### getPronunciation

    public String getPronunciation()

  Returns:
    pronunciation. null if token doesn't have pronunciation.

### getPartOfSpeech

    public String getPartOfSpeech()

  Returns:
    part of speech.

### getInflectionType

    public String getInflectionType()

  Returns:
    inflection type or null

### getInflectionForm

    public String getInflectionForm()

  Returns:
    inflection form or null

### getBaseForm

    public String getBaseForm()

  Returns:
    base form or null if token is not inflected

### isKnown

    public boolean isKnown()

  Returns true if this token is known word

  Returns:
    true if this token is in standard dictionary. false if not.

### isUnknown

    public boolean isUnknown()

  Returns true if this token is unknown word

  Returns:
    true if this token is unknown word. false if not.

### isUser

    public boolean isUser()

  Returns true if this token is defined in user dictionary

  Returns:
    true if this token is in user dictionary. false if not.

### getPosition

    public int getPosition()

  Get index of this token in input text

  Returns:
    position of token

### setPositionLength

    public void setPositionLength(int positionLength)

  Set the position length (in tokens) of this token. For normal tokens this is 1; for compound tokens it's > 1.

### getPositionLength

    public int getPositionLength()

  Get the length (in tokens) of this token. For normal tokens this is 1; for compound tokens it's > 1.

  Returns:
    position length of token

Copyright © 2000-2013 Apache Software Foundation. All Rights Reserved.