Package li.selman.jpbe.dsl.token

Class TokenSequenceBuilder

java.lang.Object

li.selman.jpbe.dsl.token.TokenSequenceBuilder

public class TokenSequenceBuilder
extends Object

Constructor Summary

Constructors

Constructor	Description
TokenSequenceBuilder(int maxLength, BiFunction<Character,Token,Optional<Token>> computeTokenForCharHook, Tokens tokens)
TokenSequenceBuilder(int maxLength, Tokens tokens)

Method Summary

Modifier and Type	Method	Description
TokenSequence	computeTokenSequence(String input, int from, int to)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

TokenSequenceBuilder

public TokenSequenceBuilder(int maxLength,
 Tokens tokens)

TokenSequenceBuilder

public TokenSequenceBuilder(int maxLength,
 BiFunction<Character,Token,Optional<Token>> computeTokenForCharHook,
 Tokens tokens)

Method Details

computeTokenSequence

public TokenSequence computeTokenSequence(String input,
 int from,
 int to)

Parameters:

input - the whole input string provided by the data set

from - start index for sub-string on input

to - end index for sub-string on input

Returns:

sequence of tokens representing the token structure of a substring on input