org.knime.base.node.io.filetokenizer
Class FileTokenizer

java.lang.Object
  extended by org.knime.base.node.io.filetokenizer.FileTokenizer

Deprecated. use Tokenizer instead. Will be removed in Ver3.0.

@Deprecated
public class FileTokenizer
extends Object


Field Summary
(package private) static String LF_STR
          Deprecated. String containing only the LF char.
static int MAX_CHAR
          Deprecated. The maximum ASCII code for the first character of patterns (like delimiter, comment, and quote patterns.
 
Constructor Summary
FileTokenizer(Reader source)
          Deprecated. Creates a new tokenizer with the default behaviour.
 
Method Summary
 void closeSourceStream()
          Deprecated. Closes the stream the tokenizer reads from.
 String getLastQuoteBeginPattern()
          Deprecated. Returns the left quote of the last token.
 String getLastQuoteEndPattern()
          Deprecated. Returns the right quote of the last token.
 int getLineNumber()
          Deprecated.  
 long getReadBytes()
          Deprecated. Returns the number of bytes returned so far.
 FileTokenizerSettings getSettings()
          Deprecated.  
 boolean isWhiteSpace(char c)
          Deprecated.  
 boolean lastTokenWasDelimiter()
          Deprecated.  
 boolean lastTokenWasQuoted()
          Deprecated. Call this to distinguish between missing and empty tokens.
 String nextToken()
          Deprecated. Reads the next token from the stream and returns it as string.
 void pushBack()
          Deprecated. After a call to this function the token returned with the last call to the nextToken() function will be returned once again with the next call the the nextToken() function.
 void resetToDefault()
          Deprecated. Resets the tokenizer to its default behavior, which is: No comments are supported, No quoted strings are supported, No line continuation is supported, and No token delimiter is set (only EOF - i.e.
 void setSettings(FileTokenizerSettings ftSettings)
          Deprecated. Set new user settings in this tokenizer.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

MAX_CHAR

public static final int MAX_CHAR
Deprecated. 
The maximum ASCII code for the first character of patterns (like delimiter, comment, and quote patterns.

See Also:
Constant Field Values

LF_STR

static final String LF_STR
Deprecated. 
String containing only the LF char. Used internally for line comment

Constructor Detail

FileTokenizer

public FileTokenizer(Reader source)
Deprecated. 
Creates a new tokenizer with the default behaviour.

Parameters:
source - A reader the tokens are read from.
See Also:
for what's the default behaviour.
Method Detail

resetToDefault

public void resetToDefault()
Deprecated. 
Resets the tokenizer to its default behavior, which is: It does not reset the line number, internal read buffers or the stream.


lastTokenWasDelimiter

public boolean lastTokenWasDelimiter()
Deprecated. 
Returns:
Returns true if last token is a delimiter token, otherwise false.

nextToken

public String nextToken()
                 throws FileTokenizerException
Deprecated. 
Reads the next token from the stream and returns it as string. Or null if no more token can be read.

Returns:
The next token from the stream or null at the EOF.
Throws:
FileTokenizerException - if something goes wrong during tokenizing.

isWhiteSpace

public boolean isWhiteSpace(char c)
Deprecated. 
Parameters:
c - the character to test.
Returns:
true if the specified character c is in the user defined Vector of whitespaces.

pushBack

public void pushBack()
Deprecated. 
After a call to this function the token returned with the last call to the nextToken() function will be returned once again with the next call the the nextToken() function. Pushing back a token does not decrease the line number accordingly.

See Also:
nextToken()

lastTokenWasQuoted

public boolean lastTokenWasQuoted()
Deprecated. 
Call this to distinguish between missing and empty tokens. If quote patterns are set, e.g. '"' for beginning and ending quotes, and the delimiter is set for example to a comma, the following line will return five tokens and all of them will be returned as empty strings: "",,"",,"" The first, third, and fifth tokens are specified - but empty. The second and fourth are not specified causing an empty token to be returned. With this function you can figure out the difference.

Returns:
true if the last token had quotes which were removed by the tokenizer.

getLastQuoteBeginPattern

public String getLastQuoteBeginPattern()
Deprecated. 
Returns the left quote of the last token. Or, if the token contained multiple quoted parts, the left quote of the last part that was quoted. If there were no quotes in the last token, null will be returned.
For example, if the tokenized stream contains ...,"foo"poo'loo',... with comma separated tokens and single and double quotes - the last quote begin pattern would be the single quote (').

Returns:
the left quote pattern of the quotes in the last token. Or null if it wasn't quoted.

getLastQuoteEndPattern

public String getLastQuoteEndPattern()
Deprecated. 
Returns the right quote of the last token. Or, if the token contained multiple quoted parts, the right quote of the last part that was quoted. If there were no quotes in the last token, null will be returned.
For example, if the tokenized stream contains ...,"foo"poo'loo',... with comma separated tokens and single and double quotes - the last quote begin pattern would be the single quote (').

Returns:
the right quote pattern of the quotes in the last token. Or null if it wasn't quoted.

getLineNumber

public int getLineNumber()
Deprecated. 
Returns:
The current line number in the stream.

getReadBytes

public long getReadBytes()
Deprecated. 
Returns the number of bytes returned so far. Due to the buffering the number of bytes read from the disk and the number of bytes returned by this tokenizer can differ.

Returns:
the number of bytes returned so far by this tokenizer

closeSourceStream

public void closeSourceStream()
Deprecated. 
Closes the stream the tokenizer reads from. After the tokenizer read the EOF from the stream it closes it automatically. If it's required to close the stream before the end is read, you can call this method. A call to nextToken() after a call to this token will return null (indicating the end of the file).


setSettings

public void setSettings(FileTokenizerSettings ftSettings)
Deprecated. 
Set new user settings in this tokenizer. The only way to configure this tokenizer is to create an instance of the FileTokenizerSettings, add all parameters there and pass the settings object through this method.

Parameters:
ftSettings - the settings object containing new settings.

getSettings

public FileTokenizerSettings getSettings()
Deprecated. 
Returns:
an object containing the current settings of the tokenizer


Copyright, 2003 - 2010. All rights reserved.
University of Konstanz, Germany.
Chair for Bioinformatics and Information Mining, Prof. Dr. Michael R. Berthold.
You may not modify, publish, transmit, transfer or sell, reproduce, create derivative works from, distribute, perform, display, or in any way exploit any of the content, in whole or in part, except as otherwise expressly permitted in writing by the copyright owner or as specified in the license file distributed with this product.