|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Objectorg.knime.core.data.container.Buffer
class Buffer
A buffer writes the rows from a DataContainer
to a file.
This class serves as connector between the DataContainer
and
the DataTable
that is returned by the container.
It "centralizes" the IO operations.
Nested Class Summary | |
---|---|
(package private) static class |
Buffer.FromFileIterator
Super class of all file iterators. |
Field Summary | |
---|---|
(package private) static String |
ZIP_ENTRY_BLOBS
Name of the zip entry containing the blob files (directory). |
(package private) static String |
ZIP_ENTRY_DATA
Name of the zip entry containing the data. |
(package private) static String |
ZIP_ENTRY_META
Name of the zip entry containing the meta information (e.g. |
Fields inherited from interface org.knime.core.data.container.KNIMEStreamConstants |
---|
BYTE_ROW_SEPARATOR, BYTE_TYPE_MISSING, BYTE_TYPE_SERIALIZATION, BYTE_TYPE_START, DUMMY_ROW_KEY, TC_ESCAPE, TC_TERMINATE |
Constructor Summary | |
---|---|
Buffer(File binFile,
File blobDir,
DataTableSpec spec,
InputStream metaIn,
int bufferID,
Map<Integer,ContainerTable> tblRep)
Creates new buffer for reading. |
|
Buffer(int maxRowsInMemory,
int bufferID,
Map<Integer,ContainerTable> globalRep,
Map<Integer,ContainerTable> localRep)
Creates new buffer for writing. |
Method Summary | |
---|---|
(package private) void |
addRow(DataRow r,
boolean isCopyOfExisting,
boolean forceCopyOfBlobs)
Adds a row to the buffer. |
(package private) void |
addToZipFile(ZipOutputStream zipOut,
ExecutionMonitor exec)
Method that's been called from the ContainerTable
to save the content. |
(package private) void |
clear()
Clears the temp file. |
(package private) void |
clearIteratorInstance(Buffer.FromFileIterator it,
boolean removeFromHash)
Clear the argument iterator (free the allocated resources. |
(package private) void |
close(DataTableSpec spec)
Flushes and closes the stream. |
(package private) boolean |
containsBlobCells()
True if any row containing blob cells is contained in this buffer. |
(package private) static File |
createBlobDirNameForTemp(File tempFile)
Guesses a "good" blob directory for a given binary temp file. |
(package private) Buffer |
createLocalCloneForWriting()
Creates a clone of this buffer for writing the content to a stream that is of the current version. |
protected void |
finalize()
Deletes the file underlying this buffer. |
(package private) File |
getBinFile()
|
(package private) File |
getBlobFile(int indexBlobInCol,
int column,
boolean createPath,
boolean isCompressed)
Determines the file location for a blob to be read/written with some given coordinates (column and index in column). |
(package private) int |
getBufferID()
Get this buffer's ID. |
(package private) Map<Integer,ContainerTable> |
getGlobalRepository()
Get reference to the table repository that this buffer was initially instantiated with. |
(package private) Map<Integer,ContainerTable> |
getLocalRepository()
Get reference to the local table repository that this buffer was initially instantiated with. |
(package private) int |
getReadVersion()
Get underlying stream version. |
DataTableSpec |
getTableSpec()
Get the table spec that was set in the constructor. |
(package private) CellClassInfo |
getTypeForChar(byte identifier)
Perform lookup for the DataCell class info given the argument byte. |
String |
getVersion()
Get the version string to write to the meta file. |
(package private) void |
incrementSize()
Increments the row counter by one, used in addRow. |
(package private) boolean |
isBinFileGZipped()
|
(package private) CloseableRowIterator |
iterator()
Get a new RowIterator , traversing all rows that have been
added. |
(package private) BlobDataCell |
readBlobDataCell(BlobDataCell.BlobAddress blobAddress,
CellClassInfo cl)
Reads the blob from the given blob address. |
(package private) void |
restoreIntoMemory()
Restore content of this buffer into main memory (using a collection implementation). |
(package private) boolean |
shouldSkipRowKey()
Get whether the buffer wants to persist row keys. |
int |
size()
Get the row count. |
(package private) boolean |
usesOutFile()
Does the buffer use a file? |
int |
validateVersion(String version)
Validate the version as read from the file if it can be parsed by this implementation. |
(package private) void |
writeDataCell(DataCell cell,
DCObjectOutputVersion2 outStream)
Writes a data cell to the outStream. |
(package private) void |
writeRowKey(RowKey key,
DCObjectOutputVersion2 outStream)
Writes the row key to the out stream. |
Methods inherited from class java.lang.Object |
---|
clone, equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
Field Detail |
---|
static final String ZIP_ENTRY_DATA
static final String ZIP_ENTRY_BLOBS
static final String ZIP_ENTRY_META
Constructor Detail |
---|
Buffer(int maxRowsInMemory, int bufferID, Map<Integer,ContainerTable> globalRep, Map<Integer,ContainerTable> localRep)
maxRowsInMemory
- Maximum numbers of rows that are kept in memory
until they will be subsequent written to the temp file. (0 to
write immediately to a file)globalRep
- Table repository for blob (de)serialization (read only).localRep
- Local table repository for blob (de)serialization.bufferID
- The id of this buffer used for blob (de)serialization.Buffer(File binFile, File blobDir, DataTableSpec spec, InputStream metaIn, int bufferID, Map<Integer,ContainerTable> tblRep) throws IOException
binFile
is the binary file as written by this class, which
will be deleted when this buffer is cleared or finalized.
binFile
- The binary file to read from (will be deleted on exit).blobDir
- temp directory containing blobs (may be null).spec
- The data table spec to which the this buffer complies to.metaIn
- An input stream from which this constructor reads the
meta information (e.g. which byte encodes which DataCell).bufferID
- The id of this buffer used for blob (de)serialization.tblRep
- Table repository for blob (de)serialization.
IOException
- If the header (the spec information) can't be read.Method Detail |
---|
public String getVersion()
NoKeyBuffer
to distinguish
streams written by the different implementations.
final int getReadVersion()
final File getBinFile()
final boolean isBinFileGZipped()
public int validateVersion(String version) throws IOException
version
- As read from file.
IOException
- If it can't be parsed.void addRow(DataRow r, boolean isCopyOfExisting, boolean forceCopyOfBlobs)
DataContainer
.
r
- The row to be added.isCopyOfExisting
- Whether to copy blobs (this is only true when
and existing buffer gets copied (version hop))forceCopyOfBlobs
- If true any blob that is not owned by this
buffer, will be copied and this buffer will take ownership. This option
is true for loop end nodes, which need to aggregate the data generated
in the loop bodyvoid incrementSize()
void close(DataTableSpec spec)
spec
- The spec the rows have to follow. No sanity check is done.boolean usesOutFile()
public DataTableSpec getTableSpec()
public int size()
boolean shouldSkipRowKey()
true
but overwritten in NoKeyBuffer
.
final void restoreIntoMemory()
Map<Integer,ContainerTable> getGlobalRepository()
Map<Integer,ContainerTable> getLocalRepository()
void writeRowKey(RowKey key, DCObjectOutputVersion2 outStream) throws IOException
NoKeyBuffer
in order to skip the row key.
key
- The key to write.outStream
- To write to.
IOException
- If that fails.void writeDataCell(DataCell cell, DCObjectOutputVersion2 outStream) throws IOException
cell
- The cell to write.outStream
- To write to.
IOException
- If stream corruption happens.BlobDataCell readBlobDataCell(BlobDataCell.BlobAddress blobAddress, CellClassInfo cl) throws IOException
blobAddress
- The address to read from.cl
- The expected class.
IOException
- If that fails.CellClassInfo getTypeForChar(byte identifier) throws IOException
identifier
- The byte as read from the stream.
IOException
- If the byte is invalid.static File createBlobDirNameForTemp(File tempFile)
tempFile
- base name
File getBlobFile(int indexBlobInCol, int column, boolean createPath, boolean isCompressed) throws IOException
indexBlobInCol
- The index in the column (generally the row number).column
- The column index.createPath
- Create the directory, if necessary (when writing)isCompressed
- If file is (to be) compressed
IOException
- If that fails (e.g. blob dir does not exist).CloseableRowIterator iterator()
RowIterator
, traversing all rows that have been
added. Calling this method makes only sense when the buffer has been
closed. However, no check is done (as it is available to package classes
only).
boolean containsBlobCells()
Buffer createLocalCloneForWriting()
void addToZipFile(ZipOutputStream zipOut, ExecutionMonitor exec) throws IOException, CanceledExecutionException
ContainerTable
to save the content. It will add zip entries to the zipOut
argument and not close the output stream when done, allowing
to add additional content elsewhere (for instance the
DataTableSpec
).
zipOut
- To write to.exec
- For progress/cancel
IOException
- If it fails to write to a file.
CanceledExecutionException
- If canceled.#saveToFile(File, NodeSettingsWO, ExecutionMonitor)
protected void finalize()
finalize
in class Object
Object.finalize()
int getBufferID()
void clearIteratorInstance(Buffer.FromFileIterator it, boolean removeFromHash)
it
- The iteratorremoveFromHash
- Whether to remove from global hash.void clear()
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |