org.apache.flink.streaming.runtime.io

Class CachedBufferStorage

java.lang.Object

org.apache.flink.streaming.runtime.io.CachedBufferStorage

All Implemented Interfaces:

AutoCloseable, BufferStorage

@Internal
public class CachedBufferStorage
extends Object
implements BufferStorage

The CachedBufferStorage takes the buffers and events from a data stream and adds them to a memory queue. After a number of elements have been cached, the CachedBufferStorage can "roll over": It presents the cached elements as a readable sequence, and creates a new memory queue.

This CachedBufferStorage can be used in credit-based flow control for better barrier alignment in exactly-once mode.

Constructor Summary

Constructors

Constructor and Description
CachedBufferStorage(int pageSize) Create a new CachedBufferStorage with unlimited storage.
CachedBufferStorage(int pageSize, long maxBufferedBytes, String taskName) Creates a new CachedBufferStorage, caching the buffers or events in memory queue.

Method Summary

Modifier and Type	Method and Description
void	add(org.apache.flink.runtime.io.network.partition.consumer.BufferOrEvent boe) Adds a buffer or event to the BufferStorage.
void	close() Cleans up all the resources in the current sequence.
long	getMaxBufferedBytes()
long	getPendingBytes()
long	getRolledBytes()
boolean	isEmpty()
boolean	isFull()
Optional<org.apache.flink.runtime.io.network.partition.consumer.BufferOrEvent>	pollNext()
void	rollOver() Start returning next sequence of stored BufferOrEvents.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CachedBufferStorage

public CachedBufferStorage(int pageSize)

Create a new CachedBufferStorage with unlimited storage.

Parameters:

pageSize - The page size used to estimate the cached size.

CachedBufferStorage

public CachedBufferStorage(int pageSize,
                           long maxBufferedBytes,
                           String taskName)

Creates a new CachedBufferStorage, caching the buffers or events in memory queue.

Parameters:

pageSize - The page size used to estimate the cached size.

Method Detail

add

public void add(org.apache.flink.runtime.io.network.partition.consumer.BufferOrEvent boe)

Description copied from interface: BufferStorage

Adds a buffer or event to the BufferStorage.

Specified by:

add in interface BufferStorage

Parameters:

boe - The buffer or event to be added into the blocker.

close

public void close()

Description copied from interface: BufferStorage

Cleans up all the resources in the current sequence.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface BufferStorage

getPendingBytes

public long getPendingBytes()

Specified by:

getPendingBytes in interface BufferStorage

Returns:

the number of pending bytes blocked in the current sequence - bytes that are have not been yet rolled, but are already blocked.

isFull

public boolean isFull()

Specified by:

isFull in interface BufferStorage

Returns:

true if size limit was exceeded.

rollOver

public void rollOver()

Description copied from interface: BufferStorage

Start returning next sequence of stored BufferOrEvents.

Specified by:

rollOver in interface BufferStorage

getRolledBytes

public long getRolledBytes()

Specified by:

getRolledBytes in interface BufferStorage

Returns:

the number of already rolled bytes in in blocked sequences.

isEmpty

public boolean isEmpty()

Specified by:

isEmpty in interface BufferStorage

Returns:

true if this BufferStorage doesn't store and data.

pollNext

public Optional<org.apache.flink.runtime.io.network.partition.consumer.BufferOrEvent> pollNext()

Specified by:

pollNext in interface BufferStorage

getMaxBufferedBytes

public long getMaxBufferedBytes()

Specified by:

getMaxBufferedBytes in interface BufferStorage