com.yahoo.sketches.hll

Interface Fields

public interface Fields

An interface that abstracts out the underlying storage of an HLL from the hashing and other activities required to maintain an HLL.

This interface is experimental, but the API is not solidified and we reserve the right to make backwards incompatible changes without pushing up the library's version.

Implement at your own risk.

Author:

Kevin Lang

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static interface	Fields.UpdateCallback An UpdateCallback is a callback provided to calls that potentially update buckets.

Field Summary

Fields

Modifier and Type	Field and Description
static byte	COMPRESSED_DENSE_VERSION Compressed dense version
static byte	HASH_SPARSE_VERSION Hash sparse version
static byte	NAIVE_DENSE_VERSION Naive dense version
static Fields.UpdateCallback	NOOP_CB Returns a new No-Op Callback
static byte	SORTED_SPARSE_VERSION Sorted sparse version

Method Summary

Modifier and Type	Method and Description
BucketIterator	getBucketIterator() Returns a BucketIterator over the buckets in this Fields object.
Preamble	getPreamble() Gets the Preamble
int	intoByteArray(byte[] array, int offset) Fills the array starting from offset with the byte array representation of the fields
int	numBytesToSerialize() Provides an indication of how many bytes would be required to serialize the fields to a byte[].
Fields	toCompact() Return the compact form of this fields object.
Fields	unionBucketIterator(BucketIterator iter, Fields.UpdateCallback cb) Unions the provided BucketIterator into the current Fields object.
Fields	unionCompressedAndExceptions(byte[] compressed, int minVal, com.yahoo.sketches.hll.OnHeapHash exceptions, Fields.UpdateCallback cb) Unions the provided compressed byte[] and exceptions hash into the current Fields object.
Fields	unionInto(Fields recipient, Fields.UpdateCallback cb) Unions the current Fields into the Fields presented as an argument.
Fields	updateBucket(int bucket, byte val, Fields.UpdateCallback callback) Potentially updates a bucket in the underlying storage.

Field Detail

NAIVE_DENSE_VERSION

static final byte NAIVE_DENSE_VERSION

Naive dense version

See Also:

Constant Field Values

HASH_SPARSE_VERSION

static final byte HASH_SPARSE_VERSION

Hash sparse version

See Also:

Constant Field Values

SORTED_SPARSE_VERSION

static final byte SORTED_SPARSE_VERSION

Sorted sparse version

See Also:

Constant Field Values

COMPRESSED_DENSE_VERSION

static final byte COMPRESSED_DENSE_VERSION

Compressed dense version

See Also:

Constant Field Values

NOOP_CB

static final Fields.UpdateCallback NOOP_CB

Returns a new No-Op Callback

Method Detail

getPreamble

Preamble getPreamble()

Gets the Preamble

Returns:

the Preamble

updateBucket

Fields updateBucket(int bucket,
                    byte val,
                    Fields.UpdateCallback callback)

Potentially updates a bucket in the underlying storage. The Fields implementation is expected to maintain the MAX val for each bucket. If the val passed in is less than the currently stored val, this method should do nothing.

A callback *must* be provided which will be called whenever the provided val is greater than the currently stored value.

Parameters:

bucket - the bucket to update

val - the val to update to

callback - the callback to be called if the provided val is greater than the current

Returns:

the Fields object that should be used from this point forward

intoByteArray

int intoByteArray(byte[] array,
                  int offset)

Fills the array starting from offset with the byte array representation of the fields

This should *not* include the preamble

Parameters:

array - given array to fill

offset - starting with this offset

Returns:

the last offset written +1

numBytesToSerialize

int numBytesToSerialize()

Provides an indication of how many bytes would be required to serialize the fields to a byte[].

Returns:

the number of bytes to serialize the fields to a byte[]

toCompact

Fields toCompact()

Return the compact form of this fields object. The compact form is intended as the most compact representation of the same data. Generally, the compact form is not updatable and it is acceptable for it to throw UnsupportedOperationExceptions out of the update calls.

Returns:

The compact representation of these fields.

getBucketIterator

BucketIterator getBucketIterator()

Returns a BucketIterator over the buckets in this Fields object.

Returns:

a Bucket iterator that walks the buckets of this Fields object.

unionInto

Fields unionInto(Fields recipient,
                 Fields.UpdateCallback cb)

Unions the current Fields into the Fields presented as an argument. This exists to allow for polymorphic dispatch to enable optimized unioning when available. That is, most implementations will end up delegating to: recipient.unionBucketIterator(getBucketIterator());

But, if there is a method that is more specific to the implementation, it can choose to delegate to, e.g: unionCompressedAndExceptions()

Parameters:

recipient - The fields to be unioned *into*

cb - The callback to be called whenever a bucket value is updated

Returns:

The new fields object to use to represent the unioned buckets

unionBucketIterator

Fields unionBucketIterator(BucketIterator iter,
                           Fields.UpdateCallback cb)

Unions the provided BucketIterator into the current Fields object.

Parameters:

iter - the BucketIterator to union into the current Fields object

cb - The callback to be called whenever a bucket value is updated

Returns:

The new fields object to use to represent the unioned buckets

unionCompressedAndExceptions

Fields unionCompressedAndExceptions(byte[] compressed,
                                    int minVal,
                                    com.yahoo.sketches.hll.OnHeapHash exceptions,
                                    Fields.UpdateCallback cb)

Unions the provided compressed byte[] and exceptions hash into the current Fields object.

Parameters:

compressed - a byte array of compressed, 4-bit values as used by OnHeapCompressedFields

minVal - the minimum value (or "offset"). I.e. a 0 in a nibble from compressed represent this actual value

exceptions - the exceptions hash

cb - The callback to be called whenever a bucket value is updated

Returns:

The new fields object to use to represent the unioned buckets