org.apache.arrow.memory

Class RootAllocator

java.lang.Object

org.apache.arrow.memory.RootAllocator

All Implemented Interfaces:

AutoCloseable, BufferAllocator

public class RootAllocator
extends Object

A root allocator for using direct memory for Arrow Vectors/Arrays. Supports creating a tree of descendant child allocators to facilitate better instrumentation of memory allocations.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BaseAllocator.Verbosity Enum for logging verbosity.

Field Summary

Fields

Modifier and Type	Field and Description
static boolean	DEBUG
static String	DEBUG_ALLOCATOR
static int	DEBUG_LOG_LENGTH
static org.apache.arrow.memory.BaseAllocator.Config	DEFAULT_CONFIG
protected org.apache.arrow.memory.Accountant	parent The parent allocator.
protected long	reservation The amount of memory reserved for this allocator.

Constructor Summary

Constructors

Constructor and Description
RootAllocator()
RootAllocator(AllocationListener listener, long limit)
RootAllocator(AllocationListener listener, long limit, RoundingPolicy roundingPolicy) Constructor.
RootAllocator(org.apache.arrow.memory.BaseAllocator.Config config)
RootAllocator(long limit)

Method Summary

Modifier and Type	Method and Description
void	assertOpen() Asserts (using java assertions) that the provided allocator is currently open.
ArrowBuf	buffer(long initialRequestSize) Allocate a new or reused buffer of the provided size.
ArrowBuf	buffer(long initialRequestSize, BufferManager manager) Allocate a new or reused buffer of the provided size.
void	close() Close this Accountant.
static ImmutableConfig.Builder	configBuilder() Returns a builder class for configuring BaseAllocator's options.
static org.apache.arrow.memory.BaseAllocator.Config	defaultConfig() Returns a default Config instance.
boolean	forceAllocate(long size) Increase the accounting.
long	getAllocatedMemory() Return the current amount of allocated memory that this Accountant is managing accounting for.
Collection<BufferAllocator>	getChildAllocators() Returns the set of child allocators.
ArrowBuf	getEmpty() Get a reference to the empty buffer associated with this allocator.
long	getHeadroom()
long	getInitReservation() Return the initial reservation.
long	getLimit() Return the current limit of this Accountant.
AllocationListener	getListener() Returns the allocation listener used by this allocator.
String	getName() Return the name of the accountant.
org.apache.arrow.memory.BaseAllocator	getParentAllocator() Returns the parent allocator.
long	getPeakMemoryAllocation() The peak memory allocated by this Accountant.
BufferAllocator	getRoot() Get the root allocator of this allocator.
RoundingPolicy	getRoundingPolicy() Gets the rounding policy of the allocator.
static boolean	isDebug()
boolean	isOverLimit()
BufferAllocator	newChildAllocator(String name, AllocationListener listener, long initReservation, long maxAllocation) Create a new child allocator.
BufferAllocator	newChildAllocator(String name, long initReservation, long maxAllocation) Create a new child allocator.
AllocationReservation	newReservation() Create an allocation reservation.
void	releaseBytes(long size)
void	setLimit(long newLimit) Set the maximum amount of memory that can be allocated in the this Accountant before failing an allocation.
String	toString()
String	toVerboseString() Provide a verbose string of the current allocator state.
void	verify() Verify the accounting state of the allocation system.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.arrow.memory.BufferAllocator

forceAllocate, getAllocatedMemory, getHeadroom, getInitReservation, getLimit, getPeakMemoryAllocation, isOverLimit, releaseBytes, setLimit

Field Detail

DEBUG_ALLOCATOR

public static final String DEBUG_ALLOCATOR

See Also:

Constant Field Values

DEBUG_LOG_LENGTH

public static final int DEBUG_LOG_LENGTH

See Also:

Constant Field Values

DEBUG

public static final boolean DEBUG

DEFAULT_CONFIG

public static final org.apache.arrow.memory.BaseAllocator.Config DEFAULT_CONFIG

parent

protected final org.apache.arrow.memory.Accountant parent

The parent allocator.

reservation

protected final long reservation

The amount of memory reserved for this allocator. Releases below this amount of memory will not be returned to the parent Accountant until this Accountant is closed.

Constructor Detail

RootAllocator

public RootAllocator()

RootAllocator

public RootAllocator(long limit)

RootAllocator

public RootAllocator(AllocationListener listener,
                     long limit)

RootAllocator

public RootAllocator(AllocationListener listener,
                     long limit,
                     RoundingPolicy roundingPolicy)

Constructor.

Parameters:

listener - the allocation listener

limit - max allocation size in bytes

roundingPolicy - the policy for rounding the buffer size

RootAllocator

public RootAllocator(org.apache.arrow.memory.BaseAllocator.Config config)

Method Detail

verify

public void verify()

Verify the accounting state of the allocation system.

getListener

public AllocationListener getListener()

Description copied from interface: BufferAllocator

Returns the allocation listener used by this allocator.

Specified by:

getListener in interface BufferAllocator

Returns:

the AllocationListener instance. Or AllocationListener.NOOP by default if no listener is configured when this allocator was created.

getParentAllocator

public org.apache.arrow.memory.BaseAllocator getParentAllocator()

Description copied from interface: BufferAllocator

Returns the parent allocator.

Specified by:

getParentAllocator in interface BufferAllocator

Returns:

parent allocator

getChildAllocators

public Collection<BufferAllocator> getChildAllocators()

Description copied from interface: BufferAllocator

Returns the set of child allocators.

Specified by:

getChildAllocators in interface BufferAllocator

Returns:

set of child allocators

isDebug

public static boolean isDebug()

assertOpen

public void assertOpen()

Description copied from interface: BufferAllocator

Asserts (using java assertions) that the provided allocator is currently open. If assertions are disabled, this is a no-op.

Specified by:

assertOpen in interface BufferAllocator

getName

public String getName()

Return the name of the accountant.

Specified by:

getName in interface BufferAllocator

Returns:

name of accountant

getEmpty

public ArrowBuf getEmpty()

Description copied from interface: BufferAllocator

Get a reference to the empty buffer associated with this allocator. Empty buffers are special because we don't worry about them leaking or managing reference counts on them since they don't actually point to any memory.

Specified by:

getEmpty in interface BufferAllocator

Returns:

the empty buffer

buffer

public ArrowBuf buffer(long initialRequestSize)

Description copied from interface: BufferAllocator

Allocate a new or reused buffer of the provided size. Note that the buffer may technically be larger than the requested size for rounding purposes. However, the buffer's capacity will be set to the configured size.

Specified by:

buffer in interface BufferAllocator

Parameters:

initialRequestSize - The size in bytes.

Returns:

a new ArrowBuf, or null if the request can't be satisfied

buffer

public ArrowBuf buffer(long initialRequestSize,
                       BufferManager manager)

Description copied from interface: BufferAllocator

Allocate a new or reused buffer of the provided size. Note that the buffer may technically be larger than the requested size for rounding purposes. However, the buffer's capacity will be set to the configured size.

Specified by:

buffer in interface BufferAllocator

Parameters:

initialRequestSize - The size in bytes.

manager - A buffer manager to manage reallocation.

Returns:

a new ArrowBuf, or null if the request can't be satisfied

getRoot

public BufferAllocator getRoot()

Description copied from interface: BufferAllocator

Get the root allocator of this allocator. If this allocator is already a root, return this directly.

Specified by:

getRoot in interface BufferAllocator

Returns:

The root allocator

newChildAllocator

public BufferAllocator newChildAllocator(String name,
                                         long initReservation,
                                         long maxAllocation)

Description copied from interface: BufferAllocator

Create a new child allocator.

Specified by:

newChildAllocator in interface BufferAllocator

Parameters:

name - the name of the allocator.

initReservation - the initial space reservation (obtained from this allocator)

maxAllocation - maximum amount of space the new allocator can allocate

Returns:

the new allocator, or null if it can't be created

newChildAllocator

public BufferAllocator newChildAllocator(String name,
                                         AllocationListener listener,
                                         long initReservation,
                                         long maxAllocation)

Description copied from interface: BufferAllocator

Create a new child allocator.

Specified by:

newChildAllocator in interface BufferAllocator

Parameters:

name - the name of the allocator.

listener - allocation listener for the newly created child

initReservation - the initial space reservation (obtained from this allocator)

maxAllocation - maximum amount of space the new allocator can allocate

Returns:

the new allocator, or null if it can't be created

newReservation

public AllocationReservation newReservation()

Description copied from interface: BufferAllocator

Create an allocation reservation. A reservation is a way of building up a request for a buffer whose size is not known in advance. See

Specified by:

newReservation in interface BufferAllocator

Returns:

the newly created reservation

See Also:

AllocationReservation

close

public void close()

Close this Accountant. This will release any reservation bytes back to a parent Accountant.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface BufferAllocator

toString

public String toString()

Overrides:

toString in class Object

toVerboseString

public String toVerboseString()

Provide a verbose string of the current allocator state. Includes the state of all child allocators, along with historical logs of each object and including stacktraces.

Specified by:

toVerboseString in interface BufferAllocator

Returns:

A Verbose string of current allocator state.

defaultConfig

public static org.apache.arrow.memory.BaseAllocator.Config defaultConfig()

Returns a default Config instance.

See Also:

ImmutableConfig.Builder

configBuilder

public static ImmutableConfig.Builder configBuilder()

Returns a builder class for configuring BaseAllocator's options.

getRoundingPolicy

public RoundingPolicy getRoundingPolicy()

Description copied from interface: BufferAllocator

Gets the rounding policy of the allocator.

Specified by:

getRoundingPolicy in interface BufferAllocator

forceAllocate

public boolean forceAllocate(long size)

Increase the accounting. Returns whether the allocation fit within limits.

Parameters:

size - to increase

Returns:

Whether the allocation fit within limits.

releaseBytes

public void releaseBytes(long size)

isOverLimit

public boolean isOverLimit()

getLimit

public long getLimit()

Return the current limit of this Accountant.

Returns:

Limit in bytes.

getInitReservation

public long getInitReservation()

Return the initial reservation.

Returns:

reservation in bytes.

setLimit

public void setLimit(long newLimit)

Set the maximum amount of memory that can be allocated in the this Accountant before failing an allocation.

Parameters:

newLimit - The limit in bytes.

getAllocatedMemory

public long getAllocatedMemory()

Return the current amount of allocated memory that this Accountant is managing accounting for. Note this does not include reservation memory that hasn't been allocated.

Returns:

Currently allocate memory in bytes.

getPeakMemoryAllocation

public long getPeakMemoryAllocation()

The peak memory allocated by this Accountant.

Returns:

The peak allocated memory in bytes.

getHeadroom

public long getHeadroom()