class nbla::Allocator

class Allocator : public std::enable_shared_from_this<Allocator>

Allocator interface class.

A derived class implements logics that manage cached device memory blocks.

A derived class NaiveAllocator which takes a Memory class as a template argument implements an memory allocator without caching.

Todo:

Support streams or events.

Note

A Allocator instance must be instantiated as a shared_ptr.

Subclassed by nbla::CachingAllocatorWithBucketsBase, nbla::NaiveAllocator< MemoryType >, nbla::VirtualCachingAllocatorBase

Public Functions

Allocator()

Constructor does nothing.

AllocatorMemoryPtr alloc(size_t bytes, const string &device_id)

Request a memory block wrapped by AllocatorMemory.

A derived class must implement alloc_impl where a memory is newly created or obtained from memory pool.

Parameters:

• bytes – [in] Number of bytes of memory block requested.

• device_id – [in] Device specifier string. A format of allow specifier is determined depending on Memory class implementation.

Returns:

A memory block wrapped by AllocatorMemory. It’s assumed that compiler RVO is enabled to prevent copying a return value. The returned AllocatorMemory must be moved to Array instance using std::move.

void free(shared_ptr<Memory> memory)

User should call this.

This is called from destructor of AllocatorMemory.

A given memory block is returned to this allocator instance. A derived class must implement Allocator::free_impl where a memory is returned to pool or simply freed if it’s non-caching allocator.

size_t free_unused_caches()

Free all unused memory blocks from all devices.

Allocator::free_unused_device_caches is called for all device IDs previously passed to Allocator::alloc function.

size_t free_unused_device_caches(const string &device_id)

Free all unused memory blocks in a specified device.

A derived class must implement Allocator::free_unused_device_caches_impl where all unused memory blocks that are origianally created, i.e. not divided, are discarded from pool.

Parameters:

device_id – [in] Specifies device by a string.

Returns:

Size of freed memory in bytes.

size_t device_memory_used_in_bytes(const string &device_id)

Get currently allocated memory size in a specifie device.

Parameters:

device_id – [in] Specifies device by a string.

Returns:

Total size of memory currently allocated by this allocator in bytes.

virtual ~Allocator()

Destructor does nothing.

inline void print_memory_cache_map()

APIs for memory cache analysis.