Staff View: :: Library Catalog

Saved in:

Bibliographic Details
Main Authors:	Husbands, Brandon, Witchborn Systems
Format:	Recurso digital
Language:
Published:	Zenodo 2026
Online Access:	https://doi.org/10.5281/zenodo.19259980
Tags:	Add Tag No Tags, Be the first to tag this record!

_version_	1866901694873337856
author	Husbands, Brandon Witchborn Systems
author_facet	Husbands, Brandon Witchborn Systems
contents	<p>Trickums is a GPU memory virtualization system designed to extend the effective VRAM capacity available to AI workloads through a multi-tiered architecture spanning local GPU memory, system RAM, NVMe storage, and distributed remote nodes. By intercepting CUDA and framework-level allocation calls, Trickums presents a unified, oversized memory space while dynamically paging data between tiers based on access patterns and execution context.</p> <p>The system implements demand-driven page migration, eviction policies, and predictive prefetching, enabling large models to execute on hardware with limited native VRAM. Integration with orchestration layers allows coordinated scheduling of model shards and data movement, reducing stall conditions and maximizing resource utilization across heterogeneous environments.</p> <p>This document describes the architecture, memory management strategies, and system design of Trickums, including comparisons to unified memory systems, GPU-direct data paths, and remote memory frameworks. It outlines how VRAM oversubscription can be achieved transparently while maintaining correctness, performance constraints, and scalability in distributed AI compute networks.</p>
format	Recurso digital
id	zenodo_https___doi_org_10_5281_zenodo_19259980
institution	Zenodo
language
publishDate	2026
publisher	Zenodo
record_format	zenodo
spellingShingle	Trickums: The VRAM Illusion System of ForgeBorn Husbands, Brandon Witchborn Systems <p>Trickums is a GPU memory virtualization system designed to extend the effective VRAM capacity available to AI workloads through a multi-tiered architecture spanning local GPU memory, system RAM, NVMe storage, and distributed remote nodes. By intercepting CUDA and framework-level allocation calls, Trickums presents a unified, oversized memory space while dynamically paging data between tiers based on access patterns and execution context.</p> <p>The system implements demand-driven page migration, eviction policies, and predictive prefetching, enabling large models to execute on hardware with limited native VRAM. Integration with orchestration layers allows coordinated scheduling of model shards and data movement, reducing stall conditions and maximizing resource utilization across heterogeneous environments.</p> <p>This document describes the architecture, memory management strategies, and system design of Trickums, including comparisons to unified memory systems, GPU-direct data paths, and remote memory frameworks. It outlines how VRAM oversubscription can be achieved transparently while maintaining correctness, performance constraints, and scalability in distributed AI compute networks.</p>
title	Trickums: The VRAM Illusion System of ForgeBorn
url	https://doi.org/10.5281/zenodo.19259980

if (filemtime($filename) + $maxlifetime < time()) { unlink($filename); $count++; } } return $count; } /** * A function that is called internally when session data is to be saved. * * @param string $sessId The current session ID * @param string $data The session data to write * * @return bool */ protected function saveSession($sessId, $data): bool { $sessFile = $this->path . '/sess_' . $sessId; if ($handle = fopen($sessFile, 'w')) { $return = false; // Lock the file for exclusive access to avoid issues with multiple // processes writing session simultaneously: if (flock($handle, LOCK_EX)) { $return = fwrite($handle, $data); // Make sure that there's no trailing data by truncating the file to // the correct length: ftruncate($handle, strlen($data)); fflush($handle); flock($handle, LOCK_UN); } fclose($handle); if ($return !== false) { return true; } } // If we got this far, something went wrong with the file output... // It is tempting to throw an exception here, but this code is called // outside of the context of exception handling, so all we can do is // echo a message.

// Anecdotal testing Today and Yesterday seems to indicate destroy() // is called by the garbage collector and everything is good. // Something to keep in mind though. return 0; } /** * Write function that is called when session data is to be saved. * * @param string $sessId The current session ID * @param string $data The session data to write * * @return bool */ public function write($sessId, $data): bool { if ($this->writesDisabled) { return true; } return $this->saveSession($sessId, $data); } /** * A function that is called internally when session data is to be saved. * * @param string $sessId The current session ID * @param string $data The session data to write * * @return bool */ abstract protected function saveSession($sessId, $data): bool; }

* * @return void */ public function writeClose() { // The assumption is that we're using PHP's ext/session. // session_write_close() will actually overwrite $_SESSION with an // empty array on completion -- which leads to a mismatch between what // is in the storage object and $_SESSION. To get around this, we // temporarily reset $_SESSION to an array, and then re-link it to // the storage object. // // Additionally, while you _can_ write to $_SESSION following a // session_write_close() operation, no changes made to it will be // flushed to the session handler. As such, we now mark the storage // object isImmutable. $storage = $this->getStorage(); if (! $storage->isImmutable()) { $_SESSION = $storage->toArray(true); session_write_close(); $storage->fromArray($_SESSION); $storage->markImmutable(); } } /** * Attempt to set the session name * * If the session has already been started, or if the name provided fails * validation, an exception will be raised. * * @param string $name * @return SessionManager * @throws Exception\InvalidArgumentException */ public function setName($name) { if ($this->sessionExists()) { throw new Exception\InvalidArgumentException( 'Cannot set session name after a session has already started'

Environment & details:

GET Data empty

POST Data empty

Files empty

Key	Value
ui	"standard"
VUFIND_SESSION	"d5ff77aec8c4f7be3541dc542addb279"
language	"en"

Session

Key	Value
__Laminas	array:2 [`"_REQUEST_ACCESS_TIME" => 1784443394.1377 "_VALID" => array:1 [ "Laminas\Session\Validator\Id" => "d5ff77aec8c4f7be3541dc542addb279" ]`]
SessionState	Laminas\Stdlib\ArrayObject {#938}
FlashMessenger	Laminas\Stdlib\ArrayObject {#1583}
SessionHelper	Laminas\Stdlib\ArrayObject {#1773}

Server/Request Data

Key	Value
REDIRECT_VUFIND_LOCAL_DIR	"/usr/local/vufind/local"
REDIRECT_HTTPS	"on"
REDIRECT_SSL_TLS_SNI	"info.umar.mx"
REDIRECT_STATUS	"200"
VUFIND_LOCAL_DIR	"/usr/local/vufind/local"
HTTPS	"on"
SSL_TLS_SNI	"info.umar.mx"
HTTP_ACCEPT	"/"
HTTP_USER_AGENT	"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)"
HTTP_COOKIE	"ui=standard; VUFIND_SESSION=d5ff77aec8c4f7be3541dc542addb279; language=en"
HTTP_ACCEPT_ENCODING	"gzip, br, zstd, deflate"
HTTP_HOST	"info.umar.mx"
PATH	"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/snap/bin"
SERVER_SIGNATURE	"<address>Apache/2.4.66 (Ubuntu) Server at info.umar.mx Port 443</address>\n"
SERVER_SOFTWARE	"Apache/2.4.66 (Ubuntu)"
SERVER_NAME	"info.umar.mx"
SERVER_ADDR	"200.23.222.19"
SERVER_PORT	"443"
REMOTE_ADDR	"216.73.217.30"
DOCUMENT_ROOT	"/usr/local/vufind/public"
REQUEST_SCHEME	"https"
CONTEXT_PREFIX	""
CONTEXT_DOCUMENT_ROOT	"/usr/local/vufind/public"
SERVER_ADMIN	"admin@localhost"
SCRIPT_FILENAME	"/usr/local/vufind/public/index.php"
REMOTE_PORT	"31046"
REDIRECT_URL	"/Record/zenodo_https___doi_org_10_5281_zenodo_19259980/Details"
GATEWAY_INTERFACE	"CGI/1.1"
SERVER_PROTOCOL	"HTTP/1.1"
REQUEST_METHOD	"GET"
QUERY_STRING	""
REQUEST_URI	"/Record/zenodo_https___doi_org_10_5281_zenodo_19259980/Details"
SCRIPT_NAME	"/index.php"
PHP_SELF	"/index.php"
REQUEST_TIME_FLOAT	1784443394.1312
REQUEST_TIME	1784443394

Environment Variables empty

Registered Handlers

0. Whoops\Handler\PrettyPageHandler

Environment & details:

GET Data empty

POST Data empty

Files empty

Key	Value
ui	"standard"
VUFIND_SESSION	"d5ff77aec8c4f7be3541dc542addb279"
language	"en"

Session

Key	Value
__Laminas	array:2 [`"_REQUEST_ACCESS_TIME" => 1784443394.1377 "_VALID" => array:1 [ "Laminas\Session\Validator\Id" => "d5ff77aec8c4f7be3541dc542addb279" ]`]
SessionState	Laminas\Stdlib\ArrayObject {#938}
FlashMessenger	Laminas\Stdlib\ArrayObject {#1583}
SessionHelper	Laminas\Stdlib\ArrayObject {#1773}

Server/Request Data

Key	Value
REDIRECT_VUFIND_LOCAL_DIR	"/usr/local/vufind/local"
REDIRECT_HTTPS	"on"
REDIRECT_SSL_TLS_SNI	"info.umar.mx"
REDIRECT_STATUS	"200"
VUFIND_LOCAL_DIR	"/usr/local/vufind/local"
HTTPS	"on"
SSL_TLS_SNI	"info.umar.mx"
HTTP_ACCEPT	"/"
HTTP_USER_AGENT	"Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko; compatible; ClaudeBot/1.0; +claudebot@anthropic.com)"
HTTP_COOKIE	"ui=standard; VUFIND_SESSION=d5ff77aec8c4f7be3541dc542addb279; language=en"
HTTP_ACCEPT_ENCODING	"gzip, br, zstd, deflate"
HTTP_HOST	"info.umar.mx"
PATH	"/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/snap/bin"
SERVER_SIGNATURE	"<address>Apache/2.4.66 (Ubuntu) Server at info.umar.mx Port 443</address>\n"
SERVER_SOFTWARE	"Apache/2.4.66 (Ubuntu)"
SERVER_NAME	"info.umar.mx"
SERVER_ADDR	"200.23.222.19"
SERVER_PORT	"443"
REMOTE_ADDR	"216.73.217.30"
DOCUMENT_ROOT	"/usr/local/vufind/public"
REQUEST_SCHEME	"https"
CONTEXT_PREFIX	""
CONTEXT_DOCUMENT_ROOT	"/usr/local/vufind/public"
SERVER_ADMIN	"admin@localhost"
SCRIPT_FILENAME	"/usr/local/vufind/public/index.php"
REMOTE_PORT	"31046"
REDIRECT_URL	"/Record/zenodo_https___doi_org_10_5281_zenodo_19259980/Details"
GATEWAY_INTERFACE	"CGI/1.1"
SERVER_PROTOCOL	"HTTP/1.1"
REQUEST_METHOD	"GET"
QUERY_STRING	""
REQUEST_URI	"/Record/zenodo_https___doi_org_10_5281_zenodo_19259980/Details"
SCRIPT_NAME	"/index.php"
PHP_SELF	"/index.php"
REQUEST_TIME_FLOAT	1784443394.1312
REQUEST_TIME	1784443394

Environment Variables empty

Registered Handlers

0. Whoops\Handler\PrettyPageHandler

Similar Items

Environment & details:

Environment & details: