문서 — Externals
Externals — big game data streamed from the playground
Some games need data that cannot be in their public repository nor be downloaded when they start: the CD version
of a game, a collection of levels, films. Externals are files the administrator of the JumPHP playground keeps
on the server (the folder externals/, outside every repository) and shares with chosen games (Administration
page → Externals). The game browses them, searches them and reads them like a disk: only the 64 KB blocks it
reads are downloaded (HTTP Range), and they stay in a cache (memory, and the player's storage: IndexedDB in the
browser), so a file read once is not downloaded again.
bin/jumphp composer <game> require jumphp/externals
Use
Externals is a service: ask for it in a scene's constructor. Everything answers with a Task
(then(fn ($result) => ..., fn (\Throwable $e) => ...)), in a later frame; nothing blocks the game.
use JumPHP\Externals\{Externals, Entry, RemoteFile};
public function __construct(private Externals $externals) {}
$this->externals->list(); // the folders shared with this game (list<Entry>)
$this->externals->list('dmmotor/wads', recursive: true); // what is in one (sub-folders too)
$this->externals->search('*.wad'); // by name: a pattern, or a part of the name
$this->externals->stat('dmmotor/wads/mod.wad'); // Entry: path, type, size, mtime, etag
$this->externals->open('dmmotor/wads/mod.wad')->then(function (RemoteFile $f) {
$f->read(0, 12)->then(fn (string $header) => ...); // these bytes only (their block)
$bytes = $f->readNow(4096, 16); // at once when cached, else null (and asked for)
$f->prefetch(0, 1 << 20); // ask for the first MB, nothing comes back
$f->readAll(fn (float $p) => $this->bar = $p) // the whole file, with its progress
->then(fn (string $data) => ...);
$f->cachedRatio(); // 0..1 of the file in the cache
});
$this->externals->pending(); // requests running: show "LOADING"
$this->externals->downloaded(); // bytes downloaded since the start
$this->externals->forProject('other-game'); // the externals shared with another project of the playground
readNow() suits a game loop that reads often (an emulator, an engine reading its data file): when it returns null,
show "loading" and try again in the next frame. Consecutive blocks go in one request (up to 16), at most parallel
requests at a time.
Where the files come from
- On the playground (the browser):
url: autois/<project id>/play/externals/on the page's own host (seturl: /<slug>/play/externals/when the project's slug differs from its id). Only the folders the administrator shared with this project answer (elseExternalsError403). - On the desktop, in tests, or with a local copy:
localmaps externals folders to folders of the computer (relative to the game). The same paths work everywhere:dmmotor/wads/mod.wadis read fromwads/mod.wadhere.
project.yaml
externals:
url: auto # auto = /<project id>/play/externals/ on the playground (browser); or an https:// URL
local: { "dmmotor/wads": "wads" } # externals folder => folder of the game (desktop, tests); takes precedence
block: 65536 # bytes per block
memory_mb: 32 # memory cache
persist: true # keep the downloaded blocks in var/ (IndexedDB in the browser: in memory there)
parallel: 4 # requests at the same time
The server (the playground)
The administrator keeps the files in the server's externals folder (mounted read-only into the playground) and shares a folder with projects in Administration → Externals. The HTTP protocol (all on the game's own host):
| Request | Answer |
|---|---|
GET <slug>/play/externals/~list?dir=<path>&recursive=0|1 |
{"dir", "entries": [{"path", "type": "file"|"dir", "size", "mtime"}]} (dir empty: the shared folders) |
GET <slug>/play/externals/~search?q=<text or pattern>&dir=<path> |
the same, files only |
GET <slug>/play/externals/~stat?path=<path> |
{"path", "type", "size", "mtime", "etag"} |
GET <slug>/play/externals/<path>?v=<etag> + Range: bytes=a-b |
206 with those bytes (?v= = the version from ~stat: kept for good by the browser's HTTP cache, immutable; without it or with an old one: 5 minutes) |
In the browser the blocks are also kept by the browser's own HTTP cache (on disk, outside the engine's memory), from
one session to the next: a game with big files (films) can set persist: false and still not download them twice.
Not shared: 403; missing: 404. Hidden files (a name starting with ".") are never shown. Anyone can read a shared file with its address (like the game's own files): share only what may be downloaded.
Samples
BrowseSample: the shared folders and their entries (list()).SearchSample: searching by name (search()).ReadSample: a file read by blocks, then all of it with its progress (read(),readAll(),cachedRatio()).