ZH

← Externals

文档 — Externals

Externals — big game data streamed from the playground

Some games need data that cannot be in their public repository nor be downloaded when they start: the CD version of a game, a collection of levels, films. Externals are files the administrator of the JumPHP playground keeps on the server (the folder externals/, outside every repository) and shares with chosen games (Administration page → Externals). The game browses them, searches them and reads them like a disk: only the 64 KB blocks it reads are downloaded (HTTP Range), and they stay in a cache (memory, and the player's storage: IndexedDB in the browser), so a file read once is not downloaded again.

bin/jumphp composer <game> require jumphp/externals

Use

Externals is a service: ask for it in a scene's constructor. Everything answers with a Task (then(fn ($result) => ..., fn (\Throwable $e) => ...)), in a later frame; nothing blocks the game.

use JumPHP\Externals\{Externals, Entry, RemoteFile};

public function __construct(private Externals $externals) {}

$this->externals->list();                                   // the folders shared with this game (list<Entry>)
$this->externals->list('dmmotor/wads', recursive: true);    // what is in one (sub-folders too)
$this->externals->search('*.wad');                          // by name: a pattern, or a part of the name
$this->externals->stat('dmmotor/wads/mod.wad');             // Entry: path, type, size, mtime, etag

$this->externals->open('dmmotor/wads/mod.wad')->then(function (RemoteFile $f) {
    $f->read(0, 12)->then(fn (string $header) => ...);       // these bytes only (their block)
    $bytes = $f->readNow(4096, 16);                          // at once when cached, else null (and asked for)
    $f->prefetch(0, 1 << 20);                                // ask for the first MB, nothing comes back
    $f->readAll(fn (float $p) => $this->bar = $p)           // the whole file, with its progress
      ->then(fn (string $data) => ...);
    $f->cachedRatio();                                       // 0..1 of the file in the cache
});

$this->externals->pending();      // requests running: show "LOADING"
$this->externals->downloaded();   // bytes downloaded since the start
$this->externals->forProject('other-game');   // the externals shared with another project of the playground

readNow() suits a game loop that reads often (an emulator, an engine reading its data file): when it returns null, show "loading" and try again in the next frame. Consecutive blocks go in one request (up to 16), at most parallel requests at a time.

Where the files come from

  • On the playground (the browser): url: auto is /<project id>/play/externals/ on the page's own host (set url: /<slug>/play/externals/ when the project's slug differs from its id). Only the folders the administrator shared with this project answer (else ExternalsError 403).
  • On the desktop, in tests, or with a local copy: local maps externals folders to folders of the computer (relative to the game). The same paths work everywhere: dmmotor/wads/mod.wad is read from wads/mod.wad here.

project.yaml

externals:
  url: auto                       # auto = /<project id>/play/externals/ on the playground (browser); or an https:// URL
  local: { "dmmotor/wads": "wads" } # externals folder => folder of the game (desktop, tests); takes precedence
  block: 65536                    # bytes per block
  memory_mb: 32                   # memory cache
  persist: true                   # keep the downloaded blocks in var/ (IndexedDB in the browser: in memory there)
  parallel: 4                     # requests at the same time

The server (the playground)

The administrator keeps the files in the server's externals folder (mounted read-only into the playground) and shares a folder with projects in Administration → Externals. The HTTP protocol (all on the game's own host):

Request Answer
GET <slug>/play/externals/~list?dir=<path>&recursive=0|1 {"dir", "entries": [{"path", "type": "file"|"dir", "size", "mtime"}]} (dir empty: the shared folders)
GET <slug>/play/externals/~search?q=<text or pattern>&dir=<path> the same, files only
GET <slug>/play/externals/~stat?path=<path> {"path", "type", "size", "mtime", "etag"}
GET <slug>/play/externals/<path>?v=<etag> + Range: bytes=a-b 206 with those bytes (?v= = the version from ~stat: kept for good by the browser's HTTP cache, immutable; without it or with an old one: 5 minutes)

In the browser the blocks are also kept by the browser's own HTTP cache (on disk, outside the engine's memory), from one session to the next: a game with big files (films) can set persist: false and still not download them twice.

Not shared: 403; missing: 404. Hidden files (a name starting with ".") are never shown. Anyone can read a shared file with its address (like the game's own files): share only what may be downloaded.

Samples

  • BrowseSample: the shared folders and their entries (list()).
  • SearchSample: searching by name (search()).
  • ReadSample: a file read by blocks, then all of it with its progress (read(), readAll(), cachedRatio()).

Externals 教程