MishkaCmsCore.Runtime.Compilers.CompiledModuleCache (Mishka CMS Core v0.0.1)

Keeps the runtime modules the CMS builds — pages, layouts, error pages and components — as .beam files on disk, so a node that restarts, or is redeployed with the same code, loads a module it built before instead of compiling it again.

Off unless it is given a directory. MishkaCmsCore.Runtime.Handlers.ModuleCompilerHandler asks for it through ModuleCompiler.compile_module/3; nothing else is cached.

Turning it on

config :mishka_cms_core, :compiled_module_cache,
  dir: Path.join(System.fetch_env!("MISHKA_DATA_DIR"), "compiled_modules"),
  max_age_days: 7

:dir — where the files go. Point it at storage that outlives the release: in Docker, the persistent data volume. Missing or nil, the cache is off and a module is compiled exactly as it is without this module: no file is read or written and nothing is hashed.
:max_age_days — how long the files of an earlier build are kept after it last ran, so a rollback to it still finds them. Defaults to 7.

The directory is created mode 0700 and every file in it 0600. A directory this node cannot make 0700 is not used at all: the files in it are code the node runs.

What a file is found by

The SHA-256 of the module's name, the file name it is compiled as, the quoted code ModuleCompiler is handed (see canonical/1), and the build that compiles it (generation/0): the Elixir, OTP and ERTS versions, the versions of :phoenix, :phoenix_live_view and :mishka_cms_core, and the MD5 of every module of the host application and of every application it depends on. A row that changed, a code generator that changed, or a macro, struct or import any generated module expands from that changed, files it under another key — there is nothing to invalidate by hand.

Configuration is not part of the key. A module reads the application environment when it runs, as it does without the cache; a value read while it compiles — Application.compile_env/3 in a helper's code, say — stays the one it was saved with until its row changes. Empty the directory after changing such a value.

What a file holds

The binary the compiler produced, with one more chunk, MCMC, carrying the key it was filed under and a SHA-256 of every other chunk. A file is loaded only when its chunks read, the module it holds is the one asked for, and both of those match. Anything else — a truncated or flipped file, one holding another module, one the code server refuses — is deleted, and the module is compiled.

Where it is on disk

<dir>/<generation id>/<module>/<key>.beam

A module has one file per build. Writing a new one deletes the one before it, and ModuleCompiler.unload/1 deletes the module's folder. The first time a node uses the cache it deletes the folders of other builds that have not run for :max_age_days, at most 16 of them, in a task of its own. Nothing runs on a timer.

Plugin events

runtime:compile:cache_hit — %{module: module, key: key}, a module loaded from its file.
runtime:compile:cache_miss — %{module: module, key: key, reason: reason}, a module that is compiled because there was no usable file. reason is :absent, :unreadable, :corrupt, :wrong_module or :load_failed.

Summary

Types

entry()

Where one module's file is, and the key it is filed under.

generation()

The build the cache files modules under: id names its folder.

miss()

Why a module was compiled instead of loaded.

Functions

canonical(chain)

quoted with the two things HEEx writes differently from one node to the next made the same, so the same template is found under the same key after a restart.

dir()

The configured directory, or nil when the cache is off.

entry(dir, module, file, quoted)

Where the module quoted defines, compiled as file, is filed in dir, or :off when it cannot be cached there: a module name that is not a plain dotted alias, or a directory this node could not make 0700.

forget(module)

Deletes module's files of the running build, when the cache is on and this node has used it. Called by ModuleCompiler.unload/1.

generation()

The build this node compiles modules with, worked out once per node: its digest is part of every key and its id names the folder its files go in.

key(module, file, quoted, digest)

The SHA-256 a module is filed under: of digest (a generation/0), the module's name, the file name it is compiled as and the quoted code that defines it, as canonical/1 writes it.

load(entry, file)

Loads the module from its file, as ModuleCompiler loads one it compiled: the version before the loaded one is purged first. :ok, or {:miss, reason} when it has to be compiled — and then a file that was there but could not be used is gone.

store(map, binary)

Writes binary, the module just compiled for entry, to its file, and deletes the module's file from before. A file that cannot be written is logged and left out; the module is loaded either way.

Types

entry()

@type entry() :: %{module: module(), key: binary(), folder: Path.t(), path: Path.t()}

Where one module's file is, and the key it is filed under.

generation()

@type generation() :: %{id: String.t(), digest: binary()}

The build the cache files modules under: id names its folder.

miss()

@type miss() :: :absent | :unreadable | :corrupt | :wrong_module | :load_failed

Why a module was compiled instead of loaded.

Functions

canonical(chain)

@spec canonical(Macro.t()) :: term()

quoted with the two things HEEx writes differently from one node to the next made the same, so the same template is found under the same key after a restart.

Phoenix.LiveView.Engine decides whether a part of a template is drawn again by a chain of changed_assign?/2 and nested_changed_assign?/4 checks joined with or, in the order it reads them out of a map — and a map of more than 32 assigns is read in an order that differs between nodes. The fingerprint of the %Phoenix.LiveView.Rendered{} or %Phoenix.LiveView.Comprehension{} holding that chain is a hash of it, so it differs too. Here the checks of such a chain are put in one order and the fingerprint left out: the checks are side-effect free, so their order does not change what the module does, and the fingerprint is a function of the code beside it.

Only the key is computed from this; the module is compiled from quoted as it is.

dir()

@spec dir() :: Path.t() | nil

The configured directory, or nil when the cache is off.

Examples

# config :mishka_cms_core, :compiled_module_cache, dir: "/data/compiled_modules"
dir()
#=> "/data/compiled_modules"

# no :compiled_module_cache, or dir: nil
dir()
#=> nil

entry(dir, module, file, quoted)

@spec entry(Path.t(), module(), String.t(), Macro.t()) :: {:ok, entry()} | :off

Hashes quoted. The first call in a node also works out generation/0 and readies dir.

forget(module)

@spec forget(module()) :: :ok

Deletes module's files of the running build, when the cache is on and this node has used it. Called by ModuleCompiler.unload/1.

generation()

@spec generation() :: generation()

The build this node compiles modules with, worked out once per node: its digest is part of every key and its id names the folder its files go in.

In a release every module is already loaded and this takes a few milliseconds. Started with mix, the modules not loaded yet are read from their .beam files, which can take a second.

key(module, file, quoted, digest)

@spec key(module(), String.t(), Macro.t(), binary()) :: binary()

The SHA-256 a module is filed under: of digest (a generation/0), the module's name, the file name it is compiled as and the quoted code that defines it, as canonical/1 writes it.

load(entry, file)

@spec load(entry(), String.t()) :: :ok | {:miss, miss()}

store(map, binary)

@spec store(entry(), binary()) :: :ok

Writes binary, the module just compiled for entry, to its file, and deletes the module's file from before. A file that cannot be written is logged and left out; the module is loaded either way.