MishkaCmsCore.Runtime.Handlers.ModuleCompilerHandler (Mishka CMS Core v0.0.1)

Copy Markdown View Source

Dynamic module compiler handler for MishkaCms runtime components with concurrent compilation management.

The ModuleCompilerHandler provides a sophisticated system for compiling Phoenix LiveView modules, components, layouts, and error pages on-demand. It manages concurrent compilation processes, prevents duplicate work, and ensures efficient resource utilization through partitioned dynamic supervision and registry-based process coordination.

Features

  • Concurrent Compilation Management: Prevents duplicate compilation work using process registry
  • Dynamic Supervision: Distributes compilation work across partitioned supervisors for scalability
  • Module Type Detection: Automatically detects and compiles different module types (Page, Component, Layout, Error)
  • Timeout Management: Provides configurable timeouts to prevent hanging compilation processes
  • Database Integration: Seamlessly integrates with Ash-based data layer for content retrieval
  • Compiler Abstraction: Uses specialized compilers for different content types
  • Process Lifecycle Management: Handles temporary workers with automatic cleanup

Architecture Overview

The handler operates within a distributed architecture:

ModuleCompilerHandler (Temporary GenServer)
├── Registry-based Process Coordination
├── Partitioned Dynamic Supervision
├── Type-specific Compilers
│   ├── LiveViewCompiler (Pages)
│   ├── ComponentCompiler (Components)
│   ├── LayoutCompiler (Layouts)
│   └── ErrorCompiler (Error Pages)
└── ModuleCompiler (AST → Runtime Module)

Compilation Process Flow

  1. Request Handling: Receives compilation request with module name and metadata
  2. Registry Check: Verifies if compilation is already in progress
  3. Process Spawning: Creates temporary GenServer worker if needed
  4. Database Query: Retrieves content data based on module type and path
  5. AST Generation: Uses type-specific compiler to generate Abstract Syntax Tree
  6. Module Compilation: Compiles AST into runtime Elixir module
  7. Cleanup: Automatically terminates worker after completion

Basic Usage

# Compile or return existing module
case ModuleCompilerHandler.compile_or_return_module(
  MyApp.Generated.Page123,
  "/blog/article",
  "my_site",
  site_id
) do
  {:ok, module} ->
    # Module is compiled and ready to use
    {:ok, _pid} = DynamicSupervisor.start_child(
      MyApp.PageSupervisor,
      {Phoenix.LiveView, module}
    )

  {:error, reason} ->
    Logger.error("Compilation failed: #{inspect(reason)}")
end

Production Considerations

  • Memory Management: Compiled modules remain in memory until explicitly purged
  • Concurrency: Uses all available CPU cores through partitioned supervision
  • Timeout Configuration: Default 1-minute timeout prevents hanging processes
  • Process Cleanup: Temporary workers automatically terminate after compilation
  • Registry Overhead: Minimal overhead for process coordination and deduplication
  • Database Load: Each compilation triggers database queries for content retrieval

Module Type Support

The handler supports compilation for:

  • Pages: Phoenix LiveView pages with full interactivity
  • Components: Reusable UI components with attributes and slots
  • Layouts: Page layouts with resource links and SEO metadata
  • Error Pages: Custom error handling pages with fallback layouts

Summary

Functions

This is helper function to boot compile or return module, do not use it directly!!

Generates child specification for temporary compilation workers.

Classifies an exception raised while building/compiling a runtime module as either a cacheable compile failure or a transient (retryable) infrastructure error.

Compiles a module or returns existing compiled module with intelligent coordination.

Resolves a module's compile outcome from already-recorded shared state — the loaded-code table plus CompileFailureState — without contacting the compiling worker process.

Selects a dynamic supervisor partition for load distribution.

Callback implementation for GenServer.init/1.

What LayoutCompiler.build_module/1 compiles layout from, as layout_module.

What LiveViewCompiler.build_module/1 compiles page from, as page_module.

Returns the number of partitions used for dynamic supervision.

Retrieves health statistics for all dynamic supervisor partitions.

Starts the worker that compiles one module, registered under the module's name.

Functions

boot_compile_or_return_module(module_and_item, path, site, site_id)

This is helper function to boot compile or return module, do not use it directly!!

child_spec(init_arg)

Generates child specification for temporary compilation workers.

This function creates a child specification used by DynamicSupervisor to spawn temporary GenServer workers for module compilation. Each worker is configured with a unique identifier based on module name and site to prevent conflicts.

Parameters

  • {module_name, path, site, site_id, type} - Tuple containing:
    • module_name - Atom representing the target module name, or {module_name, record} when the record is already in hand (boot)
    • path - String path or identifier for content lookup
    • site - String site name for multi-tenant support
    • site_id - Site identifier for database queries
    • type - :new or :update, as compile_or_return_module/5 takes it

Return Value

Returns a child specification map with:

  • id - Unique identifier combining module name and site
  • start - Start function tuple for GenServer initialization
  • restart - :temporary for automatic cleanup after completion
  • type - :worker process type

classify_compile_error(error)

@spec classify_compile_error(Exception.t()) ::
  :compile_transient | {:compilation_error, String.t()}

Classifies an exception raised while building/compiling a runtime module as either a cacheable compile failure or a transient (retryable) infrastructure error.

Genuine template errors (SyntaxError, HEEx token errors, CompileError, …) become {:compilation_error, message} and are cached in CompileFailureState so the hot path skips the broken module. Database/pool blips (DBConnection.ConnectionError, Postgrex.Error) become :compile_transient, which is NEVER cached — the next request retries instead of being permanently served a 500. Everything else is treated as a compile error so an unknown template failure still gets cached (fail-safe).

Examples

classify_compile_error(%DBConnection.ConnectionError{message: "pool timeout"})
#=> :compile_transient

classify_compile_error(%RuntimeError{message: "unclosed tag"})
#=> {:compilation_error, "unclosed tag"}

compile_or_return_module(module_name, path, site, site_id, type \\ :new)

Compiles a module or returns existing compiled module with intelligent coordination.

This is the primary entry point for module compilation. It implements sophisticated coordination logic to prevent duplicate compilation work while ensuring modules are available when needed. The function handles both new compilation and updates with different strategies based on module state and compilation type.

Parameters

  • module_name - Atom representing the target module name
  • path - String path or identifier used for content database lookup
  • site - String site name for multi-tenant content isolation
  • site_id - Integer site identifier for database queries
  • type - Compilation type (defaults to :new):
    • :new - Skip compilation if module already loaded
    • :update - Force recompilation even if module exists

Return Values

  • {:ok, module_name} - Module compiled successfully or already available
  • {:error, :not_found} - Content not found in database
  • {:error, :compilation_error} - Compilation failed due to syntax/logic errors
  • {:error, :compilation_timeout} - Compilation exceeded timeout limit
  • {:error, reason} - Other compilation failures

Coordination Strategy

A module is compiled once however many callers ask for it at the same time:

  1. Module Existence Check: For :new type, returns immediately if module loaded
  2. Registry Lookup: A worker already registered for the module — compiling, or still waiting for a CompileGate slot — is waited for, and its answer is this caller's too
  3. One Worker: Otherwise one is started; it registers before it waits for a slot, so callers after this one wait for it rather than queue at the gate for the same module
  4. One Message: The worker's exit carries its answer, so every caller waiting on it is answered the moment it ends, whether or not the caller that started it is still there

Waiting takes no gate slot: the slots are only ever held by compiles, so the pages of other sites keep compiling while many callers wait for one page.

An :update takes the answer of a compile that began after it was asked for. One already running may have read the record before it changed, so it is waited out and the next one started, and the callers that waited together share that one compile rather than each compiling in turn.

compiled_result(module)

@spec compiled_result(module()) :: {:ok, module()} | {:error, term()}

Resolves a module's compile outcome from already-recorded shared state — the loaded-code table plus CompileFailureState — without contacting the compiling worker process.

Used when the worker a caller waited for ended without handing over its answer — it was gone before the caller could watch it, or it was killed. A worker records its outcome before it ends, so a caller reads it here instead. Loaded module → {:ok, module}; otherwise the recorded failure reason, or :compilation_error when the worker died before recording one.

Examples

# a module the winner already compiled/loaded
compiled_result(MyApp.Runtime.SomePage)
#=> {:ok, MyApp.Runtime.SomePage}

# never loaded and no recorded failure
compiled_result(MyApp.Runtime.NeverCompiled)
#=> {:error, :compilation_error}

get_data_from_database_to_ast(arg)

get_dynamic_supervisor_key()

Selects a dynamic supervisor partition for load distribution.

This function implements round-robin load balancing across partitioned dynamic supervisors to distribute compilation workers evenly. It uses an atomic counter to ensure thread-safe partition selection.

Return Value

  • {:via, PartitionSupervisor, {supervisor_name, partition_number}} - Supervisor reference for worker spawning

Load Balancing Strategy

Uses atomic counter-based round-robin:

  1. Increments global atomic counter
  2. Maps counter value to partition number
  3. Returns supervisor reference for selected partition

Examples

# Get supervisor for worker spawning
supervisor = ModuleCompilerHandler.get_dynamic_supervisor_key()

# Start child worker
{:ok, pid} = DynamicSupervisor.start_child(
  supervisor,
  child_spec({module, path, site, site_id})
)

# Automatic load balancing across calls
supervisors = for _ <- 1..10 do
  ModuleCompilerHandler.get_dynamic_supervisor_key()
end
# Returns different partitions in round-robin fashion

Concurrency Safety

# Thread-safe counter operations
# Multiple processes can call simultaneously without coordination
tasks = for i <- 1..100 do
  Task.async(fn ->
    ModuleCompilerHandler.get_dynamic_supervisor_key()
  end)
end

# Each task gets a properly balanced supervisor reference
supervisor_refs = Task.await_many(tasks)

Performance Characteristics

  • Atomic Operations: O(1) counter increment with minimal contention
  • Memory Usage: Single atomic counter shared across all processes
  • Distribution: Even spread across available partitions
  • Scalability: Efficient with high concurrent access

Partition Health Impact

# Load distribution affects partition health
defmodule LoadTracker do
  def track_supervisor_usage do
    # Sample supervisor selection over time
    selections = for _ <- 1..1000 do
      ModuleCompilerHandler.get_dynamic_supervisor_key()
    end

    # Count selections per partition
    partition_counts = Enum.frequencies(selections)

    # Verify even distribution
    target_count = 1000 / ModuleCompilerHandler.partition_count()

    Enum.each(partition_counts, fn {partition, count} ->
      deviation = abs(count - target_count) / target_count

      if deviation > 0.1 do
        Logger.warning("Partition #{partition} load deviation: #{deviation}")
      end
    end)
  end
end

Integration with Worker Lifecycle

defp start_and_compile(module_name, path, site, site_id) do
  # Get balanced supervisor for this worker
  supervisor = get_dynamic_supervisor_key()

  # Supervisor selection affects resource distribution
  case DynamicSupervisor.start_child(
    supervisor,
    child_spec({module_name, path, site, site_id})
  ) do
    {:ok, pid} -> handle_worker_success(pid)
    {:error, reason} -> handle_worker_failure(reason)
  end
end

handle_continue(atom, state)

Callback implementation for GenServer.handle_continue/2.

init(arg)

Callback implementation for GenServer.init/1.

layout_params(layout, layout_module)

What LayoutCompiler.build_module/1 compiles layout from, as layout_module.

layout is a MishkaCmsCore.Runtime.Layout or a MishkaCmsCore.Runtime.LayoutDraft, loaded with :site.

page_params(page, page_module)

What LiveViewCompiler.build_module/1 compiles page from, as page_module.

page is a MishkaCmsCore.Runtime.Page loaded with :site and :loaded_layout, or a MishkaCmsCore.Runtime.PageDraft loaded with :site and layout: [:site].

partition_count()

Returns the number of partitions used for dynamic supervision.

This function provides the partition count used by the PartitionSupervisor to distribute compilation workers. The count is based on the number of available CPU schedulers, ensuring optimal resource utilization.

Return Value

  • integer() - Number of partitions (equal to CPU scheduler count)

Examples

# Get partition count
count = ModuleCompilerHandler.partition_count()
# Returns: 8 (on an 8-core system)

# Use for load balancing decisions
partition_load = total_workers / ModuleCompilerHandler.partition_count()

if partition_load > 10 do
  Logger.warning("High compilation load detected")
end

Performance Implications

The partition count directly affects:

  • Concurrency: Higher count enables more parallel compilations
  • Resource Distribution: Workers spread across CPU cores
  • Memory Usage: Each partition maintains separate worker pools
  • Scheduling Efficiency: Aligned with Erlang VM scheduler count

Monitoring Integration

# Track partition utilization
defmodule PartitionMonitor do
  def check_load_balance do
    partition_count = ModuleCompilerHandler.partition_count()
    health_stats = ModuleCompilerHandler.partition_health_stats()

    max_workers = Enum.max_by(health_stats, & &1.workers).workers
    min_workers = Enum.min_by(health_stats, & &1.workers).workers

    imbalance_ratio = max_workers / max(min_workers, 1)

    if imbalance_ratio > 2.0 do
      Logger.warning("Partition load imbalance detected: #{imbalance_ratio}")
    end
  end
end

partition_health_stats()

Retrieves health statistics for all dynamic supervisor partitions.

This function provides comprehensive health metrics for each partition in the dynamic supervisor pool, enabling monitoring, debugging, and load balancing decisions. It queries each partition's child count and status information.

Return Value

Returns a list of maps, one per partition, with the following keys:

  • active - Total number of active processes
  • specs - Number of child specifications
  • supervisors - Number of supervisor children
  • workers - Number of worker processes

Examples

# Get current health stats
stats = ModuleCompilerHandler.partition_health_stats()
# Returns: [
#   %{active: 3, specs: 3, supervisors: 0, workers: 3},
#   %{active: 1, specs: 1, supervisors: 0, workers: 1},
#   %{active: 0, specs: 0, supervisors: 0, workers: 0},
#   %{active: 2, specs: 2, supervisors: 0, workers: 2}
# ]

# Calculate total workers across all partitions
total_workers = stats
  |> Enum.map(& &1.workers)
  |> Enum.sum()

IO.puts("Total compilation workers: #{total_workers}")

start_link(args)

Starts the worker that compiles one module, registered under the module's name.

It is registered from the moment it starts — while it still waits for a CompileGate slot — so every caller that asks for the module meanwhile finds it and waits for its answer instead of compiling the module again. It compiles at once, without being asked, and stops when it is done; the reason it stops with carries its answer (see compile_or_return_module/5).

Return Values

  • {:ok, pid} - Successfully started GenServer worker
  • {:error, {:already_started, pid}} - Worker already exists for this module
  • {:error, reason} - Failed to start worker