# `MishkaCmsCore.Runtime.Handlers.ModuleCompilerHandler` [🔗](https://github.com/mishka-group/mishka_cms_core/blob/v0.0.1/lib/mishka_cms_core/runtime/handlers/module_compiler_handler.ex#L5) Dynamic module compiler handler for MishkaCms runtime components with concurrent compilation management. The `ModuleCompilerHandler` provides a sophisticated system for compiling Phoenix LiveView modules, components, layouts, and error pages on-demand. It manages concurrent compilation processes, prevents duplicate work, and ensures efficient resource utilization through partitioned dynamic supervision and registry-based process coordination. ## Features * **Concurrent Compilation Management**: Prevents duplicate compilation work using process registry * **Dynamic Supervision**: Distributes compilation work across partitioned supervisors for scalability * **Module Type Detection**: Automatically detects and compiles different module types (Page, Component, Layout, Error) * **Timeout Management**: Provides configurable timeouts to prevent hanging compilation processes * **Database Integration**: Seamlessly integrates with Ash-based data layer for content retrieval * **Compiler Abstraction**: Uses specialized compilers for different content types * **Process Lifecycle Management**: Handles temporary workers with automatic cleanup ## Architecture Overview The handler operates within a distributed architecture: ``` ModuleCompilerHandler (Temporary GenServer) ├── Registry-based Process Coordination ├── Partitioned Dynamic Supervision ├── Type-specific Compilers │ ├── LiveViewCompiler (Pages) │ ├── ComponentCompiler (Components) │ ├── LayoutCompiler (Layouts) │ └── ErrorCompiler (Error Pages) └── ModuleCompiler (AST → Runtime Module) ``` ## Compilation Process Flow 1. **Request Handling**: Receives compilation request with module name and metadata 2. **Registry Check**: Verifies if compilation is already in progress 3. **Process Spawning**: Creates temporary GenServer worker if needed 4. **Database Query**: Retrieves content data based on module type and path 5. **AST Generation**: Uses type-specific compiler to generate Abstract Syntax Tree 6. **Module Compilation**: Compiles AST into runtime Elixir module 7. **Cleanup**: Automatically terminates worker after completion ## Basic Usage # Compile or return existing module case ModuleCompilerHandler.compile_or_return_module( MyApp.Generated.Page123, "/blog/article", "my_site", site_id ) do {:ok, module} -> # Module is compiled and ready to use {:ok, _pid} = DynamicSupervisor.start_child( MyApp.PageSupervisor, {Phoenix.LiveView, module} ) {:error, reason} -> Logger.error("Compilation failed: #{inspect(reason)}") end ## Production Considerations * **Memory Management**: Compiled modules remain in memory until explicitly purged * **Concurrency**: Uses all available CPU cores through partitioned supervision * **Timeout Configuration**: Default 1-minute timeout prevents hanging processes * **Process Cleanup**: Temporary workers automatically terminate after compilation * **Registry Overhead**: Minimal overhead for process coordination and deduplication * **Database Load**: Each compilation triggers database queries for content retrieval ## Module Type Support The handler supports compilation for: * **Pages**: Phoenix LiveView pages with full interactivity * **Components**: Reusable UI components with attributes and slots * **Layouts**: Page layouts with resource links and SEO metadata * **Error Pages**: Custom error handling pages with fallback layouts # `boot_compile_or_return_module` **This is helper function to boot compile or return module, do not use it directly!!** # `child_spec` Generates child specification for temporary compilation workers. This function creates a child specification used by DynamicSupervisor to spawn temporary GenServer workers for module compilation. Each worker is configured with a unique identifier based on module name and site to prevent conflicts. ## Parameters * `{module_name, path, site, site_id, type}` - Tuple containing: * `module_name` - Atom representing the target module name, or `{module_name, record}` when the record is already in hand (boot) * `path` - String path or identifier for content lookup * `site` - String site name for multi-tenant support * `site_id` - Site identifier for database queries * `type` - `:new` or `:update`, as `compile_or_return_module/5` takes it ## Return Value Returns a child specification map with: * `id` - Unique identifier combining module name and site * `start` - Start function tuple for GenServer initialization * `restart` - `:temporary` for automatic cleanup after completion * `type` - `:worker` process type # `classify_compile_error` ```elixir @spec classify_compile_error(Exception.t()) :: :compile_transient | {:compilation_error, String.t()} ``` Classifies an exception raised while building/compiling a runtime module as either a cacheable compile failure or a transient (retryable) infrastructure error. Genuine template errors (`SyntaxError`, HEEx token errors, `CompileError`, …) become `{:compilation_error, message}` and are cached in `CompileFailureState` so the hot path skips the broken module. Database/pool blips (`DBConnection.ConnectionError`, `Postgrex.Error`) become `:compile_transient`, which is NEVER cached — the next request retries instead of being permanently served a 500. Everything else is treated as a compile error so an unknown template failure still gets cached (fail-safe). ## Examples classify_compile_error(%DBConnection.ConnectionError{message: "pool timeout"}) #=> :compile_transient classify_compile_error(%RuntimeError{message: "unclosed tag"}) #=> {:compilation_error, "unclosed tag"} # `compile_or_return_module` Compiles a module or returns existing compiled module with intelligent coordination. This is the primary entry point for module compilation. It implements sophisticated coordination logic to prevent duplicate compilation work while ensuring modules are available when needed. The function handles both new compilation and updates with different strategies based on module state and compilation type. ## Parameters * `module_name` - Atom representing the target module name * `path` - String path or identifier used for content database lookup * `site` - String site name for multi-tenant content isolation * `site_id` - Integer site identifier for database queries * `type` - Compilation type (defaults to `:new`): * `:new` - Skip compilation if module already loaded * `:update` - Force recompilation even if module exists ## Return Values * `{:ok, module_name}` - Module compiled successfully or already available * `{:error, :not_found}` - Content not found in database * `{:error, :compilation_error}` - Compilation failed due to syntax/logic errors * `{:error, :compilation_timeout}` - Compilation exceeded timeout limit * `{:error, reason}` - Other compilation failures ## Coordination Strategy A module is compiled once however many callers ask for it at the same time: 1. **Module Existence Check**: For `:new` type, returns immediately if module loaded 2. **Registry Lookup**: A worker already registered for the module — compiling, or still waiting for a `CompileGate` slot — is waited for, and its answer is this caller's too 3. **One Worker**: Otherwise one is started; it registers before it waits for a slot, so callers after this one wait for it rather than queue at the gate for the same module 4. **One Message**: The worker's exit carries its answer, so every caller waiting on it is answered the moment it ends, whether or not the caller that started it is still there Waiting takes no gate slot: the slots are only ever held by compiles, so the pages of other sites keep compiling while many callers wait for one page. An `:update` takes the answer of a compile that began after it was asked for. One already running may have read the record before it changed, so it is waited out and the next one started, and the callers that waited together share that one compile rather than each compiling in turn. # `compiled_result` ```elixir @spec compiled_result(module()) :: {:ok, module()} | {:error, term()} ``` Resolves a module's compile outcome from already-recorded shared state — the loaded-code table plus `CompileFailureState` — without contacting the compiling worker process. Used when the worker a caller waited for ended without handing over its answer — it was gone before the caller could watch it, or it was killed. A worker records its outcome before it ends, so a caller reads it here instead. Loaded module → `{:ok, module}`; otherwise the recorded failure reason, or `:compilation_error` when the worker died before recording one. ## Examples # a module the winner already compiled/loaded compiled_result(MyApp.Runtime.SomePage) #=> {:ok, MyApp.Runtime.SomePage} # never loaded and no recorded failure compiled_result(MyApp.Runtime.NeverCompiled) #=> {:error, :compilation_error} # `get_data_from_database_to_ast` # `get_dynamic_supervisor_key` Selects a dynamic supervisor partition for load distribution. This function implements round-robin load balancing across partitioned dynamic supervisors to distribute compilation workers evenly. It uses an atomic counter to ensure thread-safe partition selection. ## Return Value * `{:via, PartitionSupervisor, {supervisor_name, partition_number}}` - Supervisor reference for worker spawning ## Load Balancing Strategy Uses atomic counter-based round-robin: 1. Increments global atomic counter 2. Maps counter value to partition number 3. Returns supervisor reference for selected partition ## Examples # Get supervisor for worker spawning supervisor = ModuleCompilerHandler.get_dynamic_supervisor_key() # Start child worker {:ok, pid} = DynamicSupervisor.start_child( supervisor, child_spec({module, path, site, site_id}) ) # Automatic load balancing across calls supervisors = for _ <- 1..10 do ModuleCompilerHandler.get_dynamic_supervisor_key() end # Returns different partitions in round-robin fashion ## Concurrency Safety # Thread-safe counter operations # Multiple processes can call simultaneously without coordination tasks = for i <- 1..100 do Task.async(fn -> ModuleCompilerHandler.get_dynamic_supervisor_key() end) end # Each task gets a properly balanced supervisor reference supervisor_refs = Task.await_many(tasks) ## Performance Characteristics * **Atomic Operations**: O(1) counter increment with minimal contention * **Memory Usage**: Single atomic counter shared across all processes * **Distribution**: Even spread across available partitions * **Scalability**: Efficient with high concurrent access ## Partition Health Impact # Load distribution affects partition health defmodule LoadTracker do def track_supervisor_usage do # Sample supervisor selection over time selections = for _ <- 1..1000 do ModuleCompilerHandler.get_dynamic_supervisor_key() end # Count selections per partition partition_counts = Enum.frequencies(selections) # Verify even distribution target_count = 1000 / ModuleCompilerHandler.partition_count() Enum.each(partition_counts, fn {partition, count} -> deviation = abs(count - target_count) / target_count if deviation > 0.1 do Logger.warning("Partition #{partition} load deviation: #{deviation}") end end) end end ## Integration with Worker Lifecycle defp start_and_compile(module_name, path, site, site_id) do # Get balanced supervisor for this worker supervisor = get_dynamic_supervisor_key() # Supervisor selection affects resource distribution case DynamicSupervisor.start_child( supervisor, child_spec({module_name, path, site, site_id}) ) do {:ok, pid} -> handle_worker_success(pid) {:error, reason} -> handle_worker_failure(reason) end end # `handle_continue` # `init` # `layout_params` What `LayoutCompiler.build_module/1` compiles `layout` from, as `layout_module`. `layout` is a `MishkaCmsCore.Runtime.Layout` or a `MishkaCmsCore.Runtime.LayoutDraft`, loaded with `:site`. # `page_params` What `LiveViewCompiler.build_module/1` compiles `page` from, as `page_module`. `page` is a `MishkaCmsCore.Runtime.Page` loaded with `:site` and `:loaded_layout`, or a `MishkaCmsCore.Runtime.PageDraft` loaded with `:site` and `layout: [:site]`. # `partition_count` Returns the number of partitions used for dynamic supervision. This function provides the partition count used by the PartitionSupervisor to distribute compilation workers. The count is based on the number of available CPU schedulers, ensuring optimal resource utilization. ## Return Value * `integer()` - Number of partitions (equal to CPU scheduler count) ## Examples # Get partition count count = ModuleCompilerHandler.partition_count() # Returns: 8 (on an 8-core system) # Use for load balancing decisions partition_load = total_workers / ModuleCompilerHandler.partition_count() if partition_load > 10 do Logger.warning("High compilation load detected") end ## Performance Implications The partition count directly affects: * **Concurrency**: Higher count enables more parallel compilations * **Resource Distribution**: Workers spread across CPU cores * **Memory Usage**: Each partition maintains separate worker pools * **Scheduling Efficiency**: Aligned with Erlang VM scheduler count ## Monitoring Integration # Track partition utilization defmodule PartitionMonitor do def check_load_balance do partition_count = ModuleCompilerHandler.partition_count() health_stats = ModuleCompilerHandler.partition_health_stats() max_workers = Enum.max_by(health_stats, & &1.workers).workers min_workers = Enum.min_by(health_stats, & &1.workers).workers imbalance_ratio = max_workers / max(min_workers, 1) if imbalance_ratio > 2.0 do Logger.warning("Partition load imbalance detected: #{imbalance_ratio}") end end end # `partition_health_stats` Retrieves health statistics for all dynamic supervisor partitions. This function provides comprehensive health metrics for each partition in the dynamic supervisor pool, enabling monitoring, debugging, and load balancing decisions. It queries each partition's child count and status information. ## Return Value Returns a list of maps, one per partition, with the following keys: * `active` - Total number of active processes * `specs` - Number of child specifications * `supervisors` - Number of supervisor children * `workers` - Number of worker processes ## Examples # Get current health stats stats = ModuleCompilerHandler.partition_health_stats() # Returns: [ # %{active: 3, specs: 3, supervisors: 0, workers: 3}, # %{active: 1, specs: 1, supervisors: 0, workers: 1}, # %{active: 0, specs: 0, supervisors: 0, workers: 0}, # %{active: 2, specs: 2, supervisors: 0, workers: 2} # ] # Calculate total workers across all partitions total_workers = stats |> Enum.map(& &1.workers) |> Enum.sum() IO.puts("Total compilation workers: #{total_workers}") # `start_link` Starts the worker that compiles one module, registered under the module's name. It is registered from the moment it starts — while it still waits for a `CompileGate` slot — so every caller that asks for the module meanwhile finds it and waits for its answer instead of compiling the module again. It compiles at once, without being asked, and stops when it is done; the reason it stops with carries its answer (see `compile_or_return_module/5`). ## Return Values * `{:ok, pid}` - Successfully started GenServer worker * `{:error, {:already_started, pid}}` - Worker already exists for this module * `{:error, reason}` - Failed to start worker --- *Consult [api-reference.md](api-reference.md) for complete listing*