# JavaScript Plugin System ## Table of Contents 1. [Overview](#overview) 2. [JavaScript Engine Options](#javascript-engine-options) 3. [Architecture](#architecture) 4. [Implementation](#implementation) 5. [Async Execution](#async-execution) 6. [Named Pin Access](#named-pin-access) 7. [Security & Sandboxing](#security--sandboxing) --- ## Overview JavaScript plugins allow users to write custom blocks without recompiling. This is powerful for rapid prototyping and user extensibility, similar to how Virtools allowed custom scripting. ### Key Requirements - ✅ Execute JavaScript code within blocks - ✅ Pass pin values as named variables to JS - ✅ Handle async operations (timers, network requests, etc.) - ✅ Sandbox execution for safety - ✅ Persist JS code with the graph - ✅ Provide debugging support ### Use Cases ```javascript // Example 1: Simple math operation // Inputs: a (Float), b (Float) // Outputs: result (Float) result = Math.sqrt(a * a + b * b); // Example 2: String processing // Inputs: text (String), prefix (String) // Outputs: output (String) output = prefix + ": " + text.toUpperCase(); // Example 3: Async HTTP request // Inputs: url (String) // Outputs: data (String), error (String) async function main() { try { const response = await fetch(url); data = await response.text(); } catch (e) { error = e.message; } } ``` --- ## JavaScript Engine Options ### Comparison Matrix | Engine | Size | Speed | Async Support | Embedding Ease | License | |--------|------|-------|---------------|----------------|---------| | **QuickJS** | ~600KB | Good | ✅ Native async/await | ⭐⭐⭐⭐⭐ Easy | MIT | | **Duktape** | ~200KB | Medium | ❌ (needs wrapper) | ⭐⭐⭐⭐ Easy | MIT | | **V8** | ~20MB | Excellent | ✅ Native | ⭐⭐ Complex | BSD-3 | | **ChakraCore** | ~8MB | Excellent | ✅ Native | ⭐⭐⭐ Medium | MIT | | **MuJS** | ~100KB | Basic | ❌ No | ⭐⭐⭐⭐⭐ Very Easy | ISC | ### Recommendation: QuickJS **QuickJS** is the sweet spot for this use case: - Small footprint (~600KB compiled) - Full ES2020 support with async/await - Easy C API - Good performance for scripting - MIT license **Repository**: https://github.com/bellard/quickjs ### Alternative: Duktape (Simpler, No Async) If you don't need async support, **Duktape** is even simpler: - Tiny footprint (~200KB) - Easy single-header integration - Stable API - Good for simple scripting **Repository**: https://github.com/svaarala/duktape --- ## Architecture ### Block Architecture with JS Support ```mermaid graph TB subgraph "User Block" JSBlock[JavaScript Block] Code[JS Code String] Pins[Named Pins] end subgraph "Execution Layer" BlockBase[BlockBase] Context[ExecutionContext] JSEngine[JS Engine Instance] end subgraph "JavaScript VM" Runtime[JS Runtime] VMContext[JS Context] Globals[Global Objects] AsyncQueue[Async Task Queue] end JSBlock --> Code JSBlock --> Pins JSBlock --> BlockBase BlockBase --> Context Context --> JSEngine JSEngine --> Runtime Runtime --> VMContext VMContext --> Globals VMContext --> AsyncQueue Pins -.->|bind as variables| Globals AsyncQueue -.->|resolve| Context ``` ### Execution Flow ```mermaid sequenceDiagram participant Graph as GraphExecutor participant Block as JSBlock participant Engine as JS Engine participant VM as QuickJS Runtime Graph->>Block: Execute(ctx) Block->>Engine: PrepareContext() Engine->>VM: Create isolated context Block->>Engine: SetVariable("inputA", value) Block->>Engine: SetVariable("inputB", value) Engine->>VM: JS_SetPropertyStr(ctx, globals, "inputA", val) Block->>Engine: Execute(code) Engine->>VM: JS_Eval(ctx, code) alt Synchronous VM-->>Engine: Return immediately Engine->>Engine: GetVariable("output") Engine-->>Block: Result value Block->>Graph: SetOutput(0, result) else Asynchronous VM-->>Engine: Return promise Engine->>Engine: Queue for next frame Note over Block: Block marked as "running" loop Next frames Graph->>Block: Execute(ctx) Block->>Engine: PollAsyncTasks() Engine->>VM: JS_ExecutePendingJob() alt Promise resolved Engine->>Engine: GetVariable("output") Engine-->>Block: Result value Block->>Graph: SetOutput(0, result) Note over Block: Block marked as "complete" else Still pending Note over Block: Continue waiting end end end ``` --- ## Implementation ### JavaScript Engine Wrapper First, create a wrapper around QuickJS: ```cpp // JSEngine.h #pragma once #include "quickjs.h" #include #include #include #include using JSValue_t = std::variant; class JSEngine { public: JSEngine(); ~JSEngine(); // Context management void CreateContext(); void DestroyContext(); void ResetContext(); // Variable management void SetGlobalVariable(const std::string& name, const JSValue_t& value); JSValue_t GetGlobalVariable(const std::string& name); bool HasGlobalVariable(const std::string& name) const; // Execution bool ExecuteScript(const std::string& code, std::string& errorOut); bool ExecuteScriptAsync(const std::string& code, std::string& errorOut); // Async support bool HasPendingJobs() const; bool ExecutePendingJobs(); // Returns true if more jobs remain // Error handling std::string GetLastError() const { return m_LastError; } // Utility void RegisterFunction(const std::string& name, std::function&)> func); private: JSRuntime* m_Runtime; JSContext* m_Context; std::string m_LastError; // Convert between C++ and JS types JSValue ToJSValue(const JSValue_t& value); JSValue_t FromJSValue(JSValue val); // Registered native functions std::map&)>> m_Functions; }; ``` ### JSEngine Implementation (QuickJS) ```cpp // JSEngine.cpp #include "JSEngine.h" #include JSEngine::JSEngine() : m_Runtime(nullptr) , m_Context(nullptr) { m_Runtime = JS_NewRuntime(); if (!m_Runtime) { throw std::runtime_error("Failed to create JS runtime"); } // Set memory limit (optional) JS_SetMemoryLimit(m_Runtime, 64 * 1024 * 1024); // 64MB CreateContext(); } JSEngine::~JSEngine() { DestroyContext(); if (m_Runtime) { JS_FreeRuntime(m_Runtime); } } void JSEngine::CreateContext() { if (m_Context) { DestroyContext(); } m_Context = JS_NewContext(m_Runtime); if (!m_Context) { throw std::runtime_error("Failed to create JS context"); } // Add standard library support JS_AddIntrinsicBaseObjects(m_Context); JS_AddIntrinsicDate(m_Context); JS_AddIntrinsicEval(m_Context); JS_AddIntrinsicJSON(m_Context); JS_AddIntrinsicPromise(m_Context); // Optional: Add Math, RegExp, etc. // JS_AddIntrinsicMath(m_Context); } void JSEngine::DestroyContext() { if (m_Context) { JS_FreeContext(m_Context); m_Context = nullptr; } } void JSEngine::ResetContext() { CreateContext(); } void JSEngine::SetGlobalVariable(const std::string& name, const JSValue_t& value) { if (!m_Context) return; JSValue jsVal = ToJSValue(value); JSValue global = JS_GetGlobalObject(m_Context); JS_SetPropertyStr(m_Context, global, name.c_str(), jsVal); JS_FreeValue(m_Context, global); } JSValue_t JSEngine::GetGlobalVariable(const std::string& name) { if (!m_Context) return 0.0f; JSValue global = JS_GetGlobalObject(m_Context); JSValue jsVal = JS_GetPropertyStr(m_Context, global, name.c_str()); JS_FreeValue(m_Context, global); JSValue_t result = FromJSValue(jsVal); JS_FreeValue(m_Context, jsVal); return result; } bool JSEngine::ExecuteScript(const std::string& code, std::string& errorOut) { if (!m_Context) { errorOut = "No JS context"; return false; } JSValue result = JS_Eval(m_Context, code.c_str(), code.size(), "", JS_EVAL_TYPE_GLOBAL); if (JS_IsException(result)) { JSValue exception = JS_GetException(m_Context); const char* errStr = JS_ToCString(m_Context, exception); errorOut = errStr ? errStr : "Unknown error"; m_LastError = errorOut; JS_FreeCString(m_Context, errStr); JS_FreeValue(m_Context, exception); JS_FreeValue(m_Context, result); return false; } JS_FreeValue(m_Context, result); return true; } bool JSEngine::HasPendingJobs() const { if (!m_Runtime) return false; return JS_IsJobPending(m_Runtime) > 0; } bool JSEngine::ExecutePendingJobs() { if (!m_Runtime || !m_Context) return false; JSContext* ctx; int ret = JS_ExecutePendingJob(m_Runtime, &ctx); if (ret < 0) { // Error occurred if (ctx) { JSValue exception = JS_GetException(ctx); const char* errStr = JS_ToCString(ctx, exception); m_LastError = errStr ? errStr : "Unknown async error"; JS_FreeCString(ctx, errStr); JS_FreeValue(ctx, exception); } return false; } return ret > 0; // More jobs remain } JSValue JSEngine::ToJSValue(const JSValue_t& value) { if (std::holds_alternative(value)) { return JS_NewFloat64(m_Context, std::get(value)); } else if (std::holds_alternative(value)) { return JS_NewInt32(m_Context, std::get(value)); } else if (std::holds_alternative(value)) { return JS_NewBool(m_Context, std::get(value)); } else if (std::holds_alternative(value)) { const auto& str = std::get(value); return JS_NewString(m_Context, str.c_str()); } return JS_UNDEFINED; } JSValue_t JSEngine::FromJSValue(JSValue val) { if (JS_IsBool(val)) { return static_cast(JS_ToBool(m_Context, val)); } else if (JS_IsNumber(val)) { double d; JS_ToFloat64(m_Context, &d, val); // Check if it's an integer if (d == static_cast(d)) { return static_cast(d); } return static_cast(d); } else if (JS_IsString(val)) { const char* str = JS_ToCString(m_Context, val); std::string result = str ? str : ""; JS_FreeCString(m_Context, str); return result; } return 0.0f; // Default } ``` ### JavaScript Block Class Now create a block that executes JavaScript: ```cpp // JavaScriptBlock.h #pragma once #include "BlockBase.h" #include "JSEngine.h" #include enum class JSBlockState { Idle, // Not executing Running, // Sync execution in progress Pending, // Async operation pending Complete, // Async operation complete Error // Execution error }; class JavaScriptBlock : public BlockBase { public: JavaScriptBlock(); virtual ~JavaScriptBlock() = default; static const char* StaticGetName() { return "JavaScript"; } // BlockBase overrides const char* GetName() const override { return StaticGetName(); } const char* GetCategory() const override { return "Scripting"; } ImColor GetColor() const override { return ImColor(240, 220, 130); } bool HasFlowPins() const override { return m_HasFlowControl; } std::vector GetInputPins() const override; std::vector GetOutputPins() const override; void Execute(ExecutionContext& ctx) override; // Custom UI void OnDrawProperties() override; // Persistence void SaveCustomState(json& data) const override; void LoadCustomState(const json& data) override; // Configuration void SetScript(const std::string& script) { m_Script = script; } const std::string& GetScript() const { return m_Script; } void AddInputPin(const std::string& name, PinType type); void AddOutputPin(const std::string& name, PinType type); void RemoveInputPin(size_t index); void RemoveOutputPin(size_t index); // State query JSBlockState GetState() const { return m_State; } const std::string& GetLastError() const { return m_LastError; } private: void UpdatePinDefinitions(); void PrepareJSContext(const ExecutionContext& ctx); void ExtractOutputs(ExecutionContext& ctx); // Data std::string m_Script; std::vector m_InputDefs; std::vector m_OutputDefs; // Execution std::unique_ptr m_Engine; JSBlockState m_State; std::string m_LastError; // Configuration bool m_HasFlowControl; bool m_IsAsync; bool m_ShowCodeEditor; }; ``` ### JavaScript Block Implementation ```cpp // JavaScriptBlock.cpp #include "JavaScriptBlock.h" #include JavaScriptBlock::JavaScriptBlock() : m_Engine(std::make_unique()) , m_State(JSBlockState::Idle) , m_HasFlowControl(false) , m_IsAsync(false) , m_ShowCodeEditor(false) { // Default: simple math block m_InputDefs = { { "a", PinType::Float, PinKind::Input }, { "b", PinType::Float, PinKind::Input } }; m_OutputDefs = { { "result", PinType::Float, PinKind::Output } }; m_Script = "// Calculate result\nresult = a + b;"; } std::vector JavaScriptBlock::GetInputPins() const { auto pins = m_InputDefs; // Add flow pin if needed if (m_HasFlowControl) { pins.insert(pins.begin(), { "", PinType::Flow, PinKind::Input }); } return pins; } std::vector JavaScriptBlock::GetOutputPins() const { auto pins = m_OutputDefs; // Add flow pin if needed if (m_HasFlowControl) { pins.insert(pins.begin(), { "", PinType::Flow, PinKind::Output }); } return pins; } void JavaScriptBlock::Execute(ExecutionContext& ctx) { // Handle async completion if (m_State == JSBlockState::Pending) { if (m_Engine->HasPendingJobs()) { m_Engine->ExecutePendingJobs(); return; // Still pending, continue next frame } // Async completed - extract results m_State = JSBlockState::Complete; ExtractOutputs(ctx); m_State = JSBlockState::Idle; return; } // Normal execution m_State = JSBlockState::Running; m_LastError.clear(); try { // Prepare JS context with input values PrepareJSContext(ctx); // Execute script std::string error; bool success = m_Engine->ExecuteScript(m_Script, error); if (!success) { m_State = JSBlockState::Error; m_LastError = error; std::cerr << "JS Error: " << error << std::endl; return; } // Check if async if (m_Engine->HasPendingJobs()) { m_State = JSBlockState::Pending; // Outputs will be extracted when promise resolves } else { // Synchronous - extract outputs immediately ExtractOutputs(ctx); m_State = JSBlockState::Idle; } } catch (const std::exception& e) { m_State = JSBlockState::Error; m_LastError = e.what(); } } void JavaScriptBlock::PrepareJSContext(const ExecutionContext& ctx) { // Set input values as global variables size_t inputOffset = m_HasFlowControl ? 1 : 0; for (size_t i = 0; i < m_InputDefs.size(); ++i) { const auto& pinDef = m_InputDefs[i]; size_t ctxIndex = i + inputOffset; // Get value from context based on type JSValue_t value; switch (pinDef.type) { case PinType::Float: value = ctx.GetInput(ctxIndex); break; case PinType::Int: value = ctx.GetInput(ctxIndex); break; case PinType::Bool: value = ctx.GetInput(ctxIndex); break; case PinType::String: value = ctx.GetInput(ctxIndex); break; default: value = 0.0f; } // Set as JS global variable with pin name m_Engine->SetGlobalVariable(pinDef.name, value); } } void JavaScriptBlock::ExtractOutputs(ExecutionContext& ctx) { size_t outputOffset = m_HasFlowControl ? 1 : 0; for (size_t i = 0; i < m_OutputDefs.size(); ++i) { const auto& pinDef = m_OutputDefs[i]; size_t ctxIndex = i + outputOffset; // Get value from JS global variable if (!m_Engine->HasGlobalVariable(pinDef.name)) { continue; // Not set by script } auto value = m_Engine->GetGlobalVariable(pinDef.name); // Set output based on type switch (pinDef.type) { case PinType::Float: if (std::holds_alternative(value)) { ctx.SetOutput(ctxIndex, std::get(value)); } else if (std::holds_alternative(value)) { ctx.SetOutput(ctxIndex, static_cast(std::get(value))); } break; case PinType::Int: if (std::holds_alternative(value)) { ctx.SetOutput(ctxIndex, std::get(value)); } else if (std::holds_alternative(value)) { ctx.SetOutput(ctxIndex, static_cast(std::get(value))); } break; case PinType::Bool: if (std::holds_alternative(value)) { ctx.SetOutput(ctxIndex, std::get(value)); } break; case PinType::String: if (std::holds_alternative(value)) { ctx.SetOutput(ctxIndex, std::get(value)); } break; default: break; } } } void JavaScriptBlock::OnDrawProperties() { // Status indicator const char* stateStr = "Idle"; ImVec4 stateColor = ImVec4(0.5f, 0.5f, 0.5f, 1.0f); switch (m_State) { case JSBlockState::Idle: stateStr = "Idle"; stateColor = ImVec4(0.5f, 0.5f, 0.5f, 1.0f); break; case JSBlockState::Running: stateStr = "Running"; stateColor = ImVec4(0.3f, 0.8f, 0.3f, 1.0f); break; case JSBlockState::Pending: stateStr = "Pending..."; stateColor = ImVec4(0.9f, 0.7f, 0.2f, 1.0f); break; case JSBlockState::Complete: stateStr = "Complete"; stateColor = ImVec4(0.3f, 0.8f, 0.3f, 1.0f); break; case JSBlockState::Error: stateStr = "Error"; stateColor = ImVec4(0.9f, 0.2f, 0.2f, 1.0f); break; } ImGui::TextColored(stateColor, "State: %s", stateStr); if (m_State == JSBlockState::Error && !m_LastError.empty()) { ImGui::TextColored(ImVec4(1, 0, 0, 1), "Error: %s", m_LastError.c_str()); } // Code editor button if (ImGui::Button(m_ShowCodeEditor ? "Hide Code" : "Edit Code")) { m_ShowCodeEditor = !m_ShowCodeEditor; } if (m_ShowCodeEditor) { ImGui::Spacing(); // Multi-line code editor char buffer[4096]; strncpy(buffer, m_Script.c_str(), sizeof(buffer) - 1); buffer[sizeof(buffer) - 1] = '\0'; ImGui::PushItemWidth(300.0f); if (ImGui::InputTextMultiline("##code", buffer, sizeof(buffer), ImVec2(-1, 200))) { m_Script = buffer; m_Engine->ResetContext(); // Reset on code change } ImGui::PopItemWidth(); // Configuration ImGui::Checkbox("Has Flow Control", &m_HasFlowControl); ImGui::Checkbox("Async Execution", &m_IsAsync); } } void JavaScriptBlock::SaveCustomState(json& data) const { data["script"] = m_Script; data["hasFlowControl"] = m_HasFlowControl; data["isAsync"] = m_IsAsync; // Save pin definitions json inputs = json::array(); for (const auto& pin : m_InputDefs) { inputs.push_back({ {"name", pin.name}, {"type", static_cast(pin.type)} }); } data["inputs"] = inputs; json outputs = json::array(); for (const auto& pin : m_OutputDefs) { outputs.push_back({ {"name", pin.name}, {"type", static_cast(pin.type)} }); } data["outputs"] = outputs; } void JavaScriptBlock::LoadCustomState(const json& data) { if (data.contains("script")) { m_Script = data["script"].get(); } if (data.contains("hasFlowControl")) { m_HasFlowControl = data["hasFlowControl"].get(); } if (data.contains("isAsync")) { m_IsAsync = data["isAsync"].get(); } // Load pin definitions if (data.contains("inputs") && data["inputs"].is_array()) { m_InputDefs.clear(); for (const auto& pin : data["inputs"]) { m_InputDefs.push_back({ pin["name"].get(), static_cast(pin["type"].get()), PinKind::Input }); } } if (data.contains("outputs") && data["outputs"].is_array()) { m_OutputDefs.clear(); for (const auto& pin : data["outputs"]) { m_OutputDefs.push_back({ pin["name"].get(), static_cast(pin["type"].get()), PinKind::Output }); } } } void JavaScriptBlock::AddInputPin(const std::string& name, PinType type) { m_InputDefs.push_back({ name, type, PinKind::Input }); UpdatePinDefinitions(); } void JavaScriptBlock::AddOutputPin(const std::string& name, PinType type) { m_OutputDefs.push_back({ name, type, PinKind::Output }); UpdatePinDefinitions(); } ``` --- ## Async Execution ### Challenge: JavaScript is Async, Graph Execution is Sync The graph executor runs synchronously, but JavaScript operations might be async (fetch, setTimeout, etc.). We need to handle this mismatch. ### Solution 1: Polling Approach The block stays in `Pending` state while async operations complete: ```mermaid stateDiagram-v2 [*] --> Idle Idle --> Running: Execute() called Running --> Idle: Sync script complete Running --> Pending: Async operation started Pending --> Pending: ExecutePendingJobs() - still waiting Pending --> Complete: Promise resolved Complete --> Idle: Outputs extracted Running --> Error: Exception thrown Pending --> Error: Promise rejected Error --> Idle: Reset ``` ### Implementation in GraphExecutor ```cpp class GraphExecutor { public: void Execute(const std::vector>& blocks, const std::vector& links) { // Build execution order auto order = TopologicalSort(blocks, links); // Execute blocks for (auto nodeId : order) { auto* block = FindBlock(blocks, nodeId); if (!block) continue; // Check if it's a JS block with pending async auto* jsBlock = dynamic_cast(block); if (jsBlock && jsBlock->GetState() == JSBlockState::Pending) { // Skip - let it continue in background continue; } // Gather inputs from connected links ExecutionContext ctx; GatherInputs(ctx, block, links); // Execute block->Execute(ctx); // Store outputs for downstream blocks StoreOutputs(block, ctx); } } // Call this every frame to process async blocks void UpdateAsyncBlocks(const std::vector>& blocks) { for (auto& block : blocks) { auto* jsBlock = dynamic_cast(block.get()); if (jsBlock && jsBlock->GetState() == JSBlockState::Pending) { ExecutionContext ctx; // Re-execute to poll for completion jsBlock->Execute(ctx); if (jsBlock->GetState() == JSBlockState::Complete) { // Async completed - trigger downstream execution TriggerDownstreamExecution(jsBlock->GetNodeId()); } } } } private: void TriggerDownstreamExecution(ed::NodeId nodeId); // ... other methods }; ``` ### Solution 2: Callback Approach Alternative: Use callbacks to notify when async completes: ```cpp class JavaScriptBlock : public BlockBase { public: using AsyncCallback = std::function; void SetAsyncCallback(AsyncCallback callback) { m_AsyncCallback = callback; } void Execute(ExecutionContext& ctx) override { // ... prepare context ... if (m_IsAsync) { // Execute async m_Engine->ExecuteScriptAsync(m_Script, error); // Register completion handler m_Engine->OnComplete([this, ctx]() mutable { ExtractOutputs(ctx); // Notify via callback if (m_AsyncCallback) { m_AsyncCallback(ctx); } }); m_State = JSBlockState::Pending; } else { // Synchronous execution m_Engine->ExecuteScript(m_Script, error); ExtractOutputs(ctx); m_State = JSBlockState::Idle; } } private: AsyncCallback m_AsyncCallback; }; ``` --- ## Named Pin Access ### Making Pins Available as JS Variables The key insight: **Pin names become JavaScript variable names**. ```cpp void JavaScriptBlock::PrepareJSContext(const ExecutionContext& ctx) { size_t inputOffset = m_HasFlowControl ? 1 : 0; // Set each input pin value as a named variable for (size_t i = 0; i < m_InputDefs.size(); ++i) { const auto& pinDef = m_InputDefs[i]; // Get value from execution context JSValue_t value; switch (pinDef.type) { case PinType::Float: value = ctx.GetInput(i + inputOffset); break; case PinType::Int: value = ctx.GetInput(i + inputOffset); break; case PinType::Bool: value = ctx.GetInput(i + inputOffset); break; case PinType::String: value = ctx.GetInput(i + inputOffset); break; } // Bind to JS with pin name m_Engine->SetGlobalVariable(pinDef.name, value); } // Pre-declare output variables (so they're writable) for (const auto& pinDef : m_OutputDefs) { m_Engine->SetGlobalVariable(pinDef.name, JSValue_t(0.0f)); } } ``` ### Example: User Creates Custom Block ``` User creates JS block with: - Input pins: "velocity" (Float), "deltaTime" (Float) - Output pin: "distance" (Float) JavaScript code: distance = velocity * deltaTime; When executed: 1. Graph executor calls Execute(ctx) 2. ctx has input values [10.5, 0.016] 3. PrepareJSContext creates JS variables: - velocity = 10.5 - deltaTime = 0.016 - distance = 0.0 (pre-declared) 4. JS executes: distance = 10.5 * 0.016 5. ExtractOutputs reads JS variable "distance" = 0.168 6. Sets ctx output[0] = 0.168 ``` ### Advanced: Object and Array Support For complex data types: ```cpp // Extend JSEngine to support objects void JSEngine::SetGlobalObject(const std::string& name, const std::map& obj) { JSValue jsObj = JS_NewObject(m_Context); for (const auto& [key, value] : obj) { JSValue jsVal = ToJSValue(value); JS_SetPropertyStr(m_Context, jsObj, key.c_str(), jsVal); } JSValue global = JS_GetGlobalObject(m_Context); JS_SetPropertyStr(m_Context, global, name.c_str(), jsObj); JS_FreeValue(m_Context, global); } // Usage: Pass complex objects to JS void PrepareJSContext(const ExecutionContext& ctx) { // ... set simple values ... // Set complex object if (pinDef.type == PinType::Object) { auto objPtr = ctx.GetInput(i); auto* entity = static_cast(objPtr); // Expose entity properties to JS std::map jsEntity = { { "x", entity->position.x }, { "y", entity->position.y }, { "z", entity->position.z }, { "name", entity->name }, { "id", entity->id } }; m_Engine->SetGlobalObject(pinDef.name, jsEntity); } } ``` ### Example JS Code with Named Pins ```javascript // Block with inputs: player (Object), enemy (Object) // outputs: distance (Float), inRange (Bool) // Access object properties via pin names const dx = player.x - enemy.x; const dy = player.y - enemy.y; const dz = player.z - enemy.z; // Calculate distance distance = Math.sqrt(dx*dx + dy*dy + dz*dz); // Check range const attackRange = 5.0; inRange = distance < attackRange; ``` --- ## Async Execution Patterns ### Pattern 1: Timer/Delay ```javascript // Block: Delay // Input: duration (Float) // Output: completed (Bool) async function main() { await sleep(duration * 1000); completed = true; } // Native sleep function exposed by JSEngine function sleep(ms) { return new Promise(resolve => setTimeout(resolve, ms)); } ``` **Implementation**: Expose `setTimeout` to QuickJS: ```cpp // In JSEngine constructor void JSEngine::RegisterBuiltinFunctions() { // Register setTimeout RegisterFunction("setTimeout", [this](const std::vector& args) { // args[0] = callback function // args[1] = delay in ms // Implementation would use a timer queue return JSValue_t(0); // Return timer ID }); } ``` ### Pattern 2: HTTP Request ```javascript // Block: HTTP Get // Input: url (String) // Outputs: data (String), statusCode (Int), error (String) async function main() { try { const response = await fetch(url); statusCode = response.status; data = await response.text(); error = ""; } catch (e) { error = e.message; statusCode = 0; } } ``` **Implementation**: Expose `fetch` API: ```cpp void JSEngine::RegisterFetch() { // Use a C++ HTTP library (libcurl, cpp-httplib, etc.) RegisterAsyncFunction("fetch", [](const std::string& url) { return std::async(std::launch::async, [url]() { // Perform HTTP request auto response = httplib::Client("...").Get(url); // Return as JSValue_t return JSValue_t(response->body); }); }); } ``` ### Pattern 3: Multiple Async Operations ```javascript // Block: Parallel Requests // Inputs: url1 (String), url2 (String) // Outputs: combined (String) async function main() { const [response1, response2] = await Promise.all([ fetch(url1), fetch(url2) ]); const data1 = await response1.text(); const data2 = await response2.text(); combined = data1 + "\n---\n" + data2; } ``` --- ## Security & Sandboxing ### Threat Model JavaScript blocks can potentially: - ❌ Access file system - ❌ Make network requests to malicious sites - ❌ Consume excessive CPU/memory - ❌ Access other blocks' data ### Mitigation Strategies #### 1. Disable Dangerous APIs ```cpp void JSEngine::CreateSandboxedContext() { m_Context = JS_NewContext(m_Runtime); // Enable only safe intrinsics JS_AddIntrinsicBaseObjects(m_Context); JS_AddIntrinsicDate(m_Context); JS_AddIntrinsicJSON(m_Context); JS_AddIntrinsicPromise(m_Context); // DO NOT ADD: // - JS_AddIntrinsicEval() - prevents dynamic code execution // - File I/O modules // - Process/OS modules JSValue global = JS_GetGlobalObject(m_Context); // Remove dangerous globals JS_SetPropertyStr(m_Context, global, "eval", JS_UNDEFINED); JS_SetPropertyStr(m_Context, global, "Function", JS_UNDEFINED); JS_FreeValue(m_Context, global); } ``` #### 2. Execution Timeout ```cpp class JSEngine { public: void SetExecutionTimeout(int milliseconds) { m_TimeoutMs = milliseconds; } bool ExecuteScript(const std::string& code, std::string& errorOut) { // Set interrupt handler JS_SetInterruptHandler(m_Runtime, InterruptHandler, this); m_StartTime = std::chrono::steady_clock::now(); JSValue result = JS_Eval(m_Context, code.c_str(), code.size(), "", JS_EVAL_TYPE_GLOBAL); if (JS_IsException(result)) { // Handle error... return false; } JS_FreeValue(m_Context, result); return true; } private: static int InterruptHandler(JSRuntime* rt, void* opaque) { auto* self = static_cast(opaque); auto now = std::chrono::steady_clock::now(); auto elapsed = std::chrono::duration_cast( now - self->m_StartTime ).count(); // Interrupt if timeout exceeded return (elapsed > self->m_TimeoutMs) ? 1 : 0; } int m_TimeoutMs = 5000; // 5 second default std::chrono::steady_clock::time_point m_StartTime; }; ``` #### 3. Memory Limits ```cpp void JSEngine::CreateContext() { // Set memory limit on runtime JS_SetMemoryLimit(m_Runtime, 64 * 1024 * 1024); // 64MB // Set max stack size JS_SetMaxStackSize(m_Runtime, 1024 * 1024); // 1MB stack m_Context = JS_NewContext(m_Runtime); } ``` #### 4. Whitelisted Network Access ```cpp void JSEngine::RegisterFetch() { RegisterAsyncFunction("fetch", [this](const std::string& url) { // Whitelist check if (!IsURLAllowed(url)) { throw std::runtime_error("URL not whitelisted: " + url); } return std::async(std::launch::async, [url]() { // Perform request with timeout httplib::Client client("host"); client.set_connection_timeout(5); // 5 seconds client.set_read_timeout(10); // 10 seconds auto response = client.Get(url); return response ? JSValue_t(response->body) : JSValue_t(""); }); }); } bool JSEngine::IsURLAllowed(const std::string& url) { // Check against whitelist for (const auto& pattern : m_AllowedURLPatterns) { if (url.find(pattern) == 0) { return true; } } return false; } ``` --- ## Advanced Features ### Hot Reload Allow editing and re-running scripts without restarting: ```cpp void JavaScriptBlock::OnDrawProperties() { // ... code editor ... if (ImGui::Button("Test Run")) { // Create temporary context for testing ExecutionContext testCtx; // Set dummy input values for (size_t i = 0; i < m_InputDefs.size(); ++i) { testCtx.SetInput(i, GetDummyValue(m_InputDefs[i].type)); } // Execute Execute(testCtx); // Show results in UI m_ShowTestResults = true; } if (m_ShowTestResults) { ImGui::Text("Test Results:"); // Display output values... } } ``` ### Debugging Support Add breakpoint and console.log support: ```cpp void JSEngine::RegisterConsoleLog() { // Expose console.log to JS JSValue console = JS_NewObject(m_Context); // Create log function auto logFunc = [](JSContext* ctx, JSValueConst this_val, int argc, JSValueConst* argv) -> JSValue { for (int i = 0; i < argc; ++i) { const char* str = JS_ToCString(ctx, argv[i]); std::cout << "[JS] " << (str ? str : "(null)"); if (i < argc - 1) std::cout << " "; JS_FreeCString(ctx, str); } std::cout << std::endl; return JS_UNDEFINED; }; JSValue logFuncVal = JS_NewCFunction(m_Context, logFunc, "log", 1); JS_SetPropertyStr(m_Context, console, "log", logFuncVal); JSValue global = JS_GetGlobalObject(m_Context); JS_SetPropertyStr(m_Context, global, "console", console); JS_FreeValue(m_Context, global); } // Usage in JS: // console.log("Debug: velocity =", velocity, "distance =", distance); ``` ### Code Templates Provide templates for common patterns: ```cpp class JSBlockTemplates { public: static std::vector> GetTemplates() { return { {"Simple Math", "// Inputs: a, b\n// Output: result\nresult = a + b;"}, {"Vector Distance", "// Inputs: x1, y1, x2, y2\n" "// Output: distance\n" "const dx = x2 - x1;\n" "const dy = y2 - y1;\n" "distance = Math.sqrt(dx*dx + dy*dy);"}, {"Async Delay", "// Input: seconds\n" "// Output: done\n" "async function main() {\n" " await sleep(seconds * 1000);\n" " done = true;\n" "}"}, {"HTTP Request", "// Input: url\n" "// Outputs: data, error\n" "async function main() {\n" " try {\n" " const response = await fetch(url);\n" " data = await response.text();\n" " error = '';\n" " } catch (e) {\n" " error = e.message;\n" " }\n" "}"} }; } }; // UI for template selection void JavaScriptBlock::OnDrawProperties() { if (ImGui::BeginCombo("Template", "Select...")) { for (const auto& [name, code] : JSBlockTemplates::GetTemplates()) { if (ImGui::Selectable(name.c_str())) { m_Script = code; m_Engine->ResetContext(); } } ImGui::EndCombo(); } // ... rest of UI } ``` --- ## Complete Example: HTTP Fetch Block ```cpp // HTTPFetchBlock.h class HTTPFetchBlock : public JavaScriptBlock { public: HTTPFetchBlock() { // Configure as async block m_HasFlowControl = true; m_IsAsync = true; // Define pins m_InputDefs = { { "url", PinType::String, PinKind::Input }, { "method", PinType::String, PinKind::Input }, { "body", PinType::String, PinKind::Input } }; m_OutputDefs = { { "response", PinType::String, PinKind::Output }, { "statusCode", PinType::Int, PinKind::Output }, { "error", PinType::String, PinKind::Output } }; // Default script m_Script = R"( async function main() { try { const options = { method: method || 'GET' }; if (body && method !== 'GET') { options.body = body; } const res = await fetch(url, options); statusCode = res.status; response = await res.text(); error = ''; } catch (e) { error = e.message; statusCode = 0; response = ''; } } main(); )"; } static const char* StaticGetName() { return "HTTP Fetch"; } const char* GetName() const override { return StaticGetName(); } const char* GetCategory() const override { return "Network"; } ImColor GetColor() const override { return ImColor(150, 220, 180); } }; // Register void RegisterNetworkBlocks() { BlockRegistry::Get().Register("Network"); } ``` --- ## Alternative: Duktape Implementation If you prefer simpler (no async) implementation: ```cpp // JSEngine.h (Duktape version) #include "duktape.h" class JSEngine { public: JSEngine() { m_Context = duk_create_heap_default(); } ~JSEngine() { if (m_Context) { duk_destroy_heap(m_Context); } } void SetGlobalVariable(const std::string& name, const JSValue_t& value) { if (std::holds_alternative(value)) { duk_push_number(m_Context, std::get(value)); } else if (std::holds_alternative(value)) { duk_push_int(m_Context, std::get(value)); } else if (std::holds_alternative(value)) { duk_push_boolean(m_Context, std::get(value)); } else if (std::holds_alternative(value)) { duk_push_string(m_Context, std::get(value).c_str()); } duk_put_global_string(m_Context, name.c_str()); } JSValue_t GetGlobalVariable(const std::string& name) { duk_get_global_string(m_Context, name.c_str()); if (duk_is_number(m_Context, -1)) { double val = duk_get_number(m_Context, -1); duk_pop(m_Context); return static_cast(val); } else if (duk_is_boolean(m_Context, -1)) { bool val = duk_get_boolean(m_Context, -1); duk_pop(m_Context); return val; } else if (duk_is_string(m_Context, -1)) { const char* str = duk_get_string(m_Context, -1); std::string result = str ? str : ""; duk_pop(m_Context); return result; } duk_pop(m_Context); return 0.0f; } bool ExecuteScript(const std::string& code, std::string& errorOut) { if (duk_peval_string(m_Context, code.c_str()) != 0) { errorOut = duk_safe_to_string(m_Context, -1); duk_pop(m_Context); return false; } duk_pop(m_Context); return true; } private: duk_context* m_Context; }; ``` **Pros**: Much simpler, smaller binary **Cons**: No native async/await support --- ## Integration Checklist ### Step 1: Add JS Engine Dependency **For QuickJS**: ```cmake # CMakeLists.txt add_subdirectory(extern/quickjs) target_link_libraries(YourApp quickjs) ``` **For Duktape**: ```cmake # Just include duktape.c and duktape.h in your project add_library(duktape extern/duktape/duktape.c) target_link_libraries(YourApp duktape) ``` ### Step 2: Implement JSEngine Wrapper - [ ] Create `JSEngine.h` and `JSEngine.cpp` - [ ] Implement `ToJSValue()` and `FromJSValue()` converters - [ ] Add error handling - [ ] Implement async job polling (QuickJS only) - [ ] Add timeout/interrupt handler ### Step 3: Create JavaScriptBlock - [ ] Inherit from `BlockBase` - [ ] Implement `PrepareJSContext()` - bind named pins - [ ] Implement `ExtractOutputs()` - read named variables - [ ] Add code editor UI in `OnDrawProperties()` - [ ] Handle async state machine ### Step 4: Register Native Functions - [ ] `console.log()` for debugging - [ ] `setTimeout()` / `setInterval()` for timers - [ ] `fetch()` for HTTP (optional) - [ ] Custom domain functions (e.g., `getEntity()`, `playSound()`) ### Step 5: GraphExecutor Integration - [ ] Modify executor to handle `JSBlockState::Pending` - [ ] Call `UpdateAsyncBlocks()` every frame - [ ] Trigger downstream execution on async completion ### Step 6: UI Enhancements - [ ] Syntax highlighting (use ImGuiColorTextEdit) - [ ] Auto-completion for pin names - [ ] Error markers in code - [ ] Template library --- ## Complete Working Example Here's a minimal but complete JavaScript block: ```cpp // MinimalJSBlock.cpp #include "BlockBase.h" #include "quickjs.h" class SimpleJSBlock : public BlockBase { public: SimpleJSBlock() { m_Runtime = JS_NewRuntime(); m_Context = JS_NewContext(m_Runtime); JS_AddIntrinsicBaseObjects(m_Context); } ~SimpleJSBlock() { JS_FreeContext(m_Context); JS_FreeRuntime(m_Runtime); } const char* GetName() const override { return "JS Math"; } const char* GetCategory() const override { return "Script"; } std::vector GetInputPins() const override { return { { "a", PinType::Float, PinKind::Input }, { "b", PinType::Float, PinKind::Input } }; } std::vector GetOutputPins() const override { return { { "result", PinType::Float, PinKind::Output } }; } void Execute(ExecutionContext& ctx) override { // Set inputs float a = ctx.GetInput(0); float b = ctx.GetInput(1); JSValue global = JS_GetGlobalObject(m_Context); JS_SetPropertyStr(m_Context, global, "a", JS_NewFloat64(m_Context, a)); JS_SetPropertyStr(m_Context, global, "b", JS_NewFloat64(m_Context, b)); // Execute: result = a * a + b * b; JSValue ret = JS_Eval(m_Context, m_Script.c_str(), m_Script.size(), "", JS_EVAL_TYPE_GLOBAL); if (JS_IsException(ret)) { std::cerr << "JS Error!" << std::endl; JS_FreeValue(m_Context, ret); JS_FreeValue(m_Context, global); return; } // Get output JSValue resultVal = JS_GetPropertyStr(m_Context, global, "result"); double result; JS_ToFloat64(m_Context, &result, resultVal); ctx.SetOutput(0, static_cast(result)); // Cleanup JS_FreeValue(m_Context, resultVal); JS_FreeValue(m_Context, ret); JS_FreeValue(m_Context, global); } void OnDrawProperties() override { char buffer[1024]; strncpy(buffer, m_Script.c_str(), sizeof(buffer) - 1); if (ImGui::InputTextMultiline("##code", buffer, sizeof(buffer), ImVec2(250, 100))) { m_Script = buffer; } } void SaveCustomState(json& data) const override { data["script"] = m_Script; } void LoadCustomState(const json& data) override { if (data.contains("script")) { m_Script = data["script"].get(); } } private: JSRuntime* m_Runtime; JSContext* m_Context; std::string m_Script = "result = a * a + b * b;"; }; ``` Compile and link with QuickJS, and you have a working JS block! --- ## Best Practices ### 1. Pin Naming Conventions Use JavaScript-friendly names: ```cpp ✅ Good: "velocity", "deltaTime", "targetPosition" ❌ Bad: "Velocity In", "Delta-Time", "target position" ``` Variables in JS must be valid identifiers (no spaces, start with letter/underscore). ### 2. Error Reporting Show errors directly in the node: ```cpp void JavaScriptBlock::OnDrawProperties() override { if (m_State == JSBlockState::Error) { ImGui::PushStyleColor(ImGuiCol_Text, ImVec4(1, 0, 0, 1)); ImGui::TextWrapped("Error: %s", m_LastError.c_str()); ImGui::PopStyleColor(); if (ImGui::Button("Clear Error")) { m_State = JSBlockState::Idle; m_LastError.clear(); } } } ``` ### 3. Type Coercion JavaScript is loosely typed. Handle conversions: ```cpp JSValue_t JSEngine::FromJSValue(JSValue val) { // Try to preserve type information if (JS_IsNumber(val)) { double d; JS_ToFloat64(m_Context, &d, val); // If integer-like, return as int if (d == std::floor(d) && d >= INT_MIN && d <= INT_MAX) { return static_cast(d); } return static_cast(d); } // ... other types } ``` ### 4. Async Best Practices - Always use `async function main()` wrapper for async code - Set reasonable timeouts for network operations - Provide visual feedback (spinner, progress) during pending state - Allow cancellation of long-running async operations ### 5. Memory Management - Create new context per execution or per block instance - Use `JS_RunGC()` periodically to free memory - Set memory limits appropriate for your use case - Profile memory usage during development --- ## Performance Considerations ### Optimization Strategies 1. **Context Reuse**: Reuse JS context instead of recreating ```cpp // Good: Reuse context void Execute(ExecutionContext& ctx) override { if (!m_Initialized) { m_Engine->CreateContext(); m_Engine->RegisterBuiltinFunctions(); m_Initialized = true; } // Execute... } ``` 2. **Pre-compile Scripts**: QuickJS supports bytecode compilation ```cpp void JavaScriptBlock::CompileScript() { // Compile to bytecode size_t bytecodeSize; uint8_t* bytecode = JS_WriteObject(m_Context, &bytecodeSize, compiledFunc, JS_WRITE_OBJ_BYTECODE); // Store bytecode for faster execution m_CompiledBytecode.assign(bytecode, bytecode + bytecodeSize); js_free(m_Context, bytecode); } ``` 3. **Limit Execution Frequency**: Don't run every frame ```cpp void Execute(ExecutionContext& ctx) override { // Only execute every N frames or on trigger if (!m_TriggerPin->HasSignal()) return; // Execute... } ``` --- ## Resources ### Libraries - **QuickJS**: https://github.com/bellard/quickjs - **Duktape**: https://duktape.org/ - **QuickJS C++ Wrapper**: https://github.com/ftk/quickjspp ### Code Editor Integration For better code editing UI, consider: - **ImGuiColorTextEdit**: https://github.com/BalazsJako/ImGuiColorTextEdit - Syntax highlighting - Line numbers - Error markers - Auto-indent ### Additional Reading - QuickJS Documentation: https://bellard.org/quickjs/ - Embedding JS engines: https://v8.dev/docs/embed - Async patterns in C++: https://en.cppreference.com/w/cpp/thread/async --- ## Summary ### The Complete Picture ``` User writes JS code with named pins → JavaScriptBlock stores code → On Execute: Bind pin values as JS variables → Run JS (sync or async) → Extract output variables → Set outputs in ExecutionContext → Graph continues execution ``` ### Key Benefits ✅ **No recompilation** - Edit scripts at runtime ✅ **User extensibility** - Users create custom blocks ✅ **Named pins = variables** - Intuitive scripting ✅ **Async support** - Handle network, timers, etc. ✅ **Sandboxed** - Safe execution with limits ✅ **Persistent** - Scripts saved with graph ### Recommended Approach 1. Start with **QuickJS** for full async support 2. Use **BlockBase** architecture from `overview.md` 3. Implement **async polling** in `GraphExecutor` 4. Add **console.log** and basic debugging 5. Provide **code templates** for common patterns 6. Gradually add **native functions** as needed Good luck with your JavaScript plugin system! 🚀