Sign in Subscribe
By Denny —

Deep Understanding Google Blockly - Part 3: Code Generation


Deep Understanding Google Blockly - Part 3: Code Generation

A comprehensive technical deep-dive into Blockly's code generation system

Table of Contents

  1. Introduction
  2. Code Generation Architecture
  3. The CodeGenerator Base Class
  4. Block-to-Code Translation
  5. Operator Precedence
  6. Language-Specific Generators
  7. Writing Custom Generators
  8. Advanced Code Generation

Introduction

One of Blockly's most powerful features is its ability to generate executable code in multiple programming languages from the same visual blocks. This language-agnostic approach means:

  • Write once, generate anywhere - Same blocks → JavaScript, Python, Dart, Lua, PHP
  • Type-safe generation - Proper operator precedence and type handling
  • Customizable output - Add prefixes, suffixes, infinite loop traps
  • Human-readable code - Generated code follows language conventions

This article explores the sophisticated code generation system that makes this possible.

Code Generation Architecture

High-Level Flow

┌─────────────────────────────────────────┐
│          Workspace with Blocks          │
│     (Visual representation)             │
└─────────────────────────────────────────┘
                  ↓
         Generator.workspaceToCode()
                  ↓
┌─────────────────────────────────────────┐
│      1. GET TOP-LEVEL BLOCKS            │
│      workspace.getTopBlocks(true)       │
└─────────────────────────────────────────┘
                  ↓
┌─────────────────────────────────────────┐
│      2. INITIALIZE GENERATOR            │
│      generator.init(workspace)          │
│      - Create name database              │
│      - Reset definitions                 │
└─────────────────────────────────────────┘
                  ↓
┌─────────────────────────────────────────┐
│      3. GENERATE CODE FOR EACH BLOCK    │
│      for each block:                    │
│        generator.blockToCode(block)     │
│        - Recursive depth-first           │
│        - Returns code string             │
└─────────────────────────────────────────┘
                  ↓
┌─────────────────────────────────────────┐
│      4. ASSEMBLE FINAL CODE             │
│      - Join code blocks                 │
│      - Add definitions/imports           │
│      - Apply finish() cleanup           │
└─────────────────────────────────────────┘
                  ↓
┌─────────────────────────────────────────┐
│         Executable Code String          │
│     (JavaScript, Python, etc.)          │
└─────────────────────────────────────────┘

Key Components

Generator System (/generators/):

  • javascript/ - JavaScript code generator
  • python/ - Python code generator
  • dart/ - Dart code generator
  • lua/ - Lua code generator
  • php/ - PHP code generator

Core Generator (core/generator.ts):

  • CodeGenerator - Base class for all generators
  • Names - Variable/function name management
  • Order constants - Operator precedence definitions

The CodeGenerator Base Class

Core Structure

From core/generator.ts:

export class CodeGenerator {
  name_: string;
  
  // Block generator functions
  forBlock: Record<
    string,
    (block: Block, generator: this) => [string, number] | string | null
  > = Object.create(null);
  
  // Code injection
  INFINITE_LOOP_TRAP: string | null = null;
  STATEMENT_PREFIX: string | null = null;
  STATEMENT_SUFFIX: string | null = null;
  
  // Formatting
  INDENT = '  ';
  COMMENT_WRAP = 60;
  
  // State
  protected definitions_: {[key: string]: string} = Object.create(null);
  protected functionNames_: {[key: string]: string} = Object.create(null);
  nameDB_?: Names = undefined;
  
  constructor(name: string) {
    this.name_ = name;
  }
  
  workspaceToCode(workspace?: Workspace): string {
    if (!workspace) {
      workspace = common.getMainWorkspace();
    }
    
    const code = [];
    this.init(workspace);
    
    const blocks = workspace.getTopBlocks(true);
    for (let i = 0; i < blocks.length; i++) {
      let line = this.blockToCode(blocks[i]);
      
      if (Array.isArray(line)) {
        // Value blocks return [code, precedence]
        line = line[0];
      }
      
      if (line) {
        if (blocks[i].outputConnection) {
          // Naked value - apply scrubber
          line = this.scrubNakedValue(line);
          
          // Add statement prefix/suffix
          if (this.STATEMENT_PREFIX && !blocks[i].suppressPrefixSuffix) {
            line = this.injectId(this.STATEMENT_PREFIX, blocks[i]) + line;
          }
          if (this.STATEMENT_SUFFIX && !blocks[i].suppressPrefixSuffix) {
            line = line + this.injectId(this.STATEMENT_SUFFIX, blocks[i]);
          }
        }
        code.push(line);
      }
    }
    
    let codeString = code.join('\n');
    codeString = this.finish(codeString);
    
    // Clean whitespace
    codeString = codeString.replace(/^\s+\n/, '');
    codeString = codeString.replace(/\n\s+$/, '\n');
    codeString = codeString.replace(/[ \t]+\n/g, '\n');
    
    return codeString;
  }
}

Initialization

init(workspace: Workspace) {
  // Create name database
  if (!this.nameDB_) {
    this.nameDB_ = new Names(this.RESERVED_WORDS_);
  } else {
    this.nameDB_.reset();
  }
  
  // Populate with variable names
  const variables = workspace.getAllVariables();
  for (let i = 0; i < variables.length; i++) {
    const variable = variables[i];
    this.nameDB_.getName(variable.getId(), NameType.VARIABLE);
  }
  
  // Reset definitions
  this.definitions_ = Object.create(null);
  this.functionNames_ = Object.create(null);
  
  // Reset variable DB (for procedures)
  if (this.nameDB_.populateProcedures) {
    this.nameDB_.populateProcedures(workspace);
  }
}

Block-to-Code Translation

The blockToCode Method

blockToCode(block: Block | null, opt_thisOnly?: boolean): string | [string, number] {
  if (!block) {
    return '';
  }
  
  if (!block.isEnabled()) {
    // Skip disabled blocks, move to next
    return opt_thisOnly ? '' : this.blockToCode(block.getNextBlock());
  }
  
  if (block.isInsertionMarker()) {
    // Skip insertion markers
    return opt_thisOnly ? '' : this.blockToCode(block.getChildren(false)[0]);
  }
  
  // Look up generator function
  const func = this.forBlock[block.type];
  if (typeof func !== 'function') {
    throw Error(
      `${this.name_} generator does not know how to generate code ` +
      `for block type "${block.type}".`
    );
  }
  
  // Call generator function
  let code = func.call(block, block, this);
  
  if (Array.isArray(code)) {
    // Value blocks return [code, order]
    return [this.scrub_(block, code[0], opt_thisOnly), code[1]];
  } else if (typeof code === 'string') {
    // Statement blocks return code string
    
    // Add statement prefix/suffix
    if (this.STATEMENT_PREFIX && !block.suppressPrefixSuffix) {
      code = this.injectId(this.STATEMENT_PREFIX, block) + code;
    }
    if (this.STATEMENT_SUFFIX && !block.suppressPrefixSuffix) {
      code = code + this.injectId(this.STATEMENT_SUFFIX, block);
    }
    
    return this.scrub_(block, code, opt_thisOnly);
  } else if (code === null) {
    // Block handled code generation itself
    return '';
  }
  
  throw SyntaxError('Invalid code generated: ' + code);
}

Helper Methods

Getting Input Values:

valueToCode(block: Block, name: string, outerOrder: number): string {
  const targetBlock = block.getInputTargetBlock(name);
  if (!targetBlock) {
    return '';
  }
  
  const tuple = this.blockToCode(targetBlock);
  if (!Array.isArray(tuple)) {
    // Statement block in value input - shouldn't happen
    return '';
  }
  
  let code = tuple[0];
  const innerOrder = tuple[1];
  
  if (!code) {
    return '';
  }
  
  // Add parentheses if necessary
  if (innerOrder !== undefined && outerOrder !== undefined) {
    if (this.shouldAddParentheses_(innerOrder, outerOrder)) {
      code = '(' + code + ')';
    }
  }
  
  return code;
}

Getting Statement Code:

statementToCode(block: Block, name: string): string {
  const targetBlock = block.getInputTargetBlock(name);
  let code = this.blockToCode(targetBlock);
  
  if (Array.isArray(code)) {
    // Value block in statement input - just use code
    code = code[0];
  }
  
  // Apply infinite loop trap if configured
  if (this.INFINITE_LOOP_TRAP) {
    code = this.injectId(this.INFINITE_LOOP_TRAP, block) + code;
  }
  
  return code;
}

Prefix Lines:

prefixLines(text: string, prefix: string): string {
  return prefix + text.replace(/(?!\n$)\n/g, '\n' + prefix);
}

Operator Precedence

Why Precedence Matters

Consider this expression: 5 + 3 * 2

Without proper precedence handling:

var x = 5 + 3 * 2;  // Correct: 11
var y = (5 + 3) * 2; // Wrong if we always add parentheses: 16

Blockly's generator handles this automatically.

Order Constants

From generators/javascript/javascript_generator.ts:

export enum Order {
  ATOMIC = 0,           // 0 "" ...
  NEW = 1.1,            // new
  MEMBER = 1.2,         // . []
  FUNCTION_CALL = 2,    // ()
  INCREMENT = 3,        // ++
  DECREMENT = 3,        // --
  BITWISE_NOT = 4.1,    // ~
  UNARY_PLUS = 4.2,     // +
  UNARY_NEGATION = 4.3, // -
  LOGICAL_NOT = 4.4,    // !
  TYPEOF = 4.5,         // typeof
  VOID = 4.6,           // void
  DELETE = 4.7,         // delete
  AWAIT = 4.8,          // await
  EXPONENTIATION = 5.0, // **
  MULTIPLICATION = 5.1, // *
  DIVISION = 5.2,       // /
  MODULUS = 5.3,        // %
  SUBTRACTION = 6.1,    // -
  ADDITION = 6.2,       // +
  BITWISE_SHIFT = 7,    // << >> >>>
  RELATIONAL = 8,       // < <= > >=
  IN = 8,               // in
  INSTANCEOF = 8,       // instanceof
  EQUALITY = 9,         // == != === !==
  BITWISE_AND = 10,     // &
  BITWISE_XOR = 11,     // ^
  BITWISE_OR = 12,      // |
  LOGICAL_AND = 13,     // &&
  LOGICAL_OR = 14,      // ||
  CONDITIONAL = 15,     // ?:
  ASSIGNMENT = 16,      // = += -= **= *= /= %= <<= >>= ...
  YIELD = 17,           // yield
  COMMA = 18,           // ,
  NONE = 99,            // (...)
}

Using Precedence

Math Addition Example:

// generators/javascript/math.ts
export function math_arithmetic(
  block: Block,
  generator: JavascriptGenerator,
) {
  const OPERATORS: {[key: string]: [string, Order]} = {
    'ADD': [' + ', Order.ADDITION],
    'MINUS': [' - ', Order.SUBTRACTION],
    'MULTIPLY': [' * ', Order.MULTIPLICATION],
    'DIVIDE': [' / ', Order.DIVISION],
    'POWER': [' ** ', Order.EXPONENTIATION],
  };
  
  const tuple = OPERATORS[block.getFieldValue('OP')];
  const operator = tuple[0];
  const order = tuple[1];
  
  const argument0 = generator.valueToCode(block, 'A', order) || '0';
  const argument1 = generator.valueToCode(block, 'B', order) || '0';
  
  const code = argument0 + operator + argument1;
  return [code, order];
}

Comparison Example:

export function logic_compare(
  block: Block,
  generator: JavascriptGenerator,
) {
  const OPERATORS: {[key: string]: string} = {
    'EQ': '==',
    'NEQ': '!=',
    'LT': '<',
    'LTE': '<=',
    'GT': '>',
    'GTE': '>=',
  };
  
  const operator = OPERATORS[block.getFieldValue('OP')];
  const order = Order.RELATIONAL;
  
  const argument0 = generator.valueToCode(block, 'A', order) || '0';
  const argument1 = generator.valueToCode(block, 'B', order) || '0';
  
  const code = argument0 + ' ' + operator + ' ' + argument1;
  return [code, order];
}

Language-Specific Generators

JavaScript Generator

From generators/javascript/javascript_generator.ts:

export class JavascriptGenerator extends CodeGenerator {
  constructor() {
    super('JavaScript');
  }
  
  init(workspace: Workspace) {
    super.init(workspace);
    
    // Add common definitions
    if (!this.nameDB_.getDistinctName) {
      this.nameDB_!.getDistinctName = this.getDistinctName;
    }
  }
  
  finish(code: string): string {
    // Convert definitions to code
    const definitions = [];
    for (const name in this.definitions_) {
      definitions.push(this.definitions_[name]);
    }
    
    // Add definitions before code
    const allDefs = definitions.join('\n\n');
    return allDefs.replace(/\n\n+/g, '\n\n').replace(/\n*$/, '\n\n\n') + code;
  }
  
  scrubNakedValue(line: string): string {
    return line + ';\n';
  }
  
  scrub_(block: Block, code: string, opt_thisOnly?: boolean): string {
    let commentCode = '';
    
    // Get comment from block
    if (!block.outputConnection || !opt_thisOnly) {
      const comment = block.getCommentText();
      if (comment) {
        commentCode += this.prefixLines(comment, '// ') + '\n';
      }
      
      // Collect comments from all blocks in stack
      for (let i = 0; i < block.inputList.length; i++) {
        const input = block.inputList[i];
        if (input.type === inputTypes.STATEMENT) {
          const childBlock = input.connection!.targetBlock();
          if (childBlock) {
            const childComments = this.allNestedComments(childBlock);
            if (childComments) {
              commentCode += this.prefixLines(childComments, '// ');
            }
          }
        }
      }
    }
    
    const nextBlock = block.nextConnection?.targetBlock();
    const nextCode = opt_thisOnly ? '' : this.blockToCode(nextBlock);
    
    return commentCode + code + nextCode;
  }
}

Python Generator Differences

From generators/python/python_generator.ts:

export class PythonGenerator extends CodeGenerator {
  constructor() {
    super('Python');
    
    // Python-specific settings
    this.INDENT = '  ';
  }
  
  init(workspace: Workspace) {
    super.init(workspace);
    
    // Initialize import dictionary
    this.imports_ = Object.create(null);
  }
  
  finish(code: string): string {
    // Convert imports
    const imports = [];
    for (const name in this.imports_) {
      imports.push(this.imports_[name]);
    }
    
    // Convert definitions
    const definitions = [];
    for (const name in this.definitions_) {
      definitions.push(this.definitions_[name]);
    }
    
    // Assemble: imports, then definitions, then code
    const allDefs = imports.join('\n') + '\n\n' +
                    definitions.join('\n\n');
    
    return allDefs.replace(/\n\n+/g, '\n\n').replace(/\n*$/, '\n\n') + code;
  }
  
  scrubNakedValue(line: string): string {
    // Python doesn't need semicolons
    return line + '\n';
  }
}

Example: Loop Generator Comparison

JavaScript:

export function controls_repeat_ext(
  block: Block,
  generator: JavascriptGenerator,
) {
  const repeats = generator.valueToCode(block, 'TIMES', Order.ASSIGNMENT) || '0';
  let branch = generator.statementToCode(block, 'DO');
  branch = generator.addLoopTrap(branch, block);
  
  const loopVar = generator.nameDB_!.getDistinctName('count', NameType.VARIABLE);
  
  const code = 
    'for (var ' + loopVar + ' = 0; ' +
    loopVar + ' < ' + repeats + '; ' +
    loopVar + '++) {\n' +
    branch + '}\n';
  
  return code;
}

Python:

export function controls_repeat_ext(
  block: Block,
  generator: PythonGenerator,
) {
  const repeats = generator.valueToCode(block, 'TIMES', Order.NONE) || '0';
  let branch = generator.statementToCode(block, 'DO');
  branch = generator.addLoopTrap(branch, block) || generator.PASS;
  
  const loopVar = generator.nameDB_!.getDistinctName('count', NameType.VARIABLE);
  
  const code = 
    'for ' + loopVar + ' in range(' + repeats + '):\n' +
    branch;
  
  return code;
}

Writing Custom Generators

Basic Pattern

// 1. Register the generator function
javascriptGenerator.forBlock['custom_block'] = function(block, generator) {
  // 2. Get field values
  const fieldValue = block.getFieldValue('FIELD_NAME');
  
  // 3. Get input values
  const inputValue = generator.valueToCode(
    block, 
    'INPUT_NAME', 
    Order.ATOMIC
  ) || 'defaultValue';
  
  // 4. Get statement code
  const statementCode = generator.statementToCode(block, 'STATEMENT_NAME');
  
  // 5. Generate code
  const code = `customFunction(${inputValue})`;
  
  // 6. Return based on block type
  // For value blocks: return [code, order]
  // For statement blocks: return code
  return [code, Order.FUNCTION_CALL];
};

Example: Custom Math Block

Block Definition:

Blockly.Blocks['math_custom_operation'] = {
  init: function() {
    this.appendValueInput('A')
        .setCheck('Number')
        .appendField('calculate');
    this.appendDummyInput()
        .appendField(new Blockly.FieldDropdown([
          ['double', 'DOUBLE'],
          ['square', 'SQUARE'],
          ['negate', 'NEGATE']
        ]), 'OP');
    this.setOutput(true, 'Number');
    this.setColour(230);
  }
};

JavaScript Generator:

javascriptGenerator.forBlock['math_custom_operation'] = function(block, generator) {
  const operation = block.getFieldValue('OP');
  const value = generator.valueToCode(block, 'A', Order.ATOMIC) || '0';
  
  let code;
  switch (operation) {
    case 'DOUBLE':
      code = value + ' * 2';
      return [code, Order.MULTIPLICATION];
    case 'SQUARE':
      code = 'Math.pow(' + value + ', 2)';
      return [code, Order.FUNCTION_CALL];
    case 'NEGATE':
      code = '-' + value;
      return [code, Order.UNARY_NEGATION];
  }
};

Python Generator:

pythonGenerator.forBlock['math_custom_operation'] = function(block, generator) {
  const operation = block.getFieldValue('OP');
  const value = generator.valueToCode(block, 'A', Order.ATOMIC) || '0';
  
  let code;
  switch (operation) {
    case 'DOUBLE':
      code = value + ' * 2';
      return [code, Order.MULTIPLICATIVE];
    case 'SQUARE':
      code = value + ' ** 2';
      return [code, Order.EXPONENTIATION];
    case 'NEGATE':
      code = '-' + value;
      return [code, Order.UNARY_SIGN];
  }
};

Advanced Code Generation

Adding Function Definitions

javascriptGenerator.forBlock['procedures_callnoreturn'] = function(block, generator) {
  const funcName = generator.getVariableName(block.getFieldValue('NAME'));
  const args = [];
  
  for (let i = 0; i < block.arguments_.length; i++) {
    args[i] = generator.valueToCode(block, 'ARG' + i, Order.NONE) || 'null';
  }
  
  const code = funcName + '(' + args.join(', ') + ');\n';
  return code;
};

javascriptGenerator.forBlock['procedures_defnoreturn'] = function(block, generator) {
  const funcName = generator.getVariableName(block.getFieldValue('NAME'));
  const branch = generator.statementToCode(block, 'STACK') || '';
  
  const args = block.arguments_.map(arg => 
    generator.getVariableName(arg)
  ).join(', ');
  
  // Add to definitions instead of inline code
  const code = 
    'function ' + funcName + '(' + args + ') {\n' +
    branch +
    '}';
  
  generator.definitions_[funcName] = code;
  
  return null; // Definition blocks don't generate inline code
};

Loop Traps

Prevent infinite loops in generated code:

const workspace = Blockly.inject('blocklyDiv', {
  // ...
});

// Set infinite loop trap
javascriptGenerator.INFINITE_LOOP_TRAP = 
  'if (--window.LoopTrap == 0) throw "Infinite loop.";\n';

// Before running code:
window.LoopTrap = 1000; // Max iterations
eval(code);

Statement Prefixes

Highlight blocks during execution:

javascriptGenerator.STATEMENT_PREFIX = 'highlightBlock(%1);\n';

function highlightBlock(id) {
  workspace.highlightBlock(id);
}

// Generated code includes:
highlightBlock('block_id_123');
doSomething();

Summary

In Part 3, we've explored:

✅ Code Generation Architecture - Workspace to executable code flow
✅ CodeGenerator Class - Base class and core methods
✅ Block Translation - blockToCode and helper methods
✅ Operator Precedence - Proper parenthesization
✅ Language Generators - JavaScript, Python, and others
✅ Custom Generators - Writing your own code generators
✅ Advanced Features - Definitions, loop traps, statement prefixes


In Part 4, we'll explore advanced topics including the event system, serialization, connection management, and plugin architecture.

This article is based on analysis of Blockly v12.3.1 source code. For the latest information, refer to the official Blockly documentation.

Subscribe to Root Logic

Don’t miss out on the latest issues. Sign up now to get access to the library of members-only issues.
jamie@example.com
Subscribe
⇧