ADR-075: In-Memory Preflight Validation Model

Status: Proposed Date: 2025-12-30 Related: ADR-074 (Workflow Architecture), ADR-004 (Enhanced Table Creation), ADR-005 (Migration Scripts)

Context and Problem Statement

Currently, table and window creation operations execute directly against the database. If validation fails mid-process, partial data may remain in the Application Dictionary. Additionally, migration scripts and 2Pack files are generated without comprehensive validation against iDempiere core class constraints.

Problems:

No pre-execution validation - Errors discovered during DB operations
Partial failures - Incomplete data left in AD on errors
No format validation - 2Pack XML structure not verified before generation
No constraint checking - Column types, lengths, references not validated against iDempiere rules

Inspiration: Nx build system pattern of "plan → validate → confirm → execute"

Decision Drivers

Fail-fast principle - Catch errors before any DB changes
Atomic operations - All-or-nothing execution
Developer experience - Clear validation messages before execution
2Pack compatibility - Ensure generated packages are valid
Migration safety - Validate scripts before applying

Decision

Implement an In-Memory Preflight Validation Model that:

Builds complete in-memory representation before any DB operations
Validates against iDempiere core class rules
Validates output formats (2Pack XML, Migration SQL)
Only executes if all validations pass

Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                    PREFLIGHT VALIDATION PIPELINE                     │
├─────────────────────────────────────────────────────────────────────┤
│                                                                      │
│  INPUT                    IN-MEMORY MODEL              VALIDATION    │
│  ──────                   ───────────────              ──────────    │
│                                                                      │
│  ┌──────────────┐        ┌──────────────────┐        ┌────────────┐ │
│  │ Column Defs  │───────▶│ TableSpec        │───────▶│ Core Rules │ │
│  │ S#Name       │        │  ├─ name         │        │ ├─ MColumn  │ │
│  │ Q#Qty        │        │  ├─ columns[]    │        │ ├─ MTable   │ │
│  │ D#DueDate    │        │  │   ├─ name     │        │ ├─ Display  │ │
│  └──────────────┘        │  │   ├─ type     │        │ │   Type    │ │
│                          │  │   ├─ length   │        │ └─ Ref      │ │
│                          │  │   └─ ref      │        │     Valid   │ │
│                          │  ├─ window       │        └──────┬─────┘ │
│                          │  └─ entityType   │               │       │
│                          └──────────────────┘               │       │
│                                   │                         │       │
│                                   ▼                         ▼       │
│                          ┌──────────────────┐        ┌────────────┐ │
│                          │ Output Preview   │───────▶│ Format     │ │
│                          │  ├─ 2Pack XML    │        │ Validation │ │
│                          │  ├─ Migration SQL│        │ ├─ XML     │ │
│                          │  └─ Model Java   │        │ ├─ SQL     │ │
│                          └──────────────────┘        │ └─ Java    │ │
│                                                      └──────┬─────┘ │
│                                                             │       │
│                                   ┌─────────────────────────┘       │
│                                   ▼                                 │
│                          ┌──────────────────┐                       │
│                          │ Validation Result│                       │
│                          │  ├─ errors[]     │                       │
│                          │  ├─ warnings[]   │                       │
│                          │  └─ canProceed   │                       │
│                          └────────┬─────────┘                       │
│                                   │                                 │
│              ┌────────────────────┼────────────────────┐            │
│              ▼                    ▼                    ▼            │
│        ┌──────────┐        ┌──────────┐        ┌──────────┐         │
│        │ BLOCKED  │        │ WARNINGS │        │   OK     │         │
│        │ (errors) │        │ (proceed │        │ (execute)│         │
│        │          │        │  w/ack)  │        │          │         │
│        └──────────┘        └──────────┘        └──────────┘         │
│                                                                      │
└─────────────────────────────────────────────────────────────────────┘

In-Memory Model Classes

TableSpec (Central Model)

package org.idempiere.cli.model;

/**
 * In-memory specification for a table before creation.
 * Validated against iDempiere core classes before execution.
 */
public class TableSpec {
    private String tableName;
    private String description;
    private String entityType = &quot;U&quot;;
    private String accessLevel = &quot;3&quot;;
    private List&lt;ColumnSpec&gt; columns = new ArrayList&lt;&gt;();
    private WindowSpec window;  // Optional linked window

    // Computed fields (populated during validation)
    private Integer estimatedTableId;
    private String keyColumnName;
    private boolean hasStandardColumns = true;

    // Validation state
    private ValidationResult validationResult;

    public static TableSpec parse(String tableName, String columnDefs) {
        // Parse &quot;S#Name,Q#Qty,D#DueDate&quot; into structured model
    }

    public ValidationResult validate() {
        // Run all validations
    }

    public String toMigrationSql() {
        // Generate SQL without executing
    }

    public String to2PackXml() {
        // Generate 2Pack XML without executing
    }
}

ColumnSpec

/**
 * In-memory specification for a column.
 */
public class ColumnSpec {
    private String columnName;
    private int displayType;        // From DisplayType constants
    private String referenceTable;  // For TableDir references
    private int fieldLength;
    private boolean mandatory;
    private String defaultValue;
    private String description;

    // Validation
    private List&lt;String&gt; validationErrors = new ArrayList&lt;&gt;();

    public static ColumnSpec fromPrefix(String prefixDef) {
        // Parse &quot;S#Name&quot; → ColumnSpec with displayType=10, length=60
        // Parse &quot;ID#C_BPartner_ID&quot; → ColumnSpec with displayType=19, ref=C_BPartner
    }
}

WindowSpec

/**
 * In-memory specification for a window.
 */
public class WindowSpec {
    private String windowName;
    private String windowType = &quot;M&quot;;  // Maintain
    private String entityType = &quot;U&quot;;
    private List&lt;TabSpec&gt; tabs = new ArrayList&lt;&gt;();
    private boolean includeMenu;
}

Validation Rules

1. Core Class Validations (iDempiere Rules)

Rule	Source Class	Validation
Table name format	`MTable`	Must match `^[A-Z][A-Za-z0-9_]*$`
Table name length	`MTable`	Max 40 characters
Column name format	`MColumn`	Must match `^[A-Z][A-Za-z0-9_]*$`
Column name length	`MColumn`	Max 40 characters
DisplayType valid	`DisplayType`	Must be known type (10, 11, 12, etc.)
Reference exists	`MColumn`	TableDir reference must exist in AD
Field length	`MColumn`	Must match DisplayType constraints
Key column	`MTable`	Must have {TableName}_ID column
UUID column	`MTable`	Must have {TableName}_UU column (v11+)

2. Naming Convention Validations

Rule	Validation
Custom prefix	Custom tables should use XX_ or entity-specific prefix
No reserved names	Cannot use AD_, C_, M_ for custom tables
Unique name	Table/column name not already in AD

3. Format Validations

Format	Validations
2Pack XML	Well-formed XML, required elements, valid UUIDs
Migration SQL	Valid PostgreSQL syntax, proper escaping
Java Model	Valid class name, imports, compilation

Validation Service

@ApplicationScoped
public class PreflightValidationService {

    @Inject
    ADRegistryService registryService;  // For checking existing elements

    /**
     * Validate a TableSpec against all rules.
     */
    public ValidationResult validateTable(TableSpec spec) {
        ValidationResult result = new ValidationResult();

        // 1. Core class validations
        validateTableName(spec, result);
        validateColumns(spec, result);
        validateKeyColumn(spec, result);

        // 2. Naming conventions
        validateNamingConventions(spec, result);

        // 3. Uniqueness (requires DB check)
        validateUniqueness(spec, result);

        // 4. Preview outputs
        if (result.canProceed()) {
            result.setPreviewSql(spec.toMigrationSql());
            result.setPreview2Pack(spec.to2PackXml());
        }

        return result;
    }

    /**
     * Validate column against MColumn rules.
     */
    private void validateColumn(ColumnSpec col, ValidationResult result) {
        // DisplayType validation
        if (!DisplayType.isValid(col.getDisplayType())) {
            result.addError(&quot;Column &quot; + col.getName() +
                &quot;: Invalid DisplayType &quot; + col.getDisplayType());
        }

        // Length validation
        int maxLength = DisplayType.getMaxLength(col.getDisplayType());
        if (col.getFieldLength() &gt; maxLength) {
            result.addWarning(&quot;Column &quot; + col.getName() +
                &quot;: Length &quot; + col.getFieldLength() +
                &quot; exceeds recommended &quot; + maxLength);
        }

        // Reference validation for TableDir
        if (col.getDisplayType() == DisplayType.TableDir) {
            if (!registryService.tableExists(col.getReferenceTable())) {
                result.addError(&quot;Column &quot; + col.getName() +
                    &quot;: Reference table &quot; + col.getReferenceTable() + &quot; not found&quot;);
            }
        }
    }
}

ValidationResult

/**
 * Result of preflight validation.
 */
public class ValidationResult {
    private boolean valid = true;
    private List&lt;ValidationError&gt; errors = new ArrayList&lt;&gt;();
    private List&lt;ValidationWarning&gt; warnings = new ArrayList&lt;&gt;();

    // Preview outputs (generated if validation passes)
    private String previewSql;
    private String preview2Pack;
    private String previewJava;

    public boolean canProceed() {
        return errors.isEmpty();
    }

    public boolean hasWarnings() {
        return !warnings.isEmpty();
    }

    public record ValidationError(
        String field,
        String message,
        String rule,       // e.g., &quot;MColumn.COLUMNNAME_LENGTH&quot;
        String suggestion  // e.g., &quot;Shorten to 40 characters&quot;
    ) {}

    public record ValidationWarning(
        String field,
        String message,
        String rule
    ) {}
}

Integration with Workflows (ADR-074)

@ApplicationScoped
public class WfDeployToolLogic {

    @Inject
    PreflightValidationService validationService;

    /**
     * Plan deployment with full validation (Nx-style dry-run).
     */
    public ToolResult wfPlanDeployTable(String tableName, String columns, ...) {

        // 1. Parse into in-memory model
        TableSpec spec = TableSpec.parse(tableName, columns);

        // 2. Run preflight validation
        ValidationResult validation = validationService.validateTable(spec);

        // 3. Return plan with validation results
        if (!validation.canProceed()) {
            return ToolResult.error(&quot;Validation failed&quot;)
                .data(&quot;errors&quot;, validation.getErrors())
                .data(&quot;suggestions&quot;, validation.getSuggestions());
        }

        // 4. Return plan with previews
        return ToolResult.success(&quot;Plan validated&quot;)
            .data(&quot;plan&quot;, buildPlan(spec))
            .data(&quot;previewSql&quot;, validation.getPreviewSql())
            .data(&quot;preview2Pack&quot;, validation.getPreview2Pack());
    }

    /**
     * Execute only if validation passes.
     */
    public ToolResult wfDeployTable(String tableName, String columns, ...) {

        // 1. Parse and validate
        TableSpec spec = TableSpec.parse(tableName, columns);
        ValidationResult validation = validationService.validateTable(spec);

        if (!validation.canProceed()) {
            return ToolResult.error(&quot;Cannot execute: validation failed&quot;)
                .data(&quot;errors&quot;, validation.getErrors());
        }

        // 2. Warn about warnings but proceed
        if (validation.hasWarnings()) {
            log.warning(&quot;Proceeding with warnings: &quot; + validation.getWarnings());
        }

        // 3. Execute with validated spec
        return executeDeployment(spec);
    }
}

DisplayType Validation Reference

Based on iDempiere's DisplayType.java:

Prefix	DisplayType	ID	Max Length	Validation
S#	String	10	60 (default)	Length 1-2000
T#	Text	14	2000	Any length
M#	Memo	34	unlimited	-
I#	Integer	11	-	Numeric
N#	Number	22	-	Decimal
A#	Amount	12	-	Currency precision
Q#	Quantity	29	-	Decimal
Y#	YesNo	20	1	'Y' or 'N'
D#	Date	15	-	Date format
d#	DateTime	16	-	Timestamp
L#	List	17	-	Must have AD_Reference
ID#	TableDir	19	-	Reference table must exist
B#	Binary	23	-	BLOB
U#	URL	40	2000	URL format

Migration Validation Rules

For migration scripts, additional validations:

public class MigrationValidator {

    /**
     * Validate SQL migration script syntax.
     */
    public ValidationResult validateMigration(String sql) {
        ValidationResult result = new ValidationResult();

        // 1. PostgreSQL syntax check (basic)
        if (!sql.contains(&quot;INSERT INTO&quot;) &amp;&amp; !sql.contains(&quot;ALTER TABLE&quot;)) {
            result.addWarning(&quot;No INSERT or ALTER statements found&quot;);
        }

        // 2. AD_Sequence check
        if (sql.contains(&quot;nextval&quot;) &amp;&amp; !sql.contains(&quot;AD_Sequence&quot;)) {
            result.addError(&quot;Using nextval without AD_Sequence update&quot;);
        }

        // 3. UUID format
        Pattern uuidPattern = Pattern.compile(
            &quot;&#39;[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}&#39;&quot;
        );
        // Validate all UUIDs are proper format

        // 4. Reserved words
        if (sql.matches(&quot;(?i).*\\b(DROP|TRUNCATE|DELETE FROM AD_).*&quot;)) {
            result.addError(&quot;Dangerous operation on core AD tables&quot;);
        }

        return result;
    }
}

2Pack Validation Rules

For 2Pack XML, validate:

public class TwoPackValidator {

    public ValidationResult validate2Pack(String xml) {
        ValidationResult result = new ValidationResult();

        // 1. XML well-formed
        try {
            DocumentBuilder builder = DocumentBuilderFactory.newInstance()
                .newDocumentBuilder();
            Document doc = builder.parse(new InputSource(new StringReader(xml)));
        } catch (Exception e) {
            result.addError(&quot;Invalid XML: &quot; + e.getMessage());
            return result;
        }

        // 2. Required elements
        if (!xml.contains(&quot;&lt;AD_Package_Exp&gt;&quot;)) {
            result.addError(&quot;Missing AD_Package_Exp element&quot;);
        }

        // 3. UUID format in elements
        // Validate all AD_*_UU values are proper UUID format

        // 4. Reference integrity
        // TableDir columns reference tables that are included or exist

        return result;
    }
}

Implementation Roadmap

Phase 1: Core Model Classes

[ ] Create model/ package
[ ] Implement TableSpec, ColumnSpec, WindowSpec
[ ] Implement ValidationResult
[ ] Column prefix parser

Phase 2: Validation Rules

[ ] Core class validations (MColumn, MTable rules)
[ ] Naming convention validations
[ ] DisplayType validations
[ ] Reference validations

Phase 3: Format Validators

[ ] SQL migration validator
[ ] 2Pack XML validator
[ ] Java model validator (optional)

Phase 4: Integration

[ ] Integrate with WfDeployToolLogic
[ ] Add preview generation
[ ] Update MCP tools with validation

Consequences

Positive

Fail-fast - Errors caught before any DB changes
Better UX - Clear validation messages with suggestions
Atomic - No partial failures in database
Preview - See generated outputs before execution
Testable - Validation logic fully unit-testable

Negative

Complexity - Additional abstraction layer
Maintenance - Must keep rules in sync with iDempiere core
Performance - Extra parsing step before execution

Mitigation

Rules extracted from iDempiere source comments
Validation is fast (in-memory)
Can skip validation with --force flag if needed

References

Document Status: Proposed Next Step: Review and approve, then implement Phase 1 (Core Model Classes)