Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
bd1c7bd
Add OAL Engine V2 implementation with V1 template compatibility
wu-sheng Feb 9, 2026
251f243
Switch OAL runtime from V1 to V2
wu-sheng Feb 9, 2026
7af21eb
Add comprehensive error handling and tests for OAL parsing
wu-sheng Feb 9, 2026
1104844
Fix OAL parsing error tests to match actual parser behavior
wu-sheng Feb 9, 2026
531029d
Update OAL parsing error tests with specific line:column assertions
wu-sheng Feb 9, 2026
8758110
Remove integration tests that require server-core dependencies
wu-sheng Feb 9, 2026
3862df5
Fix OAL V2 FreeMarker templates and add integration tests
wu-sheng Feb 10, 2026
01ae4e0
Fix type casting and add production OAL parsing tests
wu-sheng Feb 10, 2026
86c4b4b
Fix critical V1 vs V2 template differences
wu-sheng Feb 10, 2026
c5b61bb
Add comprehensive filter verification tests for OAL V2
wu-sheng Feb 10, 2026
acfee92
Add comprehensive V1 vs V2 bytecode comparison tests
wu-sheng Feb 10, 2026
bbc7b47
Remove V1 OAL engine code and update V2 documentation
wu-sheng Feb 10, 2026
c97a743
Add source generation and bytecode consistency verification
wu-sheng Feb 10, 2026
c5a1c58
Write generated sources to proper package directories for IDE navigation
wu-sheng Feb 10, 2026
25ccedb
Remove oal as root folder in generated-test-sources
wu-sheng Feb 10, 2026
a1fd4af
Move JDK 11 compatibility to CLAUDE.md and update OAL V2 README
wu-sheng Feb 10, 2026
b5e8a5e
Fix incorrect avg() to longAvg() in OAL documentation example
wu-sheng Feb 10, 2026
1281c60
Improve OAL V2 engine with enhanced testing and documentation
wu-sheng Feb 11, 2026
9eefa92
Fix OAL V2 nested boolean attribute handling in function arguments
wu-sheng Feb 11, 2026
3d0651a
Merge remote-tracking branch 'origin/master' into oal-v2
wu-sheng Feb 11, 2026
8d58242
Enhance OAL V2 testing with runtime generation for all production scr…
wu-sheng Feb 11, 2026
8316e6c
Fix javadoc errors by escaping > characters in comments
wu-sheng Feb 11, 2026
b7f4b8e
Fix review comments from copilot
wu-sheng Feb 11, 2026
1f2f838
Improve OAL V2 engine with enhanced testing and documentation
wu-sheng Feb 12, 2026
ae0f123
Remove JDK 8 compatibility code from OAL V2
wu-sheng Feb 12, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 34 additions & 16 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,40 @@ public class XxxModuleProvider extends ModuleProvider {
Java, XML, and YAML/YML files must include the Apache 2.0 license header (see `HEADER` file).
JSON and Markdown files are excluded (JSON doesn't support comments, see `.licenserc.yaml`).

### JDK 11 Compatibility

All code must be compatible with JDK 11 (LTS). The project supports JDK 11, 17, and 21.

**Prohibited Java features (post-JDK 11):**

| Feature | JDK Version | Use Instead |
|---------|-------------|-------------|
| Switch expressions (`->`) | 14+ | Traditional `switch` with `case:` and `break` |
| `Stream.toList()` | 16+ | `.collect(Collectors.toList())` |
| Text blocks (`"""..."""`) | 15+ | String concatenation or `+` |
| Records | 14+ | Regular classes with Lombok `@Data` |
| Pattern matching for `instanceof` | 14+ | Traditional cast after `instanceof` |
| Sealed classes/interfaces | 15+ | Regular classes/interfaces |

**Allowed Java features (JDK 11 compatible):**
- `List.of()`, `Set.of()`, `Map.of()` - Immutable collections (Java 9+)
- `Optional` methods - `orElseThrow()`, `ifPresentOrElse()` (Java 9+)
- Lambda expressions and method references (Java 8+)
- Stream API (Java 8+)
- Lombok annotations (`@Getter`, `@Builder`, `@Data`, `@Slf4j`)

**Verification commands:**
```bash
# Check for switch expressions (should return no matches)
grep -r "switch.*->" src/ --include="*.java"

# Check for Stream.toList() (should return no matches)
grep -r "\.toList()" src/ --include="*.java"

# Check for text blocks (should return no matches)
grep -r '"""' src/ --include="*.java"
```

## Testing

### Test Frameworks
Expand Down Expand Up @@ -507,16 +541,6 @@ The project uses submodules for protocol definitions and UI:

Always use `--recurse-submodules` when cloning or update submodules manually.

## IDE Setup (IntelliJ IDEA)

1. Import as Maven project
2. Run `./mvnw compile -Dmaven.test.skip=true` to generate protobuf sources
3. Mark generated source folders:
- `*/target/generated-sources/protobuf/java`
- `*/target/generated-sources/protobuf/grpc-java`
- `*/target/generated-sources/antlr4`
4. Import `codeStyle.xml` for consistent formatting

## Key Files for Understanding the Codebase

- `oap-server/server-core/src/main/java/.../CoreModule.java` - Core module definition
Expand Down Expand Up @@ -560,12 +584,6 @@ Always use `--recurse-submodules` when cloning or update submodules manually.
- `changes/changes.md` - Changelog (update when making changes)
- `swip/` - SkyWalking Improvement Proposals

## Community

- GitHub Issues: https://github.com/apache/skywalking/issues
- Mailing List: dev@skywalking.apache.org
- Slack: #skywalking channel at Apache Slack

## Submitting Pull Requests

### Branch Strategy
Expand Down
6 changes: 6 additions & 0 deletions docs/en/changes/changes.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,12 @@
## 10.4.0

#### Project
* Introduce OAL V2 engine:
- Immutable AST models for thread safety and predictable behavior
- Type-safe enums replacing string-based filter operators
- Precise error location reporting with file, line, and column numbers
- Clean separation between parsing and code generation phases
- Enhanced testability with models that can be constructed without parsing
* Fix E2E test metrics verify: make it failure if the metric values all null.
* Support building, testing, and publishing with Java 25.
* Add `CLAUDE.md` as AI assistant guide for the project.
Expand Down
2 changes: 1 addition & 1 deletion docs/en/concepts-and-designs/oal.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,7 +174,7 @@ endpoint_p99 = from(Endpoint.latency).filter(name in ("Endpoint1", "Endpoint2"))
serv_Endpoint_p99 = from(Endpoint.latency).filter(name like "serv%").percentile2(10)

// Calculate the avg response time of each Endpoint
endpoint_resp_time = from(Endpoint.latency).avg()
endpoint_resp_time = from(Endpoint.latency).longAvg()

// Calculate the p50, p75, p90, p95 and p99 of each Endpoint by 50 ms steps.
endpoint_percentile = from(Endpoint.latency).percentile2(10)
Expand Down
103 changes: 103 additions & 0 deletions oap-server/oal-rt/CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
# OAL V2 Engine

The OAL (Observability Analysis Language) engine for Apache SkyWalking. This is the only OAL implementation.

## Package Structure

```
org.apache.skywalking.oal.v2/
├── model/ # Immutable data models (parser output)
│ ├── SourceLocation # Location in source file for error reporting
│ ├── SourceReference # from(Service.latency)
│ ├── FunctionCall # longAvg(), percentile2(10)
│ ├── FunctionArgument # Typed function arguments
│ ├── FilterOperator # Enum: ==, !=, >, <, like, in
│ ├── FilterExpression # latency > 100
│ ├── FilterValue # Typed filter values
│ └── MetricDefinition # Complete parsed metric
├── parser/ # OAL script parsing
│ ├── OALListenerV2 # ANTLR parse tree listener
│ └── OALScriptParserV2 # Parser facade
├── generator/ # Code generation
│ ├── CodeGenModel # Code generation data model
│ ├── MetricDefinitionEnricher # Metadata enrichment
│ └── OALClassGeneratorV2 # Javassist bytecode generator
├── metadata/ # Source/metrics metadata utilities
│ ├── SourceColumnsFactory
│ ├── SourceColumn
│ ├── FilterMatchers
│ └── MetricsHolder
├── util/ # Code generation utilities
│ ├── ClassMethodUtil
│ └── TypeCastUtil
└── OALEngineV2 # Main engine entry point
```

## Pipeline

```
┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ .oal file │───▶│ Parser │───▶│ Enricher │───▶│ Generator │
│ │ │ │ │ │ │ │
│ OAL script │ │ MetricDef │ │ CodeGenModel│ │ Bytecode/ │
│ │ │ (immutable) │ │ (metadata) │ │ Source │
└─────────────┘ └─────────────┘ └─────────────┘ └─────────────┘
```

1. **Parser** (`OALScriptParserV2`): Parses `.oal` files into immutable `MetricDefinition` objects
2. **Enricher** (`MetricDefinitionEnricher`): Adds metadata via reflection (source columns, persistent fields)
3. **Generator** (`OALClassGeneratorV2`): Generates Java classes using Javassist and FreeMarker templates

## Design Principles

1. **Immutable Models**: All parser output classes are immutable and thread-safe
2. **Type Safety**: Use enums and typed values instead of strings
3. **Builder Pattern**: Complex objects use fluent builders
4. **Separation of Concerns**: Parser models ≠ Code generation models
5. **Testability**: Models can be constructed without parsing for unit tests

## Key Classes

### MetricDefinition (Parser Output)

Immutable representation of a parsed OAL metric:

```java
// service_resp_time = from(Service.latency).filter(latency > 0).longAvg()
MetricDefinition metric = MetricDefinition.builder()
.name("service_resp_time")
.source(SourceReference.of("Service", "latency"))
.addFilter(FilterExpression.of("latency", ">", 0L))
.aggregationFunction(FunctionCall.of("longAvg"))
.build();
```

### CodeGenModel (Generator Input)

Enriched model with metadata for code generation:

```java
// Created by MetricDefinitionEnricher
CodeGenModel model = enricher.enrich(metricDefinition);

// Contains: source columns, persistent fields, metrics class info, etc.
model.getFieldsFromSource(); // Fields copied from source
model.getPersistentFields(); // Fields for storage
model.getMetricsClassName(); // e.g., "LongAvgMetrics"
```

## Debug Output

When `SW_OAL_ENGINE_DEBUG=true` environment variable is set, generated `.class` files are written to disk for inspection:

```
{skywalking}/oal-rt/
├── metrics/ - Generated metrics .class files
└── dispatcher/ - Generated dispatcher .class files
```

This is useful for debugging code generation issues or comparing V1 vs V2 output.

## Runtime Integration

The engine is loaded via reflection in `OALEngineLoaderService` because `server-core` compiles before `oal-rt` in the Maven reactor. Generated classes integrate with SkyWalking's stream processing pipeline.
Loading
Loading