Core Functions
sag_align
SAG ↔ OCTX data model alignment: ID mapping, cardinality validation, field-length checks, entity deduplication, and structured data export.
sag_align provides alignment utilities between the SAG application data model and
the OCTX sag-structured/0.1 Capability. It is used for import/export scenarios
including ID mapping, cardinality validation, field-length checks, entity
deduplication, and structured data export.
Functions
| Function | Purpose |
|---|---|
translate_id() | UUIDv4 → UUIDv7 (bijective reversible mapping) |
revert_id() | UUIDv7 → UUIDv4 (transitional backward compatibility) |
ensure_octx_ids() | Batch-convert UUIDv4 fields in records |
default_sag_field_profile() | Return the default SAG field profile |
validate_cardinality_for_sag() | Check if any Event links to 2+ Chunks |
validate_field_lengths_for_sag() | Validate string fields against SAG column limits |
find_entity_dedup_matches() | Find duplicate Entity names |
classify_ids() | Count ID types (octx / v4 / other) |
validate_sag_structured() | Check complete coverage (Section 6 — no orphans) |
validate_sag_constraints() | Check structural constraints (Sections 3-5) |
write_structured_to_workspace() | Write and validate structured data to an OCTX workspace |
Data Types
| Type | Description |
|---|---|
SagFieldProfile | SAG database column length limits per field |
SagCardinalityIssue | An Event linked to 2+ Chunks |
SagFieldIssue | A record field exceeding its column length |
SagDedupMatch | Two Entities flagged as duplicates by name |
SagCoverageIssue | A complete-coverage rule violation |
SagConstraintIssue | A structural constraint violation |
SagAlignmentReport | Aggregated alignment check report |
translate_id()
def translate_id(uuid_v4: str) -> strMap a UUIDv4 to a UUIDv7 by changing the version nibble (4→7).
The result is a reversible stable identity that can be used inside an OCTX
Package and later mapped back via revert_id().
| Parameter | Description |
|---|---|
uuid_v4 | A valid UUIDv4 string (36 characters) |
Returns: UUIDv7 string with the version nibble changed, lowercased.
Raises: ValueError if the input is not a valid UUIDv4.
from octx.sag_align import translate_id
v7 = translate_id("550e8400-e29b-41d4-a716-446655440000")
# → "550e8400-e29b-71d4-a716-446655440000"revert_id()
def revert_id(uuid_v7: str) -> strReverse translate_id() — back from OCTX-form ID to UUIDv4.
This is a transitional backward-compatibility tool for the period
before SAG has fully migrated to UUIDv7.
| Parameter | Description |
|---|---|
uuid_v7 | UUID string with version nibble 7 |
Raises: ValueError if the input is not a valid translated OCTX ID.
from octx.sag_align import revert_id
v4 = revert_id("550e8400-e29b-71d4-a716-446655440000")
# → "550e8400-e29b-41d4-a716-446655440000"ensure_octx_ids()
def ensure_octx_ids(
records: Iterable[dict[str, Any]],
*,
id_fields: Collection[str] = {"id", "document_id", "chunk_id", "event_id", "entity_id", "parent_id"},
) -> Iterator[dict[str, Any]]Batch-process records, transparently converting UUIDv4 fields in id_fields
to UUIDv7. Non-dict objects, already-UUIDv7 values, and non-UUID values
pass through unchanged.
| Parameter | Description |
|---|---|
records | Iterable of dict records |
id_fields | Keyword-only; set of field names to convert |
from octx.sag_align import ensure_octx_ids
records = [{"id": "550e8400-e29b-41d4-a716-446655440000", "title": "doc"}]
result = list(ensure_octx_ids(records))
# result[0]["id"] → "550e8400-e29b-71d4-a716-446655440000"default_sag_field_profile()
def default_sag_field_profile() -> SagFieldProfileReturn the default SagFieldProfile instance.
from octx.sag_align import default_sag_field_profile
profile = default_sag_field_profile()
# SagFieldProfile(id=36, name=512, entity_type=100, ...)validate_cardinality_for_sag()
def validate_cardinality_for_sag(
chunk_events: Iterable[dict[str, Any]],
*,
events: Collection[str] | None = None,
) -> list[SagCardinalityIssue]Find Events that are linked to more than one Chunk. The SAG data model stores exactly one source Chunk per Event, so OCTX packages with N:M relations need explicit handling during import.
| Parameter | Description |
|---|---|
chunk_events | Records from relations/chunk-events.jsonl |
events | Keyword-only; optional restrict-list of Event IDs |
Returns: One SagCardinalityIssue per offending Event, sorted by Event ID.
from octx.sag_align import validate_cardinality_for_sag
issues = validate_cardinality_for_sag(chunk_events)
for issue in issues:
print(issue.event_id, issue.chunk_ids)validate_field_lengths_for_sag()
def validate_field_lengths_for_sag(
chunks: Iterable[dict[str, Any]],
events: Iterable[dict[str, Any]],
entities: Iterable[dict[str, Any]],
chunk_events: Iterable[dict[str, Any]],
event_entities: Iterable[dict[str, Any]],
*,
profile: SagFieldProfile | None = None,
) -> list[SagFieldIssue]Check all text fields across the five JSONL files against SAG database
column length limits. A limit of 0 means unlimited (e.g. chunk.text).
| Parameter | Description |
|---|---|
chunks | data/chunks.jsonl records |
events | data/events.jsonl records |
entities | data/entities.jsonl records |
chunk_events | relations/chunk-events.jsonl records |
event_entities | relations/event-entities.jsonl records |
profile | Keyword-only; custom field profile, defaults to default_sag_field_profile() |
from octx.sag_align import validate_field_lengths_for_sag
issues = validate_field_lengths_for_sag(chunks, events, entities, ce, ee)find_entity_dedup_matches()
def find_entity_dedup_matches(
entities: Iterable[dict[str, Any]],
*,
normalize: bool = True,
) -> list[SagDedupMatch]Find Entity records with identical name values (NFC normalisation +
optional case folding). Only the first pair per name is reported.
| Parameter | Description |
|---|---|
entities | data/entities.jsonl records |
normalize | Keyword-only; when True, NFC-normalise and case-fold for comparison |
from octx.sag_align import find_entity_dedup_matches
matches = find_entity_dedup_matches(entities)classify_ids()
def classify_ids(
records: Iterable[dict[str, Any]],
*,
id_fields: Collection[str] = {"id", "document_id", "chunk_id", "event_id", "entity_id", "parent_id"},
) -> dict[str, int]Count ID types across records. Returns {"octx": N, "v4": N, "other": N}.
| Parameter | Description |
|---|---|
records | Iterable of dict records |
id_fields | Keyword-only; set of field names to inspect |
from octx.sag_align import classify_ids
stats = classify_ids(all_records)
# {"octx": 5, "v4": 2, "other": 0}validate_sag_structured()
def validate_sag_structured(
chunks: Iterable[dict[str, Any]],
events: Iterable[dict[str, Any]],
entities: Iterable[dict[str, Any]],
chunk_events: Iterable[dict[str, Any]],
event_entities: Iterable[dict[str, Any]],
*,
document_ids: Collection[str] | None = None,
) -> list[SagCoverageIssue]Check sag-structured/0.1 Section 6 complete coverage — no orphan records
in the chain:
- document_has_chunk — Every Concept Document has at least one Chunk
- chunk_in_relation — Every Chunk appears in a Chunk-Event relation
- event_in_chunk_relation — Every Event appears in a Chunk-Event relation
- event_in_entity_relation — Every Event appears in an Event-Entity relation
- entity_in_relation — Every Entity appears in an Event-Entity relation
| Parameter | Description |
|---|---|
chunks, events, entities, chunk_events, event_entities | Records from the five JSONL files |
document_ids | Keyword-only; optional, when provided checks Rule 1 |
from octx.sag_align import validate_sag_structured
issues = validate_sag_structured(chunks, events, entities, ce, ee, document_ids=["doc-1"])
for issue in issues:
print(issue.rule, issue.record_id)validate_sag_constraints()
def validate_sag_constraints(
chunks: Iterable[dict[str, Any]],
events: Iterable[dict[str, Any]],
entities: Iterable[dict[str, Any]],
chunk_events: Iterable[dict[str, Any]],
event_entities: Iterable[dict[str, Any]],
) -> list[SagConstraintIssue]Check sag-structured/0.1 Sections 3-5 structural constraints:
Chunks — Required fields (id, document_id, ordinal, text), non-empty text, ordinal uniqueness within a document
Events — Required fields (id, title, content), non-empty title/content, parent_id/level consistency, child event level = parent.level + 1, no cycles
Relations — (chunk_id, event_id) and (event_id, entity_id) pairs must be unique
Referential integrity — All chunk_id / event_id / entity_id references must exist
UUIDv7 format — All ID fields must be canonical lowercased UUIDv7
from octx.sag_align import validate_sag_constraints
issues = validate_sag_constraints(chunks, events, entities, ce, ee)
for issue in issues:
print(issue.code, issue.record_id)write_structured_to_workspace()
def write_structured_to_workspace(
workspace: str | Path,
*,
chunks: Iterable[dict[str, Any]] = (),
events: Iterable[dict[str, Any]] = (),
entities: Iterable[dict[str, Any]] = (),
chunk_events: Iterable[dict[str, Any]] = (),
event_entities: Iterable[dict[str, Any]] = (),
profile: SagFieldProfile | None = None,
) -> SagAlignmentReportWrite external structured records to an OCTX workspace:
- All UUIDv4 IDs are transparently mapped to UUIDv7
- Cardinality, field-length, dedup, coverage, and constraint checks are performed
- Non-empty streams are written to their corresponding JSONL files
- Returns an aggregated validation report
After writing, the workspace is ready for create_octx().
| Parameter | Description |
|---|---|
workspace | Existing OCTX workspace path |
chunks / events / entities / chunk_events / event_entities | Keyword-only; stream records |
profile | Keyword-only; custom field profile |
from octx.sag_align import write_structured_to_workspace
report = write_structured_to_workspace(
"my-workspace",
chunks=[...],
events=[...],
entities=[...],
chunk_events=[...],
event_entities=[...],
)SagFieldProfile
@dataclass(frozen=True)
class SagFieldProfile:
id: int = 36 # UUID string length
name: int = 512 # entity.name
entity_type: int = 100 # entity.type
title: int = 512 # event.title, document.title
summary: int = 4096 # event.summary
category: int = 128 # event.category
description: int = 4096 # entity.description, relation description
text: int = 0 # chunk.text / event.content (0 = unlimited)SagCardinalityIssue
@dataclass(frozen=True)
class SagCardinalityIssue:
event_id: str
chunk_ids: tuple[str, ...] # All Chunk IDs linked to this Event
primary_hint: str | None = None # Suggested primary ChunkSagFieldIssue
@dataclass(frozen=True)
class SagFieldIssue:
path: str # JSONL file path (e.g. "data/entities.jsonl")
line: int # 1-indexed line number
record_id: str | None
field: str # Field name exceeding the limit
value_length: int
limit: intSagDedupMatch
@dataclass(frozen=True)
class SagDedupMatch:
entity_id_a: str
entity_id_b: str
name: str # Matching name value
same_package: bool = True # Duplicate within the same packageSagCoverageIssue
@dataclass(frozen=True)
class SagCoverageIssue:
rule: str # Rule identifier (see validate_sag_structured)
record_id: str | None # ID of the offending record
description: str # Human-readable explanationSagConstraintIssue
@dataclass(frozen=True)
class SagConstraintIssue:
code: str # Constraint code (e.g. chunk_text_empty)
record_id: str | None # ID of the offending record
description: str # Human-readable explanationSagAlignmentReport
@dataclass
class SagAlignmentReport:
cardinality_issues: list[SagCardinalityIssue]
field_issues: list[SagFieldIssue]
dedup_within: list[SagDedupMatch]
coverage_issues: list[SagCoverageIssue]
constraint_issues: list[SagConstraintIssue]
id_translations: int # Number of UUIDv4→v7 translations
unchanged_ids: int # Number of IDs left unchanged
# Properties
has_cardinality_conflicts: bool # cardinality_issues non-empty?
has_field_issues: bool # field_issues non-empty?
has_dedup_within: bool # dedup_within non-empty?
has_coverage_issues: bool # coverage_issues non-empty?
has_constraint_issues: bool # constraint_issues non-empty?
# Methods
def merge(self, other: SagAlignmentReport) -> SagAlignmentReport
# Merge two reports (e.g. from different packages)