API/sag_align

Core Functions

sag_align

SAG ↔ OCTX data model alignment: ID mapping, cardinality validation, field-length checks, entity deduplication, and structured data export.

sag_align provides alignment utilities between the SAG application data model and the OCTX sag-structured/0.1 Capability. It is used for import/export scenarios including ID mapping, cardinality validation, field-length checks, entity deduplication, and structured data export.

Functions

FunctionPurpose
translate_id()UUIDv4 → UUIDv7 (bijective reversible mapping)
revert_id()UUIDv7 → UUIDv4 (transitional backward compatibility)
ensure_octx_ids()Batch-convert UUIDv4 fields in records
default_sag_field_profile()Return the default SAG field profile
validate_cardinality_for_sag()Check if any Event links to 2+ Chunks
validate_field_lengths_for_sag()Validate string fields against SAG column limits
find_entity_dedup_matches()Find duplicate Entity names
classify_ids()Count ID types (octx / v4 / other)
validate_sag_structured()Check complete coverage (Section 6 — no orphans)
validate_sag_constraints()Check structural constraints (Sections 3-5)
write_structured_to_workspace()Write and validate structured data to an OCTX workspace

Data Types

TypeDescription
SagFieldProfileSAG database column length limits per field
SagCardinalityIssueAn Event linked to 2+ Chunks
SagFieldIssueA record field exceeding its column length
SagDedupMatchTwo Entities flagged as duplicates by name
SagCoverageIssueA complete-coverage rule violation
SagConstraintIssueA structural constraint violation
SagAlignmentReportAggregated alignment check report

translate_id()

python
def translate_id(uuid_v4: str) -> str

Map a UUIDv4 to a UUIDv7 by changing the version nibble (4→7). The result is a reversible stable identity that can be used inside an OCTX Package and later mapped back via revert_id().

ParameterDescription
uuid_v4A valid UUIDv4 string (36 characters)

Returns: UUIDv7 string with the version nibble changed, lowercased.

Raises: ValueError if the input is not a valid UUIDv4.

python
from octx.sag_align import translate_id

v7 = translate_id("550e8400-e29b-41d4-a716-446655440000")
# → "550e8400-e29b-71d4-a716-446655440000"

revert_id()

python
def revert_id(uuid_v7: str) -> str

Reverse translate_id() — back from OCTX-form ID to UUIDv4. This is a transitional backward-compatibility tool for the period before SAG has fully migrated to UUIDv7.

ParameterDescription
uuid_v7UUID string with version nibble 7

Raises: ValueError if the input is not a valid translated OCTX ID.

python
from octx.sag_align import revert_id

v4 = revert_id("550e8400-e29b-71d4-a716-446655440000")
# → "550e8400-e29b-41d4-a716-446655440000"

ensure_octx_ids()

python
def ensure_octx_ids(
    records: Iterable[dict[str, Any]],
    *,
    id_fields: Collection[str] = {"id", "document_id", "chunk_id", "event_id", "entity_id", "parent_id"},
) -> Iterator[dict[str, Any]]

Batch-process records, transparently converting UUIDv4 fields in id_fields to UUIDv7. Non-dict objects, already-UUIDv7 values, and non-UUID values pass through unchanged.

ParameterDescription
recordsIterable of dict records
id_fieldsKeyword-only; set of field names to convert
python
from octx.sag_align import ensure_octx_ids

records = [{"id": "550e8400-e29b-41d4-a716-446655440000", "title": "doc"}]
result = list(ensure_octx_ids(records))
# result[0]["id"] → "550e8400-e29b-71d4-a716-446655440000"

default_sag_field_profile()

python
def default_sag_field_profile() -> SagFieldProfile

Return the default SagFieldProfile instance.

python
from octx.sag_align import default_sag_field_profile

profile = default_sag_field_profile()
# SagFieldProfile(id=36, name=512, entity_type=100, ...)

validate_cardinality_for_sag()

python
def validate_cardinality_for_sag(
    chunk_events: Iterable[dict[str, Any]],
    *,
    events: Collection[str] | None = None,
) -> list[SagCardinalityIssue]

Find Events that are linked to more than one Chunk. The SAG data model stores exactly one source Chunk per Event, so OCTX packages with N:M relations need explicit handling during import.

ParameterDescription
chunk_eventsRecords from relations/chunk-events.jsonl
eventsKeyword-only; optional restrict-list of Event IDs

Returns: One SagCardinalityIssue per offending Event, sorted by Event ID.

python
from octx.sag_align import validate_cardinality_for_sag

issues = validate_cardinality_for_sag(chunk_events)
for issue in issues:
    print(issue.event_id, issue.chunk_ids)

validate_field_lengths_for_sag()

python
def validate_field_lengths_for_sag(
    chunks: Iterable[dict[str, Any]],
    events: Iterable[dict[str, Any]],
    entities: Iterable[dict[str, Any]],
    chunk_events: Iterable[dict[str, Any]],
    event_entities: Iterable[dict[str, Any]],
    *,
    profile: SagFieldProfile | None = None,
) -> list[SagFieldIssue]

Check all text fields across the five JSONL files against SAG database column length limits. A limit of 0 means unlimited (e.g. chunk.text).

ParameterDescription
chunksdata/chunks.jsonl records
eventsdata/events.jsonl records
entitiesdata/entities.jsonl records
chunk_eventsrelations/chunk-events.jsonl records
event_entitiesrelations/event-entities.jsonl records
profileKeyword-only; custom field profile, defaults to default_sag_field_profile()
python
from octx.sag_align import validate_field_lengths_for_sag

issues = validate_field_lengths_for_sag(chunks, events, entities, ce, ee)

find_entity_dedup_matches()

python
def find_entity_dedup_matches(
    entities: Iterable[dict[str, Any]],
    *,
    normalize: bool = True,
) -> list[SagDedupMatch]

Find Entity records with identical name values (NFC normalisation + optional case folding). Only the first pair per name is reported.

ParameterDescription
entitiesdata/entities.jsonl records
normalizeKeyword-only; when True, NFC-normalise and case-fold for comparison
python
from octx.sag_align import find_entity_dedup_matches

matches = find_entity_dedup_matches(entities)

classify_ids()

python
def classify_ids(
    records: Iterable[dict[str, Any]],
    *,
    id_fields: Collection[str] = {"id", "document_id", "chunk_id", "event_id", "entity_id", "parent_id"},
) -> dict[str, int]

Count ID types across records. Returns {"octx": N, "v4": N, "other": N}.

ParameterDescription
recordsIterable of dict records
id_fieldsKeyword-only; set of field names to inspect
python
from octx.sag_align import classify_ids

stats = classify_ids(all_records)
# {"octx": 5, "v4": 2, "other": 0}

validate_sag_structured()

python
def validate_sag_structured(
    chunks: Iterable[dict[str, Any]],
    events: Iterable[dict[str, Any]],
    entities: Iterable[dict[str, Any]],
    chunk_events: Iterable[dict[str, Any]],
    event_entities: Iterable[dict[str, Any]],
    *,
    document_ids: Collection[str] | None = None,
) -> list[SagCoverageIssue]

Check sag-structured/0.1 Section 6 complete coverage — no orphan records in the chain:

  1. document_has_chunk — Every Concept Document has at least one Chunk
  2. chunk_in_relation — Every Chunk appears in a Chunk-Event relation
  3. event_in_chunk_relation — Every Event appears in a Chunk-Event relation
  4. event_in_entity_relation — Every Event appears in an Event-Entity relation
  5. entity_in_relation — Every Entity appears in an Event-Entity relation
ParameterDescription
chunks, events, entities, chunk_events, event_entitiesRecords from the five JSONL files
document_idsKeyword-only; optional, when provided checks Rule 1
python
from octx.sag_align import validate_sag_structured

issues = validate_sag_structured(chunks, events, entities, ce, ee, document_ids=["doc-1"])
for issue in issues:
    print(issue.rule, issue.record_id)

validate_sag_constraints()

python
def validate_sag_constraints(
    chunks: Iterable[dict[str, Any]],
    events: Iterable[dict[str, Any]],
    entities: Iterable[dict[str, Any]],
    chunk_events: Iterable[dict[str, Any]],
    event_entities: Iterable[dict[str, Any]],
) -> list[SagConstraintIssue]

Check sag-structured/0.1 Sections 3-5 structural constraints:

Chunks — Required fields (id, document_id, ordinal, text), non-empty text, ordinal uniqueness within a document

Events — Required fields (id, title, content), non-empty title/content, parent_id/level consistency, child event level = parent.level + 1, no cycles

Relations — (chunk_id, event_id) and (event_id, entity_id) pairs must be unique

Referential integrity — All chunk_id / event_id / entity_id references must exist

UUIDv7 format — All ID fields must be canonical lowercased UUIDv7

python
from octx.sag_align import validate_sag_constraints

issues = validate_sag_constraints(chunks, events, entities, ce, ee)
for issue in issues:
    print(issue.code, issue.record_id)

write_structured_to_workspace()

python
def write_structured_to_workspace(
    workspace: str | Path,
    *,
    chunks: Iterable[dict[str, Any]] = (),
    events: Iterable[dict[str, Any]] = (),
    entities: Iterable[dict[str, Any]] = (),
    chunk_events: Iterable[dict[str, Any]] = (),
    event_entities: Iterable[dict[str, Any]] = (),
    profile: SagFieldProfile | None = None,
) -> SagAlignmentReport

Write external structured records to an OCTX workspace:

  1. All UUIDv4 IDs are transparently mapped to UUIDv7
  2. Cardinality, field-length, dedup, coverage, and constraint checks are performed
  3. Non-empty streams are written to their corresponding JSONL files
  4. Returns an aggregated validation report

After writing, the workspace is ready for create_octx().

ParameterDescription
workspaceExisting OCTX workspace path
chunks / events / entities / chunk_events / event_entitiesKeyword-only; stream records
profileKeyword-only; custom field profile
python
from octx.sag_align import write_structured_to_workspace

report = write_structured_to_workspace(
    "my-workspace",
    chunks=[...],
    events=[...],
    entities=[...],
    chunk_events=[...],
    event_entities=[...],
)

SagFieldProfile

python
@dataclass(frozen=True)
class SagFieldProfile:
    id: int = 36            # UUID string length
    name: int = 512         # entity.name
    entity_type: int = 100  # entity.type
    title: int = 512        # event.title, document.title
    summary: int = 4096     # event.summary
    category: int = 128     # event.category
    description: int = 4096 # entity.description, relation description
    text: int = 0           # chunk.text / event.content (0 = unlimited)

SagCardinalityIssue

python
@dataclass(frozen=True)
class SagCardinalityIssue:
    event_id: str
    chunk_ids: tuple[str, ...]  # All Chunk IDs linked to this Event
    primary_hint: str | None = None  # Suggested primary Chunk

SagFieldIssue

python
@dataclass(frozen=True)
class SagFieldIssue:
    path: str               # JSONL file path (e.g. "data/entities.jsonl")
    line: int               # 1-indexed line number
    record_id: str | None
    field: str              # Field name exceeding the limit
    value_length: int
    limit: int

SagDedupMatch

python
@dataclass(frozen=True)
class SagDedupMatch:
    entity_id_a: str
    entity_id_b: str
    name: str               # Matching name value
    same_package: bool = True  # Duplicate within the same package

SagCoverageIssue

python
@dataclass(frozen=True)
class SagCoverageIssue:
    rule: str               # Rule identifier (see validate_sag_structured)
    record_id: str | None   # ID of the offending record
    description: str        # Human-readable explanation

SagConstraintIssue

python
@dataclass(frozen=True)
class SagConstraintIssue:
    code: str               # Constraint code (e.g. chunk_text_empty)
    record_id: str | None   # ID of the offending record
    description: str        # Human-readable explanation

SagAlignmentReport

python
@dataclass
class SagAlignmentReport:
    cardinality_issues: list[SagCardinalityIssue]
    field_issues: list[SagFieldIssue]
    dedup_within: list[SagDedupMatch]
    coverage_issues: list[SagCoverageIssue]
    constraint_issues: list[SagConstraintIssue]
    id_translations: int        # Number of UUIDv4→v7 translations
    unchanged_ids: int          # Number of IDs left unchanged

    # Properties
    has_cardinality_conflicts: bool  # cardinality_issues non-empty?
    has_field_issues: bool           # field_issues non-empty?
    has_dedup_within: bool           # dedup_within non-empty?
    has_coverage_issues: bool        # coverage_issues non-empty?
    has_constraint_issues: bool      # constraint_issues non-empty?

    # Methods
    def merge(self, other: SagAlignmentReport) -> SagAlignmentReport
        # Merge two reports (e.g. from different packages)