triplets.tools¶

Triplet data manipulation tools with pandas/polars/duckdb engine support.

Provides query, filter, diff, transform, and mutate operations on triplet DataFrames ([ID, KEY, VALUE, INSTANCE_ID]).

Engines: - pandas_engine (default, always available) - polars_engine (optional, uses polars-native operations for speed) - duckdb_engine (optional, connection-first: functions take the connection)

Every public function here dispatches by the input object’s flavor (pandas / polars DataFrame or DuckDB connection), or by an explicit engine= name that must match the input. Frame ops always run in the input’s engine — never auto-hop flavors. Methods registered on DataFrames and connections (see triplets._accessor) bind the engine functions directly — the object a method is called on already determines the engine.

triplets.tools.content_hash(data, ignore_types=('Distribution', 'NamespaceMap', 'FullModel'), columns=('ID', 'KEY', 'VALUE'), order_sensitive=False, *, engine='auto')¶

Deterministic identity hash of the triplet content (blake2b hex).

Row-order-invariant by default: vectorized row hashes (hash_pandas_object) combined with commutative count/sum/xor — no sort, single pass. With order_sensitive=True the row hashes are digested in row order, so the same rows in a different order produce a different digest. Engine-specific — the pandas/polars/duckdb engines use their native row-hash primitives for speed, so digests are stable within an engine and library version but NOT comparable across engines. ignore_types drops whole objects of volatile metadata types (export bookkeeping like FullModel timestamps), so the same grid content hashes the same across re-exports; pass () to hash everything. columns excludes INSTANCE_ID by default, making the hash independent of how the content is split across instances.

triplets.tools.diff_between_INSTANCE(data, INSTANCE_ID_1, INSTANCE_ID_2, *, engine='auto')¶

Identify differences between two loaded INSTANCES, by thier INSTACE_ID in the same Triplet DataFrame.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing two or more INSTANCE.
INSTANCE_ID_1 (str) – UUID of the first INSTANCE.
INSTANCE_ID_2 (str) – UUID of the second INSTANCE.

Returns:

DataFrame containing triplets that differ between the two model parts.

Return type:

pandas.DataFrame

Examples

>>> diff = diff_triplets_by_instance('uuid1', 'uuid2')

triplets.tools.diff_between_triplet(old_data, new_data, *, engine='auto')¶

Compute the difference between two Triplet DataFrames.

Parameters:

old_data (pandas.DataFrame) – Original triplet dataset.
new_data (pandas.DataFrame) – New triplet dataset to compare against.

Returns:

DataFrame containing triplets unique to old_data or new_data, with an ‘_merge’ column indicating ‘left_only’ (in old_data) or ‘right_only’ (in new_data).

Return type:

pandas.DataFrame

Examples

>>> diff = diff_triplets(old_data, new_data)

triplets.tools.diff_triplets(old_data, new_data, *, engine='auto')¶

Compute the difference between two Triplet DataFrames.

Parameters:

old_data (pandas.DataFrame) – Original triplet dataset.
new_data (pandas.DataFrame) – New triplet dataset to compare against.

Returns:

DataFrame containing triplets unique to old_data or new_data, with an ‘_merge’ column indicating ‘left_only’ (in old_data) or ‘right_only’ (in new_data).

Return type:

pandas.DataFrame

Examples

>>> diff = diff_triplets(old_data, new_data)

triplets.tools.diff_triplets_by_instance(data, INSTANCE_ID_1, INSTANCE_ID_2, *, engine='auto')¶

Identify differences between two loaded INSTANCES, by thier INSTACE_ID in the same Triplet DataFrame.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing two or more INSTANCE.
INSTANCE_ID_1 (str) – UUID of the first INSTANCE.
INSTANCE_ID_2 (str) – UUID of the second INSTANCE.

Returns:

DataFrame containing triplets that differ between the two model parts.

Return type:

pandas.DataFrame

Examples

>>> diff = diff_triplets_by_instance('uuid1', 'uuid2')

triplets.tools.filter_by_triplet(data, filter_triplet, *, engine='auto')¶

Filter riplet DataFrame using IDs from another DataFrame.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing CGMES data.
filter_triplet (pandas.DataFrame) – DataFrame containing atleast colum ID to filter by.

Returns:

Filtered DataFrame with columns [‘ID, ‘KEY’, ‘VALUE’, ‘INSTANCE_ID’].

Return type:

pandas.DataFrame

Examples

>>> filtered = filter_triplets_by_triplets(data, filter_triplet)

triplets.tools.filter_by_type(data, type_name, type_key='Type', *, engine='auto')¶

Filter triplet dataset by objects of a specific type.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
type_name (str) – Object type to filter by (e.g., ‘ACLineSegment’).
type_key (str) – Key used in triplet to indicate type, by default “Type”

Returns:

Filtered triplet dataset containing only objects of the specified type.

Return type:

pandas.DataFrame

Examples

>>> filtered = filter_triplets_by_type(data, "ACLineSegment")

triplets.tools.filter_triplets(data, ID=None, KEY=None, VALUE=None, INSTANCE_ID=None, regex=False, *, engine='auto')¶

Filter triplets by any combination of columns with optional regex.

Parameters:

data (pandas.DataFrame) – Triplet dataset with columns [ID, KEY, VALUE, INSTANCE_ID].
ID (str or list of str, optional) – Filter value. A list keeps rows matching any of its values. With regex=True the value(s) are regex patterns (re.search).
KEY (str or list of str, optional) – Filter value. A list keeps rows matching any of its values. With regex=True the value(s) are regex patterns (re.search).
VALUE (str or list of str, optional) – Filter value. A list keeps rows matching any of its values. With regex=True the value(s) are regex patterns (re.search).
INSTANCE_ID (str or list of str, optional) – Filter value. A list keeps rows matching any of its values. With regex=True the value(s) are regex patterns (re.search).
regex (bool, default False) – If True, use regex matching (re.search). If False, exact match.

Returns:

Filtered triplet dataset.

Return type:

pandas.DataFrame

Examples

>>> filter_triplets(data, KEY="Type", VALUE="ACLineSegment")
>>> filter_triplets(data, KEY=["Type", "IdentifiedObject.name"])
>>> filter_triplets(data, VALUE=".*Substation.*", regex=True)

triplets.tools.filter_triplets_by_triplets(data, filter_triplet, *, engine='auto')¶

Filter riplet DataFrame using IDs from another DataFrame.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing CGMES data.
filter_triplet (pandas.DataFrame) – DataFrame containing atleast colum ID to filter by.

Returns:

Filtered DataFrame with columns [‘ID, ‘KEY’, ‘VALUE’, ‘INSTANCE_ID’].

Return type:

pandas.DataFrame

Examples

>>> filtered = filter_triplets_by_triplets(data, filter_triplet)

triplets.tools.filter_triplets_by_type(data, type_name, type_key='Type', *, engine='auto')¶

Filter triplet dataset by objects of a specific type.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
type_name (str) – Object type to filter by (e.g., ‘ACLineSegment’).
type_key (str) – Key used in triplet to indicate type, by default “Type”

Returns:

Filtered triplet dataset containing only objects of the specified type.

Return type:

pandas.DataFrame

Examples

>>> filtered = filter_triplets_by_type(data, "ACLineSegment")

triplets.tools.filter_triplets_by_value(data, VALUE, detailed=False, type_key='Type', regex=False, *, engine='auto')¶

Filter to objects that have any triplet matching VALUE.

Selects every object (ID) with at least one triplet whose VALUE matches VALUE, then summarizes those objects.

Parameters:

data (pandas.DataFrame) – Triplet dataset with columns [ID, KEY, VALUE, INSTANCE_ID].
VALUE (str) – Value to match. Exact match by default; regex pattern if regex=True.
detailed (bool, default False) – If True, return all triplets of the matched objects. If False, return only the matching rows plus each matched object’s type_key row, sorted by ID with the type row first within each ID group.
type_key (str) – Key identifying each object’s type row, by default “Type”.
regex (bool, default False) – If True, use regex matching (re.search). If False, exact match.

Returns:

Filtered triplet dataset.

Return type:

pandas.DataFrame

Examples

>>> filter_triplets_by_value(data, "DOWGEN_LD1", regex=True)

triplets.tools.get_namespace_map(data: DataFrame, *, engine='auto')¶

Extract namespace prefix-to-URI mapping and optional xml:base from a triplet dataset.

This function searches for a NamespaceMap object (identified by KEY='Type' and VALUE='NamespaceMap') within the dataset. It then collects all key-value pairs under that instance where: - KEY is the namespace prefix (e.g., “cim”, “rdf”) - VALUE is the full URI (e.g., “http://iec.ch/TC57/2013/CIM-schema-cim16#”)

Special keys: - xml_base: Extracted separately if present (used as base URI in RDF). - Type: Automatically excluded.

Parameters:

data (pandas.DataFrame) – Triplet dataset with columns [‘INSTANCE_ID’, ‘ID’, ‘KEY’, ‘VALUE’]. Must contain a NamespaceMap instance for successful extraction.

Returns:

namespace_map (dict) – Mapping of namespace prefixes to URIs (e.g., {"cim": "...", "rdf": "..."}). Empty dict if no NamespaceMap is found.
xml_base (str) – Value of xml_base if defined within the NamespaceMap; otherwise empty str.

Examples

>>> ns_map, base = get_namespace_map(triplet_data)
>>> print(ns_map)
{'cim': 'http://iec.ch/TC57/2013/CIM-schema-cim16#', 'rdf': 'http://www.w3.org/1999/02/22-rdf-syntax-ns#'}
>>> print(base)
'http://example.com/base/'

>>> ns_map, base = get_namespace_map(empty_data)
>>> print(ns_map, base)
{} ""

Notes

The function is idempotent and safe to call on any dataset.
Uses inner merge on ID to scope entries to the correct NamespaceMap instance.
Always returns a tuple of length 2: (dict, str).

triplets.tools.get_object_data(data, object_UUID, *, engine='auto')¶

Retrieve data for a specific object by its UUID.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
object_UUID (str) – UUID of the object to retrieve.

Returns:

Series with keys as index and values for the specified object.

Return type:

pandas.Series

Examples

>>> obj_data = data.get_object_data("uuid1")

triplets.tools.get_types_count(data, contains=None, case_insensitive=True, *, engine='auto')¶

Return a dictionary of object types and their occurrence counts.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
contains (str, optional) – If given, keep only types whose name contains this substring.
case_insensitive (bool, default True) – If True, the contains match ignores letter case.

Returns:

Dictionary with object types as keys and their counts as values.

Return type:

dict

Examples

>>> types = data.types_dict()
>>> print(types)
{'ACLineSegment': 10, 'PowerTransformer': 5, ...}
>>> data.types_dict(contains="Settlement")
{'MarketEvaluationPoint': 3, ...}

triplets.tools.id_tableview(data, id, string_to_number=True, multivalue=False, *, engine='auto')¶

Create a tabular view of a CGMES triplet dataset filtered by ID-s.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing CGMES data.
id (str or list or pandas.DataFrame) – ID(s) to filter by (single ID, list of IDs, or DataFrame with an ID column).
string_to_number (bool, optional) – If True, convert columns containing numbers to numeric types (default is True).
multivalue (bool, optional) – If True, aggregate duplicate (ID, KEY) pairs into lists (default is False).

Returns:

Pivoted DataFrame with IDs as index and KEYs as columns.

Return type:

pandas.DataFrame or None

Examples

>>> table = id_tableview(data, 'UUID')
>>> table = id_tableview(data, ['UUID_1', 'UUID_2'])
>>> table = id_tableview(data, pandas.DataFrame({"ID": ['UUID_1', 'UUID_2']}))

triplets.tools.key_tableview(data, key, string_to_number=True, multivalue=False, *, engine='auto')¶

Create a table view of all objects with a specified key.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
key (str) – The key to filter objects by (e.g., ‘GeneratingUnit.maxOperatingP’).
string_to_number (bool, optional) – If True, convert columns containing numbers to numeric types (default is True).
multivalue (bool, optional) – If True, aggregate duplicate (ID, KEY) pairs into lists (default is False).

Returns:

Pivoted DataFrame with IDs as index and keys as columns, or None if no data is found.

Return type:

pandas.DataFrame or None

Examples

>>> table = data.key_tableview("GeneratingUnit.maxOperatingP")

triplets.tools.print_triplet_diff(old_data, new_data, file_id_object='Distribution', file_id_key='label', exclude_objects=None, *, engine='auto')¶

Print a human-readable diff of two triplet datasets.

Parameters:

old_data (pandas.DataFrame) – Original triplet dataset.
new_data (pandas.DataFrame) – New triplet dataset to compare against.
file_id_object (str, optional) – Object type containing file identifiers (default is ‘Distribution’).
file_id_key (str, optional) – Key containing file identifiers (default is ‘label’).
exclude_objects (list, optional) – List of object types to exclude from the diff (default is None).

Notes

Outputs a diff format showing removed, added, and changed objects.
Nice diff viewer https://diffy.org/
TODO: Add name field for better reporting with Type.

Examples

>>> print_triplets_diff(old_data, new_data, exclude_objects=["NamespaceMap"])

triplets.tools.print_triplets_diff(old_data, new_data, file_id_object='Distribution', file_id_key='label', exclude_objects=None, *, engine='auto')¶

Print a human-readable diff of two triplet datasets.

Parameters:

old_data (pandas.DataFrame) – Original triplet dataset.
new_data (pandas.DataFrame) – New triplet dataset to compare against.
file_id_object (str, optional) – Object type containing file identifiers (default is ‘Distribution’).
file_id_key (str, optional) – Key containing file identifiers (default is ‘label’).
exclude_objects (list, optional) – List of object types to exclude from the diff (default is None).

Notes

Outputs a diff format showing removed, added, and changed objects.
Nice diff viewer https://diffy.org/
TODO: Add name field for better reporting with Type.

Examples

>>> print_triplets_diff(old_data, new_data, exclude_objects=["NamespaceMap"])

triplets.tools.references(data, ID, levels=1, *, engine='auto')¶

Retrieve all references (to and from) a specified object.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
ID (str) – ID of the object to find references for.
levels (int, optional) – Number of reference levels to traverse (default is 1).

Returns:

DataFrame containing triplets of all references to and from the object.

Return type:

pandas.DataFrame

Examples

>>> refs = data.references("99722373_VL_TN1", levels=2)

triplets.tools.references_all(data, *, engine='auto')¶

Find all unique references (links) in the dataset.

Parameters:: data (pandas.DataFrame) – Triplet dataset containing RDF data.
Returns:: DataFrame with columns [‘ID_FROM’, ‘KEY’, ‘ID_TO’] representing all references.
Return type:: pandas.DataFrame

Notes

Does not consider INSTANCE_ID in reference matching.

Examples

>>> refs = data.references_all()

triplets.tools.references_from(data, reference, levels=1, *, engine='auto')¶

Retrieve all objects a specified object points to.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
reference (str) – ID of the reference object.
levels (int, optional) – Number of reference levels to traverse (default is 1).

Returns:

DataFrame containing triplets of objects referenced by the input, with a ‘level’ column.

Return type:

pandas.DataFrame

Notes

TODO: Add the key on which the connection was made.

Examples

>>> refs = data.references_from("99722373_VL_TN1", levels=2)

triplets.tools.references_from_simple(data, reference, columns=['Type'], *, engine='auto')¶

Create a simplified table view of objects a specified object refers to.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
reference (str) – ID of the object to find references from.
columns (list, optional) – Columns to include in the output table (default is [‘Type’]).

Returns:

Pivoted DataFrame with IDs of referenced objects and specified columns.

Return type:

pandas.DataFrame

Examples

>>> table = data.references_from_simple("99722373_VL_TN1")

triplets.tools.references_simple(data, reference, columns=None, levels=1, *, engine='auto')¶

Create a simplified table view of all references to and from a specified object.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
reference (str) – ID of the object to find references for.
columns (list, optional) – Columns to include in the output table (default is [‘Type’, ‘IdentifiedObject.name’] if available).
levels (int, optional) – Number of reference levels to traverse (default is 1).

Returns:

Pivoted DataFrame with IDs, specified columns, and reference levels.

Return type:

pandas.DataFrame

Examples

>>> table = data.references_simple("99722373_VL_TN1", columns=["Type"])

triplets.tools.references_to(data, reference, levels=1, *, engine='auto')¶

Retrieve all objects pointing to a specified reference object.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
reference (str) – ID of the reference object.
levels (int, optional) – Number of reference levels to traverse (default is 1).

Returns:

DataFrame containing triplets of objects pointing to the reference, with a ‘level’ column.

Return type:

pandas.DataFrame

Notes

TODO: Add the key on which the connection was made.

Examples

>>> refs = data.references_to("99722373_VL_TN1", levels=2)

triplets.tools.references_to_simple(data, reference, columns=['Type'], *, engine='auto')¶

Create a simplified table view of objects referencing a specified object.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
reference (str) – ID of the object to find references to.
columns (list, optional) – Columns to include in the output table (default is [‘Type’]).

Returns:

Pivoted DataFrame with IDs of referencing objects and specified columns.

Return type:

pandas.DataFrame

Examples

>>> table = data.references_to_simple("99722373_VL_TN1")

triplets.tools.remove_triplet_from_triplet(from_triplet, what_triplet, columns=['ID', 'KEY', 'VALUE'], *, engine='auto')¶

Remove triplets from one dataset that match another.

Parameters:

from_triplet (pandas.DataFrame) – Original triplet dataset.
what_triplet (pandas.DataFrame) – Triplet dataset to remove from the original.
columns (list, optional) – Columns to match for removal (default is [‘ID’, ‘KEY’, ‘VALUE’]).

Returns:

Dataset with matching triplets removed.

Return type:

pandas.DataFrame

Examples

>>> result = remove_triplets_from_triplets(data, to_remove)

triplets.tools.remove_triplets_from_triplets(from_triplet, what_triplet, columns=['ID', 'KEY', 'VALUE'], *, engine='auto')¶

Remove triplets from one dataset that match another.

Parameters:

from_triplet (pandas.DataFrame) – Original triplet dataset.
what_triplet (pandas.DataFrame) – Triplet dataset to remove from the original.
columns (list, optional) – Columns to match for removal (default is [‘ID’, ‘KEY’, ‘VALUE’]).

Returns:

Dataset with matching triplets removed.

Return type:

pandas.DataFrame

Examples

>>> result = remove_triplets_from_triplets(data, to_remove)

triplets.tools.set_VALUE_at_KEY(data, key, value, *, engine='auto')¶

Set the value for all instances of a specified key.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
key (str) – The key to update.
value (str) – The new value to set for the specified key.

Notes

TODO: Add debug logging for key, initial value, and new value.
TODO: Store changes in a changes DataFrame.

Examples

>>> data.set_value_at_key("label", "new_label")

triplets.tools.set_VALUE_at_KEY_and_ID(data, key, value, id, *, engine='auto')¶

Set the value for a specific key and ID.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
key (str) – The key to update.
value (str) – The new value to set.
id (str) – The ID of the object to update.

Examples

>>> data.set_value_at_key_and_id("label", "new_label", "uuid1")

triplets.tools.set_value_at_key(data, key, value, *, engine='auto')¶

Set the value for all instances of a specified key.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
key (str) – The key to update.
value (str) – The new value to set for the specified key.

Notes

TODO: Add debug logging for key, initial value, and new value.
TODO: Store changes in a changes DataFrame.

Examples

>>> data.set_value_at_key("label", "new_label")

triplets.tools.set_value_at_key_and_id(data, key, value, id, *, engine='auto')¶

Set the value for a specific key and ID.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
key (str) – The key to update.
value (str) – The new value to set.
id (str) – The ID of the object to update.

Examples

>>> data.set_value_at_key_and_id("label", "new_label", "uuid1")

triplets.tools.tableview_by_id(data, id, string_to_number=True, multivalue=False, *, engine='auto')¶

Create a tabular view of a CGMES triplet dataset filtered by ID-s.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing CGMES data.
id (str or list or pandas.DataFrame) – ID(s) to filter by (single ID, list of IDs, or DataFrame with an ID column).
string_to_number (bool, optional) – If True, convert columns containing numbers to numeric types (default is True).
multivalue (bool, optional) – If True, aggregate duplicate (ID, KEY) pairs into lists (default is False).

Returns:

Pivoted DataFrame with IDs as index and KEYs as columns.

Return type:

pandas.DataFrame or None

Examples

>>> table = id_tableview(data, 'UUID')
>>> table = id_tableview(data, ['UUID_1', 'UUID_2'])
>>> table = id_tableview(data, pandas.DataFrame({"ID": ['UUID_1', 'UUID_2']}))

triplets.tools.tableview_by_key(data, key, string_to_number=True, multivalue=False, *, engine='auto')¶

Create a table view of all objects with a specified key.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
key (str) – The key to filter objects by (e.g., ‘GeneratingUnit.maxOperatingP’).
string_to_number (bool, optional) – If True, convert columns containing numbers to numeric types (default is True).
multivalue (bool, optional) – If True, aggregate duplicate (ID, KEY) pairs into lists (default is False).

Returns:

Pivoted DataFrame with IDs as index and keys as columns, or None if no data is found.

Return type:

pandas.DataFrame or None

Examples

>>> table = data.key_tableview("GeneratingUnit.maxOperatingP")

triplets.tools.tableview_by_type(data, type_name, string_to_number=True, type_key='Type', multivalue=False, *, engine='auto')¶

Create a table view of all objects of a specified type.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
type_name (str or sequence of str) – Type(s) to select (e.g. "ACLineSegment" or ("FullModel", "Dataset")). A plain string is treated as a one-item list.
string_to_number (bool, optional) – If True, convert columns containing numbers to numeric types (default is True).
type_key (str, optional) – Key used to identify object types in the dataset (default is ‘Type’).
multivalue (bool, optional) – If True, aggregate duplicate (ID, KEY) pairs into lists (default is False).

Returns:

Pivoted DataFrame with IDs as index and keys as columns, or None if no data is found.

Return type:

pandas.DataFrame or None

Examples

>>> table = data.type_tableview("ACLineSegment", multivalue=True)
>>> headers = data.type_tableview(("FullModel", "Dataset"), string_to_number=False)

triplets.tools.tableview_to_triplet(data, multivalue=False, instance_id=None, *, engine='auto')¶

Convert a table view back to a triplet format.

Parameters:

data (pandas.DataFrame) – Pivoted DataFrame (table view) to convert.
multivalue (bool, optional) – If True, unpack list values into separate triplets (default is False).
instance_id (str, optional) – If given, stamp an INSTANCE_ID column on the result (default None).

Returns:

Triplet DataFrame with columns [‘ID’, ‘KEY’, ‘VALUE’] (plus ‘INSTANCE_ID’ when instance_id is given).

Return type:

pandas.DataFrame

Notes

An empty tableview cell is not a triplet — those holes are dropped so this is a faithful inverse of the tableview build (matches the duckdb engine’s WHERE VALUE IS NOT NULL). INSTANCE_ID is not carried by a tableview; pass instance_id to stamp it, the same way update_triplets_from_tableview does.

triplets.tools.tableview_to_triplets(data, multivalue=False, instance_id=None, *, engine='auto')¶

Convert a table view back to a triplet format.

Parameters:

data (pandas.DataFrame) – Pivoted DataFrame (table view) to convert.
multivalue (bool, optional) – If True, unpack list values into separate triplets (default is False).
instance_id (str, optional) – If given, stamp an INSTANCE_ID column on the result (default None).

Returns:

Triplet DataFrame with columns [‘ID’, ‘KEY’, ‘VALUE’] (plus ‘INSTANCE_ID’ when instance_id is given).

Return type:

pandas.DataFrame

Notes

triplets.tools.tableviews_to_triplet(tableviews, multivalue=False, *, engine='auto')¶

Convert dict of tableview DataFrames to triplet DataFrame.

Parameters:

tableviews (dict) – {class_name: tableview_df}
multivalue (bool, default False) – If True, unpack list values into separate triplets.

Returns:

Triplet DataFrame with columns [ID, KEY, VALUE, INSTANCE_ID].

Return type:

pandas.DataFrame

triplets.tools.tableviews_to_triplets(tableviews, multivalue=False, *, engine='auto')¶

Convert dict of tableview DataFrames to triplet DataFrame.

Parameters:

tableviews (dict) – {class_name: tableview_df}
multivalue (bool, default False) – If True, unpack list values into separate triplets.

Returns:

Triplet DataFrame with columns [ID, KEY, VALUE, INSTANCE_ID].

Return type:

pandas.DataFrame

triplets.tools.triplet_to_tableviews(triplet_df, multivalue=False, *, engine='auto')¶

Convert triplet DataFrame to dict of tableview DataFrames.

Parameters:

triplet_df (pandas.DataFrame) – Triplet dataset with columns [ID, KEY, VALUE, INSTANCE_ID].
multivalue (bool, default False) – If True, aggregate duplicate (ID, KEY) pairs into lists.

Returns:

{class_name: tableview_df}

Return type:

dict

triplets.tools.triplets_to_tableviews(triplet_df, multivalue=False, *, engine='auto')¶

Convert triplet DataFrame to dict of tableview DataFrames.

Parameters:

triplet_df (pandas.DataFrame) – Triplet dataset with columns [ID, KEY, VALUE, INSTANCE_ID].
multivalue (bool, default False) – If True, aggregate duplicate (ID, KEY) pairs into lists.

Returns:

{class_name: tableview_df}

Return type:

dict

triplets.tools.type_tableview(data, type_name, string_to_number=True, type_key='Type', multivalue=False, *, engine='auto')¶

Create a table view of all objects of a specified type.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
type_name (str or sequence of str) – Type(s) to select (e.g. "ACLineSegment" or ("FullModel", "Dataset")). A plain string is treated as a one-item list.
string_to_number (bool, optional) – If True, convert columns containing numbers to numeric types (default is True).
type_key (str, optional) – Key used to identify object types in the dataset (default is ‘Type’).
multivalue (bool, optional) – If True, aggregate duplicate (ID, KEY) pairs into lists (default is False).

Returns:

Pivoted DataFrame with IDs as index and keys as columns, or None if no data is found.

Return type:

pandas.DataFrame or None

Examples

>>> table = data.type_tableview("ACLineSegment", multivalue=True)
>>> headers = data.type_tableview(("FullModel", "Dataset"), string_to_number=False)

triplets.tools.types_dict(data, contains=None, case_insensitive=True, *, engine='auto')¶

Return a dictionary of object types and their occurrence counts.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
contains (str, optional) – If given, keep only types whose name contains this substring.
case_insensitive (bool, default True) – If True, the contains match ignores letter case.

Returns:

Dictionary with object types as keys and their counts as values.

Return type:

dict

Examples

>>> types = data.types_dict()
>>> print(types)
{'ACLineSegment': 10, 'PowerTransformer': 5, ...}
>>> data.types_dict(contains="Settlement")
{'MarketEvaluationPoint': 3, ...}

triplets.tools.update_triplet_from_tableview(data, tableview, update=True, add=True, instance_id=None, *, engine='auto')¶

Update or add triplets from a table view.

Parameters:

data (pandas.DataFrame) – Original triplet dataset to update.
tableview (pandas.DataFrame) – Table view containing updates or new data.
update (bool, optional) – If True, update existing ID-KEY pairs (default is True).
add (bool, optional) – If True, add new ID-KEY pairs (default is True).
instance_id (str, optional) – Instance ID to assign to new triplets (default is None).

Returns:

Updated triplet dataset.

Return type:

pandas.DataFrame

Examples

>>> updated_data = data.update_triplets_from_tableview(table_view, instance_id="uuid1")

triplets.tools.update_triplet_from_triplet(data, update_data, update=True, add=True, *, engine='auto')¶

Update or add triplets from another triplet dataset.

Parameters:

data (pandas.DataFrame) – Original triplet dataset to update.
update_data (pandas.DataFrame) – Triplet dataset containing updates or new data.
update (bool, optional) – If True, update existing ID-KEY pairs (default is True).
add (bool, optional) – If True, add new ID-KEY pairs (default is True).

Returns:

Updated triplet dataset.

Return type:

pandas.DataFrame

Notes

TODO: Add a changes DataFrame to track modifications.
TODO: Support updating ID and KEY fields.

Examples

>>> updated_data = data.update_triplets_from_triplets(update_data)

triplets.tools.update_triplets_from_tableview(data, tableview, update=True, add=True, instance_id=None, *, engine='auto')¶

Update or add triplets from a table view.

Parameters:

data (pandas.DataFrame) – Original triplet dataset to update.
tableview (pandas.DataFrame) – Table view containing updates or new data.
update (bool, optional) – If True, update existing ID-KEY pairs (default is True).
add (bool, optional) – If True, add new ID-KEY pairs (default is True).
instance_id (str, optional) – Instance ID to assign to new triplets (default is None).

Returns:

Updated triplet dataset.

Return type:

pandas.DataFrame

Examples

>>> updated_data = data.update_triplets_from_tableview(table_view, instance_id="uuid1")

triplets.tools.update_triplets_from_triplets(data, update_data, update=True, add=True, *, engine='auto')¶

Update or add triplets from another triplet dataset.

Parameters:

data (pandas.DataFrame) – Original triplet dataset to update.
update_data (pandas.DataFrame) – Triplet dataset containing updates or new data.
update (bool, optional) – If True, update existing ID-KEY pairs (default is True).
add (bool, optional) – If True, add new ID-KEY pairs (default is True).

Returns:

Updated triplet dataset.

Return type:

pandas.DataFrame

Notes

TODO: Add a changes DataFrame to track modifications.
TODO: Support updating ID and KEY fields.

Examples

>>> updated_data = data.update_triplets_from_triplets(update_data)

triplets.tools.pandas_engine¶

triplets.tools.pandas_engine.get_namespace_map(data: DataFrame)[source]¶

Extract namespace prefix-to-URI mapping and optional xml:base from a triplet dataset.

Special keys: - xml_base: Extracted separately if present (used as base URI in RDF). - Type: Automatically excluded.

Parameters:

data (pandas.DataFrame) – Triplet dataset with columns [‘INSTANCE_ID’, ‘ID’, ‘KEY’, ‘VALUE’]. Must contain a NamespaceMap instance for successful extraction.

Returns:

namespace_map (dict) – Mapping of namespace prefixes to URIs (e.g., {"cim": "...", "rdf": "..."}). Empty dict if no NamespaceMap is found.
xml_base (str) – Value of xml_base if defined within the NamespaceMap; otherwise empty str.

Examples

>>> ns_map, base = get_namespace_map(triplet_data)
>>> print(ns_map)
{'cim': 'http://iec.ch/TC57/2013/CIM-schema-cim16#', 'rdf': 'http://www.w3.org/1999/02/22-rdf-syntax-ns#'}
>>> print(base)
'http://example.com/base/'

>>> ns_map, base = get_namespace_map(empty_data)
>>> print(ns_map, base)
{} ""

Notes

The function is idempotent and safe to call on any dataset.
Uses inner merge on ID to scope entries to the correct NamespaceMap instance.
Always returns a tuple of length 2: (dict, str).

triplets.tools.pandas_engine.type_tableview(data, type_name, string_to_number=True, type_key='Type', multivalue=False)[source]¶

Create a table view of all objects of a specified type.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
type_name (str or sequence of str) – Type(s) to select (e.g. "ACLineSegment" or ("FullModel", "Dataset")). A plain string is treated as a one-item list.
string_to_number (bool, optional) – If True, convert columns containing numbers to numeric types (default is True).
type_key (str, optional) – Key used to identify object types in the dataset (default is ‘Type’).
multivalue (bool, optional) – If True, aggregate duplicate (ID, KEY) pairs into lists (default is False).

Returns:

Pivoted DataFrame with IDs as index and keys as columns, or None if no data is found.

Return type:

pandas.DataFrame or None

Examples

>>> table = data.type_tableview("ACLineSegment", multivalue=True)
>>> headers = data.type_tableview(("FullModel", "Dataset"), string_to_number=False)

triplets.tools.pandas_engine.key_tableview(data, key, string_to_number=True, multivalue=False)[source]¶

Create a table view of all objects with a specified key.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
key (str) – The key to filter objects by (e.g., ‘GeneratingUnit.maxOperatingP’).
string_to_number (bool, optional) – If True, convert columns containing numbers to numeric types (default is True).
multivalue (bool, optional) – If True, aggregate duplicate (ID, KEY) pairs into lists (default is False).

Returns:

Pivoted DataFrame with IDs as index and keys as columns, or None if no data is found.

Return type:

pandas.DataFrame or None

Examples

>>> table = data.key_tableview("GeneratingUnit.maxOperatingP")

triplets.tools.pandas_engine.id_tableview(data, id, string_to_number=True, multivalue=False)[source]¶

Create a tabular view of a CGMES triplet dataset filtered by ID-s.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing CGMES data.
id (str or list or pandas.DataFrame) – ID(s) to filter by (single ID, list of IDs, or DataFrame with an ID column).
string_to_number (bool, optional) – If True, convert columns containing numbers to numeric types (default is True).
multivalue (bool, optional) – If True, aggregate duplicate (ID, KEY) pairs into lists (default is False).

Returns:

Pivoted DataFrame with IDs as index and KEYs as columns.

Return type:

pandas.DataFrame or None

Examples

>>> table = id_tableview(data, 'UUID')
>>> table = id_tableview(data, ['UUID_1', 'UUID_2'])
>>> table = id_tableview(data, pandas.DataFrame({"ID": ['UUID_1', 'UUID_2']}))

triplets.tools.pandas_engine.references_to_simple(data, reference, columns=['Type'])[source]¶

Create a simplified table view of objects referencing a specified object.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
reference (str) – ID of the object to find references to.
columns (list, optional) – Columns to include in the output table (default is [‘Type’]).

Returns:

Pivoted DataFrame with IDs of referencing objects and specified columns.

Return type:

pandas.DataFrame

Examples

>>> table = data.references_to_simple("99722373_VL_TN1")

triplets.tools.pandas_engine.references_to(data, reference, levels=1)[source]¶

Retrieve all objects pointing to a specified reference object.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
reference (str) – ID of the reference object.
levels (int, optional) – Number of reference levels to traverse (default is 1).

Returns:

DataFrame containing triplets of objects pointing to the reference, with a ‘level’ column.

Return type:

pandas.DataFrame

Notes

TODO: Add the key on which the connection was made.

Examples

>>> refs = data.references_to("99722373_VL_TN1", levels=2)

triplets.tools.pandas_engine.references_from_simple(data, reference, columns=['Type'])[source]¶

Create a simplified table view of objects a specified object refers to.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
reference (str) – ID of the object to find references from.
columns (list, optional) – Columns to include in the output table (default is [‘Type’]).

Returns:

Pivoted DataFrame with IDs of referenced objects and specified columns.

Return type:

pandas.DataFrame

Examples

>>> table = data.references_from_simple("99722373_VL_TN1")

triplets.tools.pandas_engine.references_from(data, reference, levels=1)[source]¶

Retrieve all objects a specified object points to.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
reference (str) – ID of the reference object.
levels (int, optional) – Number of reference levels to traverse (default is 1).

Returns:

DataFrame containing triplets of objects referenced by the input, with a ‘level’ column.

Return type:

pandas.DataFrame

Notes

TODO: Add the key on which the connection was made.

Examples

>>> refs = data.references_from("99722373_VL_TN1", levels=2)

triplets.tools.pandas_engine.references_all(data)[source]¶

Find all unique references (links) in the dataset.

Parameters:: data (pandas.DataFrame) – Triplet dataset containing RDF data.
Returns:: DataFrame with columns [‘ID_FROM’, ‘KEY’, ‘ID_TO’] representing all references.
Return type:: pandas.DataFrame

Notes

Does not consider INSTANCE_ID in reference matching.

Examples

>>> refs = data.references_all()

triplets.tools.pandas_engine.references_simple(data, reference, columns=None, levels=1)[source]¶

Create a simplified table view of all references to and from a specified object.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
reference (str) – ID of the object to find references for.
columns (list, optional) – Columns to include in the output table (default is [‘Type’, ‘IdentifiedObject.name’] if available).
levels (int, optional) – Number of reference levels to traverse (default is 1).

Returns:

Pivoted DataFrame with IDs, specified columns, and reference levels.

Return type:

pandas.DataFrame

Examples

>>> table = data.references_simple("99722373_VL_TN1", columns=["Type"])

triplets.tools.pandas_engine.references(data, ID, levels=1)[source]¶

Retrieve all references (to and from) a specified object.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
ID (str) – ID of the object to find references for.
levels (int, optional) – Number of reference levels to traverse (default is 1).

Returns:

DataFrame containing triplets of all references to and from the object.

Return type:

pandas.DataFrame

Examples

>>> refs = data.references("99722373_VL_TN1", levels=2)

triplets.tools.pandas_engine.types_dict(data, contains=None, case_insensitive=True)[source]¶

Return a dictionary of object types and their occurrence counts.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
contains (str, optional) – If given, keep only types whose name contains this substring.
case_insensitive (bool, default True) – If True, the contains match ignores letter case.

Returns:

Dictionary with object types as keys and their counts as values.

Return type:

dict

Examples

>>> types = data.types_dict()
>>> print(types)
{'ACLineSegment': 10, 'PowerTransformer': 5, ...}
>>> data.types_dict(contains="Settlement")
{'MarketEvaluationPoint': 3, ...}

triplets.tools.pandas_engine.set_value_at_key(data, key, value)[source]¶

Set the value for all instances of a specified key.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
key (str) – The key to update.
value (str) – The new value to set for the specified key.

Notes

TODO: Add debug logging for key, initial value, and new value.
TODO: Store changes in a changes DataFrame.

Examples

>>> data.set_value_at_key("label", "new_label")

triplets.tools.pandas_engine.set_value_at_key_and_id(data, key, value, id)[source]¶

Set the value for a specific key and ID.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
key (str) – The key to update.
value (str) – The new value to set.
id (str) – The ID of the object to update.

Examples

>>> data.set_value_at_key_and_id("label", "new_label", "uuid1")

triplets.tools.pandas_engine.triplets_to_tableviews(triplet_df, multivalue=False)[source]¶

Convert triplet DataFrame to dict of tableview DataFrames.

Parameters:

triplet_df (pandas.DataFrame) – Triplet dataset with columns [ID, KEY, VALUE, INSTANCE_ID].
multivalue (bool, default False) – If True, aggregate duplicate (ID, KEY) pairs into lists.

Returns:

{class_name: tableview_df}

Return type:

dict

triplets.tools.pandas_engine.get_object_data(data, object_UUID)[source]¶

Retrieve data for a specific object by its UUID.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
object_UUID (str) – UUID of the object to retrieve.

Returns:

Series with keys as index and values for the specified object.

Return type:

pandas.Series

Examples

>>> obj_data = data.get_object_data("uuid1")

triplets.tools.pandas_engine.tableview_to_triplets(data, multivalue=False, instance_id=None)[source]¶

Convert a table view back to a triplet format.

Parameters:

data (pandas.DataFrame) – Pivoted DataFrame (table view) to convert.
multivalue (bool, optional) – If True, unpack list values into separate triplets (default is False).
instance_id (str, optional) – If given, stamp an INSTANCE_ID column on the result (default None).

Returns:

Triplet DataFrame with columns [‘ID’, ‘KEY’, ‘VALUE’] (plus ‘INSTANCE_ID’ when instance_id is given).

Return type:

pandas.DataFrame

Notes

triplets.tools.pandas_engine.update_triplets_from_triplets(data, update_data, update=True, add=True)[source]¶

Update or add triplets from another triplet dataset.

Parameters:

data (pandas.DataFrame) – Original triplet dataset to update.
update_data (pandas.DataFrame) – Triplet dataset containing updates or new data.
update (bool, optional) – If True, update existing ID-KEY pairs (default is True).
add (bool, optional) – If True, add new ID-KEY pairs (default is True).

Returns:

Updated triplet dataset.

Return type:

pandas.DataFrame

Notes

TODO: Add a changes DataFrame to track modifications.
TODO: Support updating ID and KEY fields.

Examples

>>> updated_data = data.update_triplets_from_triplets(update_data)

triplets.tools.pandas_engine.update_triplets_from_tableview(data, tableview, update=True, add=True, instance_id=None)[source]¶

Update or add triplets from a table view.

Parameters:

data (pandas.DataFrame) – Original triplet dataset to update.
tableview (pandas.DataFrame) – Table view containing updates or new data.
update (bool, optional) – If True, update existing ID-KEY pairs (default is True).
add (bool, optional) – If True, add new ID-KEY pairs (default is True).
instance_id (str, optional) – Instance ID to assign to new triplets (default is None).

Returns:

Updated triplet dataset.

Return type:

pandas.DataFrame

Examples

>>> updated_data = data.update_triplets_from_tableview(table_view, instance_id="uuid1")

triplets.tools.pandas_engine.remove_triplets_from_triplets(from_triplet, what_triplet, columns=['ID', 'KEY', 'VALUE'])[source]¶

Remove triplets from one dataset that match another.

Parameters:

from_triplet (pandas.DataFrame) – Original triplet dataset.
what_triplet (pandas.DataFrame) – Triplet dataset to remove from the original.
columns (list, optional) – Columns to match for removal (default is [‘ID’, ‘KEY’, ‘VALUE’]).

Returns:

Dataset with matching triplets removed.

Return type:

pandas.DataFrame

Examples

>>> result = remove_triplets_from_triplets(data, to_remove)

triplets.tools.pandas_engine.filter_triplets_by_triplets(data, filter_triplet)[source]¶

Filter riplet DataFrame using IDs from another DataFrame.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing CGMES data.
filter_triplet (pandas.DataFrame) – DataFrame containing atleast colum ID to filter by.

Returns:

Filtered DataFrame with columns [‘ID, ‘KEY’, ‘VALUE’, ‘INSTANCE_ID’].

Return type:

pandas.DataFrame

Examples

>>> filtered = filter_triplets_by_triplets(data, filter_triplet)

triplets.tools.pandas_engine.filter_triplets_by_type(data, type_name, type_key='Type')[source]¶

Filter triplet dataset by objects of a specific type.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
type_name (str) – Object type to filter by (e.g., ‘ACLineSegment’).
type_key (str) – Key used in triplet to indicate type, by default “Type”

Returns:

Filtered triplet dataset containing only objects of the specified type.

Return type:

pandas.DataFrame

Examples

>>> filtered = filter_triplets_by_type(data, "ACLineSegment")

triplets.tools.pandas_engine.filter_triplets(data, ID=None, KEY=None, VALUE=None, INSTANCE_ID=None, regex=False)[source]¶

Filter triplets by any combination of columns with optional regex.

Parameters:

data (pandas.DataFrame) – Triplet dataset with columns [ID, KEY, VALUE, INSTANCE_ID].
ID (str or list of str, optional) – Filter value. A list keeps rows matching any of its values. With regex=True the value(s) are regex patterns (re.search).
KEY (str or list of str, optional) – Filter value. A list keeps rows matching any of its values. With regex=True the value(s) are regex patterns (re.search).
VALUE (str or list of str, optional) – Filter value. A list keeps rows matching any of its values. With regex=True the value(s) are regex patterns (re.search).
INSTANCE_ID (str or list of str, optional) – Filter value. A list keeps rows matching any of its values. With regex=True the value(s) are regex patterns (re.search).
regex (bool, default False) – If True, use regex matching (re.search). If False, exact match.

Returns:

Filtered triplet dataset.

Return type:

pandas.DataFrame

Examples

>>> filter_triplets(data, KEY="Type", VALUE="ACLineSegment")
>>> filter_triplets(data, KEY=["Type", "IdentifiedObject.name"])
>>> filter_triplets(data, VALUE=".*Substation.*", regex=True)

triplets.tools.pandas_engine.filter_triplets_by_value(data, VALUE, detailed=False, type_key='Type', regex=False)[source]¶

Filter to objects that have any triplet matching VALUE.

Selects every object (ID) with at least one triplet whose VALUE matches VALUE, then summarizes those objects.

Parameters:

data (pandas.DataFrame) – Triplet dataset with columns [ID, KEY, VALUE, INSTANCE_ID].
VALUE (str) – Value to match. Exact match by default; regex pattern if regex=True.
detailed (bool, default False) – If True, return all triplets of the matched objects. If False, return only the matching rows plus each matched object’s type_key row, sorted by ID with the type row first within each ID group.
type_key (str) – Key identifying each object’s type row, by default “Type”.
regex (bool, default False) – If True, use regex matching (re.search). If False, exact match.

Returns:

Filtered triplet dataset.

Return type:

pandas.DataFrame

Examples

>>> filter_triplets_by_value(data, "DOWGEN_LD1", regex=True)

triplets.tools.pandas_engine.diff_triplets(old_data, new_data)[source]¶

Compute the difference between two Triplet DataFrames.

Parameters:

old_data (pandas.DataFrame) – Original triplet dataset.
new_data (pandas.DataFrame) – New triplet dataset to compare against.

Returns:

DataFrame containing triplets unique to old_data or new_data, with an ‘_merge’ column indicating ‘left_only’ (in old_data) or ‘right_only’ (in new_data).

Return type:

pandas.DataFrame

Examples

>>> diff = diff_triplets(old_data, new_data)

triplets.tools.pandas_engine.diff_triplets_by_instance(data, INSTANCE_ID_1, INSTANCE_ID_2)[source]¶

Identify differences between two loaded INSTANCES, by thier INSTACE_ID in the same Triplet DataFrame.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing two or more INSTANCE.
INSTANCE_ID_1 (str) – UUID of the first INSTANCE.
INSTANCE_ID_2 (str) – UUID of the second INSTANCE.

Returns:

DataFrame containing triplets that differ between the two model parts.

Return type:

pandas.DataFrame

Examples

>>> diff = diff_triplets_by_instance('uuid1', 'uuid2')

triplets.tools.pandas_engine.print_triplets_diff(old_data, new_data, file_id_object='Distribution', file_id_key='label', exclude_objects=None)[source]¶

Print a human-readable diff of two triplet datasets.

Parameters:

old_data (pandas.DataFrame) – Original triplet dataset.
new_data (pandas.DataFrame) – New triplet dataset to compare against.
file_id_object (str, optional) – Object type containing file identifiers (default is ‘Distribution’).
file_id_key (str, optional) – Key containing file identifiers (default is ‘label’).
exclude_objects (list, optional) – List of object types to exclude from the diff (default is None).

Notes

Outputs a diff format showing removed, added, and changed objects.
Nice diff viewer https://diffy.org/
TODO: Add name field for better reporting with Type.

Examples

>>> print_triplets_diff(old_data, new_data, exclude_objects=["NamespaceMap"])

triplets.tools.pandas_engine.content_hash(data, ignore_types=('Distribution', 'NamespaceMap', 'FullModel'), columns=('ID', 'KEY', 'VALUE'), order_sensitive=False)[source]¶

Deterministic identity hash of the triplet content (blake2b hex).

triplets.tools.polars_engine¶

Polars-native implementation of triplet data manipulation tools.

Uses polars lazy evaluation, hash joins, and native operations for performance. All functions accept and return polars DataFrames.

triplets.tools.polars_engine.type_tableview(data, type_name, string_to_number=True, type_key='Type', multivalue=False)[source]¶

Create a table view of all objects of a specified type using polars pivot.

type_name is a string or a sequence of strings; a plain string is treated as a one-item list.

triplets.tools.polars_engine.key_tableview(data, key, string_to_number=True, multivalue=False)[source]¶: Create a table view of all objects with a specified key.

triplets.tools.polars_engine.id_tableview(data, id, string_to_number=True, multivalue=False)[source]¶: Create a table view of objects by ID (single ID, list of IDs, or DataFrame with ID column).

triplets.tools.polars_engine.types_dict(data, contains=None, case_insensitive=True)[source]¶

Return dict of {type_name: count}.

With contains, keep only types whose name contains that substring (case-insensitive unless case_insensitive=False).

triplets.tools.polars_engine.get_object_data(data, object_UUID)[source]¶: Get all data for a specific object.

triplets.tools.polars_engine.get_namespace_map(data)[source]¶

Extract namespace map from triplet data.

Returns (namespace_map dict, xml_base) — same contract as the pandas engine.

triplets.tools.polars_engine.references_to(data, reference, levels=1)[source]¶: Objects that reference reference, traversing up to levels. Matches the pandas engine: level 0 is the object itself; level N rows carry level/ID_TO/ID_FROM.

triplets.tools.polars_engine.references_from(data, reference, levels=1)[source]¶: Objects referenced BY reference, traversing up to levels. Matches pandas: level 0 is the object itself; level N rows carry level/ID_TO/ID_FROM.

triplets.tools.polars_engine.references(data, ID, levels=1)[source]¶: All references to and from an object (both directions), matching pandas.

triplets.tools.polars_engine.references_all(data)[source]¶: All reference links as (ID_FROM, KEY, ID_TO), matching pandas.

triplets.tools.polars_engine.references_to_simple(data, reference, columns=['Type'])[source]¶: Pivot of objects referencing reference (index ID_FROM), limited to columns.

triplets.tools.polars_engine.references_from_simple(data, reference, columns=['Type'])[source]¶: Pivot of objects referenced BY reference (index ID_TO), limited to columns.

triplets.tools.polars_engine.references_simple(data, reference, columns=None, levels=1)[source]¶: Pivot of the object and everything linked to/from it (index ID), with the level/ID_FROM/ID_TO metadata merged back, matching pandas.

triplets.tools.polars_engine.filter_triplets_by_type(data, type_name, type_key='Type')[source]¶: Filter triplet data to only include objects of a specific type.

triplets.tools.polars_engine.filter_triplets_by_triplets(data, filter_triplet)[source]¶: Return all triplets whose ID appears in filter_triplet (matches pandas: merge on ID).

triplets.tools.polars_engine.filter_triplets(data, ID=None, KEY=None, VALUE=None, INSTANCE_ID=None, regex=False)[source]¶

Filter triplets by any combination of columns with optional regex.

Parameters:

data (polars.DataFrame) – Triplet dataset with columns [ID, KEY, VALUE, INSTANCE_ID].
ID (str or list of str, optional) – Filter value. A list keeps rows matching any of its values. With regex=True the value(s) are regex patterns (str.contains).
KEY (str or list of str, optional) – Filter value. A list keeps rows matching any of its values. With regex=True the value(s) are regex patterns (str.contains).
VALUE (str or list of str, optional) – Filter value. A list keeps rows matching any of its values. With regex=True the value(s) are regex patterns (str.contains).
INSTANCE_ID (str or list of str, optional) – Filter value. A list keeps rows matching any of its values. With regex=True the value(s) are regex patterns (str.contains).
regex (bool, default False) – If True, use regex matching (str.contains). If False, exact match.

Returns:

Filtered triplet dataset.

Return type:

polars.DataFrame

triplets.tools.polars_engine.filter_triplets_by_value(data, VALUE, detailed=False, type_key='Type', regex=False)[source]¶

Filter to objects that have any triplet matching VALUE.

Selects every object (ID) with at least one triplet whose VALUE matches VALUE, then summarizes those objects.

Parameters:

data (polars.DataFrame) – Triplet dataset with columns [ID, KEY, VALUE, INSTANCE_ID].
VALUE (str) – Value to match. Exact match by default; regex pattern if regex=True.
detailed (bool, default False) – If True, return all triplets of the matched objects. If False, return only the matching rows plus each matched object’s type_key row, sorted by ID with the type row first within each ID group.
type_key (str, default "Type") – KEY identifying each object’s type row.
regex (bool, default False) – If True, match VALUE as a regex (str.contains). If False, exact match.

Returns:

Filtered triplet dataset.

Return type:

polars.DataFrame

Examples

>>> filter_triplets_by_value(data, "DOWGEN_LD1", regex=True)

triplets.tools.polars_engine.set_value_at_key(data, key, value)[source]¶: Set VALUE for all rows with a given KEY (in-place mutation via reassignment).

triplets.tools.polars_engine.set_value_at_key_and_id(data, key, value, id)[source]¶: Set VALUE for a specific KEY and ID combination.

triplets.tools.polars_engine.triplets_to_tableviews(triplet_df, multivalue=False)[source]¶: Convert triplet DataFrame to dict of tableview DataFrames.

triplets.tools.polars_engine.tableview_to_triplets(data, multivalue=False, instance_id=None)[source]¶

Convert a table view back to triplet format.

An empty tableview cell is not a triplet — those holes are dropped (matches the pandas and duckdb engines). Pass instance_id to stamp an INSTANCE_ID column, the same way update_triplets_from_tableview does.

triplets.tools.polars_engine.update_triplets_from_triplets(data, update_data, update=True, add=True)[source]¶

Update existing and/or add new rows from another triplet dataset.

Merge keys are ID+KEY (plus INSTANCE_ID when update_data carries it), matching the pandas engine. Join keys are cast to Utf8 so a Categorical KEY in data joins cleanly with a string KEY in update_data.

triplets.tools.polars_engine.update_triplets_from_tableview(data, tableview, update=True, add=True, instance_id=None)[source]¶: Update triplet data from a tableview DataFrame.

triplets.tools.polars_engine.remove_triplets_from_triplets(from_triplet, what_triplet, columns=['ID', 'KEY', 'VALUE'])[source]¶: Remove rows from one triplet that match another (anti-join).

triplets.tools.polars_engine.diff_triplets(old_data, new_data)[source]¶: Rows unique to old (left_only) or new (right_only), matching the pandas outer-merge shape: columns [ID, KEY, VALUE, INSTANCE_ID_OLD, INSTANCE_ID_NEW, _merge].

triplets.tools.polars_engine.diff_triplets_by_instance(data, INSTANCE_ID_1, INSTANCE_ID_2)[source]¶: Triplets that differ between two instances in the dataset (symmetric difference on ID/KEY/VALUE), matching pandas drop_duplicates(keep=False).

triplets.tools.polars_engine.print_triplets_diff(old_data, new_data, file_id_object='Distribution', file_id_key='label', exclude_objects=None)[source]¶: Print a human-readable diff between two triplet datasets.

triplets.tools.polars_engine.content_hash(data, ignore_types=('Distribution', 'NamespaceMap', 'FullModel'), columns=('ID', 'KEY', 'VALUE'), order_sensitive=False)[source]¶: Deterministic identity hash of the triplet content (blake2b hex) — row-order-invariant by default via native row hashes (hash_rows) combined with commutative count/sum/xor — no sort, single pass. With order_sensitive=True the row hashes are digested in row order, so the same rows in a different order produce a different digest. Engine-specific: not comparable across engines (see pandas_engine).

triplets.tools.duckdb_engine¶

DuckDB SQL-based implementation of triplet tools.

Functions are monkey-patched onto duckdb.DuckDBPyConnection. The *_tableview helpers create a named SQL view and return a relation over it (named/re-queryable, still lazy); other query/reference helpers return a DuckDBPyRelation; types_dict and get_namespace_map return python values; triplets_to_tableviews returns a dict.

The connection holds the dataset in one table (default triplets; per- connection table/schema from duckdb.connect / set_triplets_table). Mutating helpers run in-place DML (UPDATE/DELETE/INSERT — no full-table rewrites, extra user columns survive) and return the connection for chaining.

Per-connection defaults live in a WeakKeyDictionary (DuckDBPyConnection has no __dict__) and, once explicitly configured, in a tiny main."_triplets_config" key/value table inside the database itself — so a persisted file remembers its table/schema across reopen, and cursors/duplicates resolve the same config. A bare duckdb.connect() never writes anything (keeps :memory: clean); read-only connections update the in-process config only. Resolution order: call kwargs → in-process config → DB-stored config → package defaults; SQL always uses double-quoted identifiers. ATTACHed extra catalogs are out of scope — the config lives in the default catalog.

triplets.tools.duckdb_engine.types_dict(self, contains=None, case_insensitive=True, table=None, schema=None, table_name=None)[source]¶

Return dict of {type_name: count}.

With contains, keep only types whose name contains that substring (case-insensitive unless case_insensitive=False).

triplets.tools.duckdb_engine.type_tableview(self, type_name, table=None, schema=None, table_name=None, view_name=None, string_to_number=False, multivalue=False)[source]¶: Create a named SQL view pivoting all objects of a type; return a relation over it. type_name is a string or a sequence of strings (a plain string is a one-item list). The view defaults to the type name, or FullModel/Dataset style join for several types (override with view_name). multivalue=True renders multi-valued keys as the literal [‘a’, ‘b’] text (pandas/polars encoding); string_to_number is accepted for signature parity but not implemented.

triplets.tools.duckdb_engine.filter_triplets(self, ID=None, KEY=None, VALUE=None, INSTANCE_ID=None, regex=False, table=None, schema=None, table_name=None)[source]¶

Filter triplets by any combination of columns. Returns DuckDBPyRelation (lazy).

A list value keeps rows matching any of its values; with regex=True the value(s) are regex patterns matched anywhere in the value, like pandas str.contains.

triplets.tools.duckdb_engine.filter_triplets_by_type(self, type_name, table=None, schema=None, table_name=None)[source]¶: Filter to only objects of a specific type. Returns DuckDBPyRelation (lazy).

triplets.tools.duckdb_engine.filter_triplets_by_value(self, VALUE, detailed=False, type_key='Type', regex=False, table=None, schema=None, table_name=None)[source]¶

Filter to objects that have any triplet matching VALUE. Returns DuckDBPyRelation.

Selects every object (ID) with at least one triplet whose VALUE matches VALUE (with regex=True, a regex pattern matched anywhere in the value, like pandas str.contains). With detailed=True, returns all their triplets; otherwise the matching rows plus each matched object’s type_key row, type row first within each ID.

triplets.tools.duckdb_engine.references_to(self, reference, levels=1, table=None, schema=None, table_name=None)[source]¶: Objects that reference the given ID, multi-level. Returns DuckDBPyRelation.

triplets.tools.duckdb_engine.references_from(self, reference, levels=1, table=None, schema=None, table_name=None)[source]¶: Objects referenced BY the given ID, multi-level. Returns DuckDBPyRelation.

triplets.tools.duckdb_engine.key_tableview(self, key, table=None, schema=None, table_name=None, view_name=None, string_to_number=False, multivalue=False)[source]¶: Create a named SQL view pivoting objects carrying a given KEY; return a relation over it. The view defaults to the key name (override with view_name).

triplets.tools.duckdb_engine.id_tableview(self, id, table=None, schema=None, table_name=None, view_name=None, string_to_number=False, multivalue=False)[source]¶: Create a named SQL view pivoting the given ID(s) — a single id or an iterable — and return a relation over it. The view defaults to the id when a single one is given, else ‘id_tableview’ (override with view_name).

triplets.tools.duckdb_engine.get_object_data(self, object_UUID, table=None, schema=None, table_name=None)[source]¶: All (KEY, VALUE) rows for one object. Returns DuckDBPyRelation (lazy).

triplets.tools.duckdb_engine.get_namespace_map(self, table=None, schema=None, table_name=None)[source]¶: Return (namespace_map dict, xml_base) from the NamespaceMap object.

triplets.tools.duckdb_engine.triplets_to_tableviews(self, table=None, schema=None, table_name=None, string_to_number=False, multivalue=False)[source]¶: Return {type_name: tableview relation} for every type in the dataset.

triplets.tools.duckdb_engine.references(self, ID, levels=1, table=None, schema=None, table_name=None)[source]¶: All references to and from an object (both directions). Returns DuckDBPyRelation.

triplets.tools.duckdb_engine.references_to_simple(self, reference, columns=['Type'], table=None, schema=None, table_name=None)[source]¶: Pivot of objects referencing reference (index ID_FROM), limited to columns.

triplets.tools.duckdb_engine.references_from_simple(self, reference, columns=['Type'], table=None, schema=None, table_name=None)[source]¶: Pivot of objects referenced BY reference (index ID_TO), limited to columns.

triplets.tools.duckdb_engine.references_simple(self, reference, columns=None, levels=1, table=None, schema=None, table_name=None)[source]¶: Pivot of the object and everything linked to/from it (index ID), with the level/ID_FROM/ID_TO metadata merged back, matching pandas.

triplets.tools.duckdb_engine.references_all(self, table=None, schema=None, table_name=None)[source]¶: All reference links as (ID_FROM, KEY, ID_TO). Returns DuckDBPyRelation.

triplets.tools.duckdb_engine.filter_triplets_by_triplets(self, filter_triplet, table=None, schema=None, table_name=None)[source]¶: Keep triplets whose ID appears in filter_triplet. Returns DuckDBPyRelation.

triplets.tools.duckdb_engine.set_value_at_key(self, key, value, table=None, schema=None, table_name=None)[source]¶: Set VALUE for every row with the given KEY. Mutates the table; returns self.

triplets.tools.duckdb_engine.set_value_at_key_and_id(self, key, value, id, table=None, schema=None, table_name=None)[source]¶: Set VALUE for the row with the given KEY and ID. Mutates the table; returns self.

triplets.tools.duckdb_engine.update_triplets_from_triplets(self, update_data, update=True, add=True, table=None, schema=None, table_name=None)[source]¶: Update existing and/or add new rows from another triplet dataset. Merges on ID+KEY (plus INSTANCE_ID when update_data has it). Mutates the table; returns self.

triplets.tools.duckdb_engine.update_triplets_from_tableview(self, tableview, update=True, add=True, instance_id=None, table=None, schema=None, table_name=None)[source]¶: Unpivot a tableview to triplets, then update/add them. When instance_id is None the merge is on ID+KEY only (mirrors the pandas engine). Mutates; returns self.

triplets.tools.duckdb_engine.remove_triplets_from_triplets(self, what_triplet, columns=['ID', 'KEY', 'VALUE'], table=None, schema=None, table_name=None)[source]¶: Remove rows matching what_triplet on columns. Mutates the table; returns self.

triplets.tools.duckdb_engine.diff_triplets(self, new_data, table=None, schema=None, table_name=None)[source]¶: Rows unique to the table (left_only) or to new_data (right_only), with a _merge column. Returns DuckDBPyRelation.

triplets.tools.duckdb_engine.diff_triplets_by_instance(self, INSTANCE_ID_1, INSTANCE_ID_2, table=None, schema=None, table_name=None)[source]¶: Triplets that differ between two instances in the table. Returns DuckDBPyRelation.

triplets.tools.duckdb_engine.print_triplets_diff(self, new_data, file_id_object='Distribution', file_id_key='label', exclude_objects=None, table=None, schema=None, table_name=None)[source]¶: Print a simple removed/added diff of the table against new_data.

triplets.tools.duckdb_engine.tableview_to_triplets(self, table=None, schema=None, table_name=None, multivalue=False, instance_id=None)[source]¶

Unpivot a wide tableview table back to triplets (ID, KEY, VALUE).

Point table_name at a tableview table (the default triplets table is already long-form). Empty cells (NULL VALUE) are dropped — not real triplets. Pass instance_id to stamp an INSTANCE_ID column, the same way update_triplets_from_tableview does. Returns DuckDBPyRelation.

triplets.tools.duckdb_engine.content_hash(self, ignore_types=('Distribution', 'NamespaceMap', 'FullModel'), columns=('ID', 'KEY', 'VALUE'), order_sensitive=False, table=None, schema=None, table_name=None)[source]¶: Deterministic identity hash of the triplet content (blake2b hex) — row-order-invariant by default via in-database hash() combined with streaming count/sum/xor aggregates (no sort, no materialization — works larger-than-memory). With order_sensitive=True each row’s position (insertion order via rowid) is mixed into its hash, so the same rows inserted in a different order produce a different digest. Engine-specific: not comparable across engines (see pandas_engine).