triplets.tools¶

Triplet data manipulation tools with pandas/polars engine support.

Provides query, filter, diff, transform, and mutate operations on triplet DataFrames ([ID, KEY, VALUE, INSTANCE_ID]).

Engines: - pandas_engine (default, always available) - polars_engine (optional, uses polars-native operations for speed)

The engine is auto-detected from the input DataFrame type, or can be specified explicitly with engine=”pandas” or engine=”polars”.

triplets.tools.type_tableview(data, type_name, string_to_number=True, type_key='Type', multivalue=False, engine='auto')[source]¶

triplets.tools.key_tableview(data, key, string_to_number=True, multivalue=False, engine='auto')[source]¶

triplets.tools.id_tableview(data, id, string_to_number=True, multivalue=False, engine='auto')[source]¶

triplets.tools.types_dict(data, engine='auto')[source]¶

triplets.tools.get_object_data(data, object_UUID, engine='auto')[source]¶

triplets.tools.get_namespace_map(data, engine='auto')[source]¶

triplets.tools.references_to_simple(data, reference, columns=['Type'], engine='auto')[source]¶

triplets.tools.references_to(data, reference, levels=1, engine='auto')[source]¶

triplets.tools.references_from_simple(data, reference, columns=['Type'], engine='auto')[source]¶

triplets.tools.references_from(data, reference, levels=1, engine='auto')[source]¶

triplets.tools.references_all(data, engine='auto')[source]¶

triplets.tools.references_simple(data, reference, columns=None, levels=1, engine='auto')[source]¶

triplets.tools.references(data, ID, levels=1, engine='auto')[source]¶

triplets.tools.filter_triplets_by_type(data, type_name, type_key='Type', engine='auto')[source]¶

triplets.tools.filter_triplets_by_triplets(data, filter_triplet, engine='auto')[source]¶

triplets.tools.filter_triplets(data, ID=None, KEY=None, VALUE=None, INSTANCE_ID=None, regex=False, engine='auto')[source]¶

triplets.tools.set_value_at_key(data, key, value, engine='auto')[source]¶

triplets.tools.set_value_at_key_and_id(data, key, value, id, engine='auto')[source]¶

triplets.tools.triplets_to_tableviews(triplet_df, multivalue=False, engine='auto')[source]¶

triplets.tools.tableviews_to_triplets(tableviews, multivalue=False, engine='auto')[source]¶

triplets.tools.tableview_to_triplets(data, multivalue=False, instance_id=None, engine='auto')[source]¶

triplets.tools.update_triplets_from_triplets(data, update_data, update=True, add=True, engine='auto')[source]¶

triplets.tools.update_triplets_from_tableview(data, tableview, update=True, add=True, instance_id=None, engine='auto')[source]¶

triplets.tools.remove_triplets_from_triplets(from_triplet, what_triplet, columns=['ID', 'KEY', 'VALUE'], engine='auto')[source]¶

triplets.tools.diff_triplets(old_data, new_data, engine='auto')[source]¶

triplets.tools.diff_triplets_by_instance(data, INSTANCE_ID_1, INSTANCE_ID_2, engine='auto')[source]¶

triplets.tools.print_triplets_diff(old_data, new_data, file_id_object='Distribution', file_id_key='label', exclude_objects=None, engine='auto')[source]¶

triplets.tools.diff_between_INSTANCE(data, INSTANCE_ID_1, INSTANCE_ID_2, engine='auto')¶

triplets.tools.diff_between_triplet(old_data, new_data, engine='auto')¶

triplets.tools.filter_by_triplet(data, filter_triplet, engine='auto')¶

triplets.tools.filter_by_type(data, type_name, type_key='Type', engine='auto')¶

triplets.tools.get_types_count(data, engine='auto')¶

triplets.tools.print_triplet_diff(old_data, new_data, file_id_object='Distribution', file_id_key='label', exclude_objects=None, engine='auto')¶

triplets.tools.remove_triplet_from_triplet(from_triplet, what_triplet, columns=['ID', 'KEY', 'VALUE'], engine='auto')¶

triplets.tools.set_VALUE_at_KEY(data, key, value, engine='auto')¶

triplets.tools.set_VALUE_at_KEY_and_ID(data, key, value, id, engine='auto')¶

triplets.tools.tableview_by_id(data, id, string_to_number=True, multivalue=False, engine='auto')¶

triplets.tools.tableview_by_key(data, key, string_to_number=True, multivalue=False, engine='auto')¶

triplets.tools.tableview_by_type(data, type_name, string_to_number=True, type_key='Type', multivalue=False, engine='auto')¶

triplets.tools.tableview_to_triplet(data, multivalue=False, instance_id=None, engine='auto')¶

triplets.tools.tableviews_to_triplet(tableviews, multivalue=False, engine='auto')¶

triplets.tools.triplet_to_tableviews(triplet_df, multivalue=False, engine='auto')¶

triplets.tools.update_triplet_from_tableview(data, tableview, update=True, add=True, instance_id=None, engine='auto')¶

triplets.tools.update_triplet_from_triplet(data, update_data, update=True, add=True, engine='auto')¶

triplets.tools.pandas_engine¶

triplets.tools.pandas_engine.get_namespace_map(data: DataFrame)[source]¶

Extract namespace prefix-to-URI mapping and optional xml:base from a triplet dataset.

This function searches for a NamespaceMap object (identified by KEY='Type' and VALUE='NamespaceMap') within the dataset. It then collects all key-value pairs under that instance where: - KEY is the namespace prefix (e.g., “cim”, “rdf”) - VALUE is the full URI (e.g., “http://iec.ch/TC57/2013/CIM-schema-cim16#”)

Special keys: - xml_base: Extracted separately if present (used as base URI in RDF). - Type: Automatically excluded.

Parameters:

data (pandas.DataFrame) – Triplet dataset with columns [‘INSTANCE_ID’, ‘ID’, ‘KEY’, ‘VALUE’]. Must contain a NamespaceMap instance for successful extraction.

Returns:

namespace_map (dict) – Mapping of namespace prefixes to URIs (e.g., {"cim": "...", "rdf": "..."}). Empty dict if no NamespaceMap is found.
xml_base (str) – Value of xml_base if defined within the NamespaceMap; otherwise empty str.

Examples

>>> ns_map, base = get_namespace_map(triplet_data)
>>> print(ns_map)
{'cim': 'http://iec.ch/TC57/2013/CIM-schema-cim16#', 'rdf': 'http://www.w3.org/1999/02/22-rdf-syntax-ns#'}
>>> print(base)
'http://example.com/base/'

>>> ns_map, base = get_namespace_map(empty_data)
>>> print(ns_map, base)
{} ""

Notes

The function is idempotent and safe to call on any dataset.
Uses inner merge on ID to scope entries to the correct NamespaceMap instance.
Always returns a tuple of length 2: (dict, str).

triplets.tools.pandas_engine.type_tableview(data, type_name, string_to_number=True, type_key='Type', multivalue=False)[source]¶

Create a table view of all objects of a specified type.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
type_name (str) – The type of objects to filter (e.g., ‘ACLineSegment’).
string_to_number (bool, optional) – If True, convert columns containing numbers to numeric types (default is True).
type_key (str, optional) – Key used to identify object types in the dataset (default is ‘Type’).
multivalue (bool, optional) – If True, aggregate duplicate (ID, KEY) pairs into lists (default is False).

Returns:

Pivoted DataFrame with IDs as index and keys as columns, or None if no data is found.

Return type:

pandas.DataFrame or None

Examples

>>> table = data.type_tableview("ACLineSegment", multivalue=True)

triplets.tools.pandas_engine.key_tableview(data, key, string_to_number=True, multivalue=False)[source]¶

Create a table view of all objects with a specified key.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
key (str) – The key to filter objects by (e.g., ‘GeneratingUnit.maxOperatingP’).
string_to_number (bool, optional) – If True, convert columns containing numbers to numeric types (default is True).
multivalue (bool, optional) – If True, aggregate duplicate (ID, KEY) pairs into lists (default is False).

Returns:

Pivoted DataFrame with IDs as index and keys as columns, or None if no data is found.

Return type:

pandas.DataFrame or None

Examples

>>> table = data.key_tableview("GeneratingUnit.maxOperatingP")

triplets.tools.pandas_engine.id_tableview(data, id, string_to_number=True, multivalue=False)[source]¶

Create a tabular view of a CGMES triplet dataset filtered by ID-s.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing CGMES data.
id (str or list or pandas.DataFrame) – ID(s) to filter by (single ID, list of IDs, or DataFrame with an ID column).
string_to_number (bool, optional) – If True, convert columns containing numbers to numeric types (default is True).
multivalue (bool, optional) – If True, aggregate duplicate (ID, KEY) pairs into lists (default is False).

Returns:

Pivoted DataFrame with IDs as index and KEYs as columns.

Return type:

pandas.DataFrame or None

Examples

>>> table = id_tableview(data, 'UUID')
>>> table = id_tableview(data, ['UUID_1', 'UUID_2'])
>>> table = id_tableview(data, pandas.DataFrame({"ID": ['UUID_1', 'UUID_2']}))

triplets.tools.pandas_engine.references_to_simple(data, reference, columns=['Type'])[source]¶

Create a simplified table view of objects referencing a specified object.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
reference (str) – ID of the object to find references to.
columns (list, optional) – Columns to include in the output table (default is [‘Type’]).

Returns:

Pivoted DataFrame with IDs of referencing objects and specified columns.

Return type:

pandas.DataFrame

Examples

>>> table = data.references_to_simple("99722373_VL_TN1")

triplets.tools.pandas_engine.references_to(data, reference, levels=1)[source]¶

Retrieve all objects pointing to a specified reference object.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
reference (str) – ID of the reference object.
levels (int, optional) – Number of reference levels to traverse (default is 1).

Returns:

DataFrame containing triplets of objects pointing to the reference, with a ‘level’ column.

Return type:

pandas.DataFrame

Notes

TODO: Add the key on which the connection was made.

Examples

>>> refs = data.references_to("99722373_VL_TN1", levels=2)

triplets.tools.pandas_engine.references_from_simple(data, reference, columns=['Type'])[source]¶

Create a simplified table view of objects a specified object refers to.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
reference (str) – ID of the object to find references from.
columns (list, optional) – Columns to include in the output table (default is [‘Type’]).

Returns:

Pivoted DataFrame with IDs of referenced objects and specified columns.

Return type:

pandas.DataFrame

Examples

>>> table = data.references_from_simple("99722373_VL_TN1")

triplets.tools.pandas_engine.references_from(data, reference, levels=1)[source]¶

Retrieve all objects a specified object points to.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
reference (str) – ID of the reference object.
levels (int, optional) – Number of reference levels to traverse (default is 1).

Returns:

DataFrame containing triplets of objects referenced by the input, with a ‘level’ column.

Return type:

pandas.DataFrame

Notes

TODO: Add the key on which the connection was made.

Examples

>>> refs = data.references_from("99722373_VL_TN1", levels=2)

triplets.tools.pandas_engine.references_all(data)[source]¶

Find all unique references (links) in the dataset.

Parameters:: data (pandas.DataFrame) – Triplet dataset containing RDF data.
Returns:: DataFrame with columns [‘ID_FROM’, ‘KEY’, ‘ID_TO’] representing all references.
Return type:: pandas.DataFrame

Notes

Does not consider INSTANCE_ID in reference matching.

Examples

>>> refs = data.references_all()

triplets.tools.pandas_engine.references_simple(data, reference, columns=None, levels=1)[source]¶

Create a simplified table view of all references to and from a specified object.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
reference (str) – ID of the object to find references for.
columns (list, optional) – Columns to include in the output table (default is [‘Type’, ‘IdentifiedObject.name’] if available).
levels (int, optional) – Number of reference levels to traverse (default is 1).

Returns:

Pivoted DataFrame with IDs, specified columns, and reference levels.

Return type:

pandas.DataFrame

Examples

>>> table = data.references_simple("99722373_VL_TN1", columns=["Type"])

triplets.tools.pandas_engine.references(data, ID, levels=1)[source]¶

Retrieve all references (to and from) a specified object.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
ID (str) – ID of the object to find references for.
levels (int, optional) – Number of reference levels to traverse (default is 1).

Returns:

DataFrame containing triplets of all references to and from the object.

Return type:

pandas.DataFrame

Examples

>>> refs = data.references("99722373_VL_TN1", levels=2)

triplets.tools.pandas_engine.types_dict(data)[source]¶

Return a dictionary of object types and their occurrence counts.

Parameters:: data (pandas.DataFrame) – Triplet dataset containing RDF data.
Returns:: Dictionary with object types as keys and their counts as values.
Return type:: dict

Examples

>>> types = data.types_dict()
>>> print(types)
{'ACLineSegment': 10, 'PowerTransformer': 5, ...}

triplets.tools.pandas_engine.set_value_at_key(data, key, value)[source]¶

Set the value for all instances of a specified key.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
key (str) – The key to update.
value (str) – The new value to set for the specified key.

Notes

TODO: Add debug logging for key, initial value, and new value.
TODO: Store changes in a changes DataFrame.

Examples

>>> data.set_value_at_key("label", "new_label")

triplets.tools.pandas_engine.set_value_at_key_and_id(data, key, value, id)[source]¶

Set the value for a specific key and ID.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
key (str) – The key to update.
value (str) – The new value to set.
id (str) – The ID of the object to update.

Examples

>>> data.set_value_at_key_and_id("label", "new_label", "uuid1")

triplets.tools.pandas_engine.triplets_to_tableviews(triplet_df, multivalue=False)[source]¶

Convert triplet DataFrame to dict of tableview DataFrames.

Parameters:

triplet_df (pandas.DataFrame) – Triplet dataset with columns [ID, KEY, VALUE, INSTANCE_ID].
multivalue (bool, default False) – If True, aggregate duplicate (ID, KEY) pairs into lists.

Returns:

{class_name: tableview_df}

Return type:

dict

triplets.tools.pandas_engine.get_object_data(data, object_UUID)[source]¶

Retrieve data for a specific object by its UUID.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
object_UUID (str) – UUID of the object to retrieve.

Returns:

Series with keys as index and values for the specified object.

Return type:

pandas.Series

Examples

>>> obj_data = data.get_object_data("uuid1")

triplets.tools.pandas_engine.tableview_to_triplets(data, multivalue=False, instance_id=None)[source]¶

Convert a table view back to a triplet format.

Parameters:

data (pandas.DataFrame) – Pivoted DataFrame (table view) to convert.
multivalue (bool, optional) – If True, unpack list values into separate triplets (default is False).
instance_id (str, optional) – If given, stamp an INSTANCE_ID column on the result (default None).

Returns:

Triplet DataFrame with columns [‘ID’, ‘KEY’, ‘VALUE’] (plus ‘INSTANCE_ID’ when instance_id is given).

Return type:

pandas.DataFrame

Notes

An empty tableview cell is not a triplet — those holes are dropped so this is a faithful inverse of the tableview build (matches the duckdb engine’s WHERE VALUE IS NOT NULL). INSTANCE_ID is not carried by a tableview; pass instance_id to stamp it, the same way update_triplets_from_tableview does.

triplets.tools.pandas_engine.update_triplets_from_triplets(data, update_data, update=True, add=True)[source]¶

Update or add triplets from another triplet dataset.

Parameters:

data (pandas.DataFrame) – Original triplet dataset to update.
update_data (pandas.DataFrame) – Triplet dataset containing updates or new data.
update (bool, optional) – If True, update existing ID-KEY pairs (default is True).
add (bool, optional) – If True, add new ID-KEY pairs (default is True).

Returns:

Updated triplet dataset.

Return type:

pandas.DataFrame

Notes

TODO: Add a changes DataFrame to track modifications.
TODO: Support updating ID and KEY fields.

Examples

>>> updated_data = data.update_triplets_from_triplets(update_data)

triplets.tools.pandas_engine.update_triplets_from_tableview(data, tableview, update=True, add=True, instance_id=None)[source]¶

Update or add triplets from a table view.

Parameters:

data (pandas.DataFrame) – Original triplet dataset to update.
tableview (pandas.DataFrame) – Table view containing updates or new data.
update (bool, optional) – If True, update existing ID-KEY pairs (default is True).
add (bool, optional) – If True, add new ID-KEY pairs (default is True).
instance_id (str, optional) – Instance ID to assign to new triplets (default is None).

Returns:

Updated triplet dataset.

Return type:

pandas.DataFrame

Examples

>>> updated_data = data.update_triplets_from_tableview(table_view, instance_id="uuid1")

triplets.tools.pandas_engine.remove_triplets_from_triplets(from_triplet, what_triplet, columns=['ID', 'KEY', 'VALUE'])[source]¶

Remove triplets from one dataset that match another.

Parameters:

from_triplet (pandas.DataFrame) – Original triplet dataset.
what_triplet (pandas.DataFrame) – Triplet dataset to remove from the original.
columns (list, optional) – Columns to match for removal (default is [‘ID’, ‘KEY’, ‘VALUE’]).

Returns:

Dataset with matching triplets removed.

Return type:

pandas.DataFrame

Examples

>>> result = remove_triplets_from_triplets(data, to_remove)

triplets.tools.pandas_engine.filter_triplets_by_triplets(data, filter_triplet)[source]¶

Filter riplet DataFrame using IDs from another DataFrame.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing CGMES data.
filter_triplet (pandas.DataFrame) – DataFrame containing atleast colum ID to filter by.

Returns:

Filtered DataFrame with columns [‘ID, ‘KEY’, ‘VALUE’, ‘INSTANCE_ID’].

Return type:

pandas.DataFrame

Examples

>>> filtered = filter_triplets_by_triplets(data, filter_triplet)

triplets.tools.pandas_engine.filter_triplets_by_type(data, type_name, type_key='Type')[source]¶

Filter triplet dataset by objects of a specific type.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing RDF data.
type_name (str) – Object type to filter by (e.g., ‘ACLineSegment’).
type_key (str) – Key used in triplet to indicate type, by default “Type”

Returns:

Filtered triplet dataset containing only objects of the specified type.

Return type:

pandas.DataFrame

Examples

>>> filtered = filter_triplets_by_type(data, "ACLineSegment")

triplets.tools.pandas_engine.filter_triplets(data, ID=None, KEY=None, VALUE=None, INSTANCE_ID=None, regex=False)[source]¶

Filter triplets by any combination of columns with optional regex.

Parameters:

data (pandas.DataFrame) – Triplet dataset with columns [ID, KEY, VALUE, INSTANCE_ID].
ID (str, optional) – Filter value. If regex=True, treated as regex pattern.
KEY (str, optional) – Filter value. If regex=True, treated as regex pattern.
VALUE (str, optional) – Filter value. If regex=True, treated as regex pattern.
INSTANCE_ID (str, optional) – Filter value. If regex=True, treated as regex pattern.
regex (bool, default False) – If True, use regex matching (re.search). If False, exact match.

Returns:

Filtered triplet dataset.

Return type:

pandas.DataFrame

Examples

>>> filter_triplets(data, KEY="Type", VALUE="ACLineSegment")
>>> filter_triplets(data, VALUE=".*Substation.*", regex=True)

triplets.tools.pandas_engine.diff_triplets(old_data, new_data)[source]¶

Compute the difference between two Triplet DataFrames.

Parameters:

old_data (pandas.DataFrame) – Original triplet dataset.
new_data (pandas.DataFrame) – New triplet dataset to compare against.

Returns:

DataFrame containing triplets unique to old_data or new_data, with an ‘_merge’ column indicating ‘left_only’ (in old_data) or ‘right_only’ (in new_data).

Return type:

pandas.DataFrame

Examples

>>> diff = diff_triplets(old_data, new_data)

triplets.tools.pandas_engine.diff_triplets_by_instance(data, INSTANCE_ID_1, INSTANCE_ID_2)[source]¶

Identify differences between two loaded INSTANCES, by thier INSTACE_ID in the same Triplet DataFrame.

Parameters:

data (pandas.DataFrame) – Triplet dataset containing two or more INSTANCE.
INSTANCE_ID_1 (str) – UUID of the first INSTANCE.
INSTANCE_ID_2 (str) – UUID of the second INSTANCE.

Returns:

DataFrame containing triplets that differ between the two model parts.

Return type:

pandas.DataFrame

Examples

>>> diff = diff_triplets_by_instance('uuid1', 'uuid2')

triplets.tools.pandas_engine.print_triplets_diff(old_data, new_data, file_id_object='Distribution', file_id_key='label', exclude_objects=None)[source]¶

Print a human-readable diff of two triplet datasets.

Parameters:

old_data (pandas.DataFrame) – Original triplet dataset.
new_data (pandas.DataFrame) – New triplet dataset to compare against.
file_id_object (str, optional) – Object type containing file identifiers (default is ‘Distribution’).
file_id_key (str, optional) – Key containing file identifiers (default is ‘label’).
exclude_objects (list, optional) – List of object types to exclude from the diff (default is None).

Notes

Outputs a diff format showing removed, added, and changed objects.
Nice diff viewer https://diffy.org/
TODO: Add name field for better reporting with Type.

Examples

>>> print_triplets_diff(old_data, new_data, exclude_objects=["NamespaceMap"])

triplets.tools¶

triplets.tools.pandas_engine¶

triplets.tools.polars_engine¶