triplets.sparql package

Submodules

triplets.sparql.sparql_oxigraph module

SPARQL engine — pyoxigraph (embedded Rust oxigraph, in-memory store).

The portable performance engine: a plain pip wheel, no compiled extension, Rust-speed queries. Sits between qlever (fastest, needs the compiled extension) and rdflib (pure-Python reference) in auto preference.

Data is loaded through the N-Quads export (datatype-annotated, INSTANCE_ID as named graph) via Store.bulk_load — loaded stores are cached in-process by content key, same logic as the qlever engine’s index cache. After loading, the deduplicated union of the named graphs is projected into the store’s default graph, so unscoped queries match rdflib’s default_union set semantics (oxigraph’s own union of named graphs keeps one solution per graph).

Scope is not a data operation: the scoped instances’ named graphs are passed to Store.query as SPARQL-protocol default graphs, so one store serves every scope and the query text is never modified. Caveat: a multi-instance scope is a protocol dataset union — a triple present in several scoped instances yields one solution per instance (DISTINCT dedupes; rdflib and the unscoped default graph deduplicate by construction).

Results follow the engine contract shared with qlever — all values are lexical strings (triplets are all-string; consumers cast):

  • SELECT decodes via oxigraph’s SPARQL-CSV serializer (lexical forms: IRIs bare, literals unquoted; unbound → null, indistinguishable from an empty-string literal — the W3C CSV-results tradeoff) straight into pandas/polars, no per-term Python loop.

  • CONSTRUCT/DESCRIBE serializes to N-Quads (Rust-side) and comes back through read_nquads — the same round-trip that loaded the data.

  • ASK stays a bool.

triplets.sparql.sparql_oxigraph.query(data, query_string, rdf_map=None, scope=None, return_type='auto', data_unchanged=False)[source]

Execute query_string over data; shape the result by query type.

Queries are executed exactly as given — the text is never modified. A rejected query raises ValueError carrying oxigraph’s message plus the query text, so the failure is directly actionable. scope travels beside the query as SPARQL-protocol default graphs: the query runs against exactly the union of the scoped instances’ named graphs on the one shared store, and per the protocol these take precedence over any FROM inside the query. None → the store’s default graph (the deduplicated union of all named graphs).

triplets.sparql.sparql_qlever module

SPARQL performance engine — embedded qlever (C++), no server.

Uses qlever’s official embedding facade (src/libqlever) through the triplets.sparql._qlever Cython extension (build: setup_qlever.py).

Flavor-blind by construction: the input (pandas / polars DataFrame or a DuckDB connection) carries everything as registered methods — content_hash keys the engine state, export_to_arrow feeds the index build — so this module never inspects input types. Scope is not a data operation: the scoped instances’ named graphs are passed to qlever as SPARQL-protocol dataset clauses (default-graph-uri), so one index serves every scope and the query text is never modified.

Both boundaries are zero-copy Arrow, following the read_rdf pattern — all heavy lifting happens on the C++ side:

  • in: the triplet columns go to the index builder as Arrow batches, feeding an injected parser (no N-Quads serialization or text re-parsing anywhere; the term mapping applies the same rules as the N-Quads export, from the same build_key_metadata schema interpretation).

  • out: the wrapper decodes the query result straight into Arrow string buffers (unbound → null), the Cython layer wraps them zero-copy, and this module only maps conventions and finalizes the output flavor (return_type: “auto” matches the input — polars in, polars out; explicit “pandas” / “polars” / “arrow” also accepted). All values are strings (triplets are all-string; consumers cast); ASK stays a bool via the tiny JSON path.

Engine state = one on-disk qlever index per content key (data + rdf_map), loaded engines cached in-process. The index directory is $TRIPLETS_QLEVER_DIR (point it at /dev/shm for RAM-backed indexes) or the temp dir; loaded index files are memory-mapped, so hot pages live in the OS page cache either way. The GIL is released during index building, queries and decoding: Python threads parallelize, no fork needed.

Benchmarked 3.5–216x faster than the alternatives on CGMES data; index load from disk ~4 ms.

triplets.sparql.sparql_qlever.query(data, query_string, rdf_map=None, scope=None, return_type='auto', data_unchanged=False)[source]

Execute query_string over data; shape the result by query type.

Queries are executed exactly as given — the text is never modified. qlever’s parser is strict; a rejected query raises ValueError carrying qlever’s message plus the query text, so the failure is directly actionable (broken constraint queries belong upstream, see TODO.md). scope travels beside the query as SPARQL-protocol dataset clauses (default-graph-uri): the query runs against exactly the union of the scoped instances’ named graphs on the one shared index, and per the protocol these take precedence over any FROM inside the query.

triplets.sparql.sparql_rdflib module

SPARQL reference engine — rdflib’s built-in SPARQL 1.1.

Correctness-first reference; the data is loaded into an in-memory rdflib Dataset via the N-Quads export — cached in-process by content key (see _rdflib_loader), so the export/parse runs only on a cache miss, same logic as the qlever engine’s index cache. Scope is applied after loading (named graphs), so one cached dataset serves all scopes.

Results follow the shared engine contract — all SELECT values are lexical strings (triplets are all-string; consumers cast), decoded through rdflib’s SPARQL-CSV result serializer (measured ~1.5x faster than iterating the result’s term objects, and the same decode the oxigraph engine uses). rdf_map still matters: typed literals in the loaded graph drive comparisons/ORDER BY inside the query — only the returned representation is string.

triplets.sparql.sparql_rdflib.query(data, query_string, rdf_map=None, scope=None, return_type='auto', data_unchanged=False)[source]

Execute query_string over data; shape the result by query type.

Module contents

SPARQL querying over triplet data.

Engines (registry dispatch, mirroring triplets.parser): - qlever — performance (embedded C++ via the official libqlever facade; needs

the compiled extension, see setup_qlever.py; takes auto priority when built)

  • oxigraph — portable performance (embedded Rust via the pyoxigraph wheel, oxigraph extra; auto priority when qlever is not built)

  • rdflib — reference, built-in SPARQL 1.1, always available with the sparql extra

Data reaches every engine through the N-Quads export conventions (INSTANCE_ID as named graph): rdflib and oxigraph load the export directly, qlever ingests the same term mapping as Arrow batches.

triplets.sparql.register_engine(name: str, module: Any) → None[source]

Register a custom SPARQL engine for future extensibility.

triplets.sparql.get_engine(name: str = 'auto')[source]

Resolve SPARQL engine name (with aliases) and return (name, module).

triplets.sparql.query(data, query_string, rdf_map=None, scope=None, engine='auto', return_type='auto', data_unchanged=False)[source]

Run a SPARQL query over triplet data.

Parameters:

  • data (triplet DataFrame (pandas/polars) or DuckDB connection)

  • query_string (str) – SPARQL query. SELECT → DataFrame (columns = projected vars), ASK → bool, CONSTRUCT/DESCRIBE → triplet DataFrame.

  • rdf_map (dict or str, optional) – Export schema — enables xsd-typed literals in the queried graph (optional).

  • scope (iterable of INSTANCE_ID, optional) – Restrict the queried data to these instances’ named graphs; all data stays loaded for reference resolution. None = full union.

  • engine (str, default "auto") – “qlever” (performance, embedded C++), “oxigraph” (portable performance, embedded Rust) or “rdflib” (reference). “auto” picks the first available in that order.

  • return_type (str, default "auto") – Output flavor for data results: “auto” matches the input (polars in → polars out; pandas/duckdb → pandas), or explicit “pandas” / “polars” / “arrow”. All SELECT values are lexical strings in every engine (triplets are all-string; consumers cast).

  • data_unchanged (bool, default False) – Assert that data has not been mutated since it was last hashed: the engine reuses the stored content digest for this exact object and skips the content_hash (the dominant cost of small warm queries). Only skips work when this object was hashed before; otherwise the hash runs and is remembered.