RDF & SPARQL

Toni
2018-10-18

What and Why?

Before.

Plot the unemployment rate?

Obtain a spreadsheet with the data:

Type the values into your favorite plotting program:

rate={5.70,4.20,4.50,3.80,3.80,3.60};

Make the plot:

ListPlot[rate,Joined->True]

What and Why?

Now.

Make a map of places around you?

PacletDirectoryAdd["~/git/graphstore"];
Get["GraphStore`"];

The Wikidata SPARQL endpoint:

endpoint="https://query.wikidata.org/sparql";

Around query:

wikibase[s_]:=IRI["http://wikiba.se/ontology#"<>s];
bd[s_]:=IRI["http://www.bigdata.com/rdf#"<>s];
wdt[s_]:=IRI["http://www.wikidata.org/prop/direct/"<>s];

aroundQuery[center_,radius_]:=SPARQLService[wikibase["around"],{
RDFTriple[SPARQLVariable["place"],wdt["P625"],SPARQLVariable["location"]],
RDFTriple[bd["serviceParam"],wikibase["center"],center],
RDFTriple[bd["serviceParam"],wikibase["radius"],radius(*km*)]
}];

Label query:

labelQuery[lang_:"en"]:=SPARQLService[wikibase["label"],{
RDFTriple[bd["serviceParam"],wikibase["language"],lang]
}];

Execute the query:

places=SPARQLExecute[
endpoint,
SPARQLSelect[{
aroundQuery[Here,1],
labelQuery[]
}->{SPARQLVariable["placeLabel"],SPARQLVariable["location"]}]
];

Length[places]

Make the map:

GeoListPlot[Tooltip[#[SPARQLVariable["location"]],#[SPARQLVariable["placeLabel"]]]&/@places]

Overview

RDF

representation of knowledge
retrieve information about resources

SPARQL

SPARQL crash-course
local vs remote execution
query string vs symbolic query
SPARQL features
Query entities
SQL vs SPARQL

Future

R2RML
Entailment

RDF

Representation of knowledge

RDF - resource description framework

A “resource”, Albert Einstein:

IRI["http://www.wikidata.org/entity/Q937"]

Properties are resources, like “label”:

IRI[(URLArgument["http://www.w3.org/2000/01/rdf-schema#label"])];

A literal:

RDFLiteral["23.0",IRI[(URLArgument["http://qudt.org/vocab/unit#DegreeCelsius"])]];

Representation of knowledge

Make a statement, “Einstein is a Person”:

rdf[s_]:=IRI["http://www.w3.org/1999/02/22-rdf-syntax-ns#"<>s];
schema[s_]:=IRI["http://schema.org/"<>s];
wd[s_]:=IRI["http://www.wikidata.org/entity/"<>s];

RDFTriple[wd["Q937"],rdf["type"],schema["Person"]]

Collect multiple statements into an RDF graph:

rdfs[s_]:=IRI["http://www.w3.org/2000/01/rdf-schema#"<>s];

RDFStore[{
RDFTriple[wd["Q937"],rdf["type"],schema["Person"]],
RDFTriple[wd["Q937"],rdfs["label"],"Albert Einstein"]
}]

Retrieve information about resources

Caffeine:

IRI[(URLArgument["http://www.wikidata.org/entity/Q60235"])];

Request RDF data about a resource:

IRIInformation[IRI[i_String]]:=URLExecute[
HTTPRequest[i,<|"Headers"->{"Accept"->"text/turtle"}|>],
"Turtle"
];

IRIInformation[IRI[(URLArgument["http://www.wikidata.org/entity/Q60235"])]]

Other linked data providers:

IRIInformation[IRI[(URLArgument["http://purl.uniprot.org/uniprot/Q9C1S9"])]]

IRIInformation[IRI[(URLArgument["http://www.w3.org/2006/time#seconds"])]]

SPARQL

SPARQL crash-course

In memory RDF data:

store=RDFStore[];

Find the properties:

SPARQLQuery["select distinct ?p where {?s ?p ?o .}"][store]

Find the subjects:

SPARQLQuery["select distinct ?s where {?s ?p ?o .}"][store]

Find Toni’s workplace:

SPARQLQuery["select * where {
  <http://example.org/toni> <http://example.org/worksAt> ?w .
}"][store]

Name of all people that work at Wolfram:

SPARQLQuery["select ?name where {
  ?p <http://example.org/worksAt> <http://example.org/wolfram> .
  ?p <http://www.w3.org/2000/01/rdf-schema#label> ?name .
}"][store]

Local vs remote execution

A query to count all triples:

triplesQuery=SPARQLQuery["select (count(*) as ?count) where {?x ?y ?z}"];

Local

localStore=RDFStore[];

Apply the query to an RDF graph:

triplesQuery[localStore]

Remote

remoteEndpoint=IRI[(URLArgument["https://query.wikidata.org/sparql"])];

Execute the query on a remote SPARQL endpoint:

SPARQLExecute[remoteEndpoint,triplesQuery]

Query string vs symbolic query

String

Formula and structure image of caffeine (Q60235):

SPARQLExecute["https://query.wikidata.org/sparql","
select * where {
  wd:Q60235                # caffeine
    wdt:P274 ?formula;
    wdt:P117 ?image.
}
"]

Symbolic

The same query:

SPARQLExecute["https://query.wikidata.org/sparql",SPARQLSelect[{
RDFTriple[wd["Q60235"],wdt["P274"],SPARQLVariable["formula"]],
RDFTriple[wd["Q60235"],wdt["P117"],SPARQLVariable["image"]]
}]]

SPARQL features

Graph patterns

Books published before 1800:

ex[s_]:=IRI["http://example.org/"<>s];

SPARQLSelect[{
RDFTriple[SPARQLVariable["book"],rdfs["label"],SPARQLVariable["bookLabel"]],
RDFTriple[SPARQLVariable["book"],ex["publicationYear"],SPARQLVariable["year"]]
}/;SPARQLVariable["year"]<1800];

Optional

SPARQLSelect[{
RDFTriple[SPARQLVariable["book"],rdfs["label"],SPARQLVariable["bookLabel"]],
SPARQLOptional[
RDFTriple[SPARQLVariable["book"],ex["publicationYear"],SPARQLVariable["year"]]
]
}];

Union

SPARQLSelect[
RDFTriple[SPARQLVariable["item"],rdfs["label"],SPARQLVariable["label"]]|
RDFTriple[SPARQLVariable["item"],ex["name"],SPARQLVariable["label"]]
];

Negation

People without e-mail address:

foaf[s_]:=IRI["http://xmlns.com/foaf/0.1/"<>s];

SPARQLSelect[{
RDFTriple[SPARQLVariable["person"],foaf["name"],SPARQLVariable["name"]]/;!SPARQLEvaluation["exists"][{
RDFTriple[SPARQLVariable["person"],foaf["mbox"],SPARQLVariable["mail"]]
}]
}];

Property paths

Inverse, sequence, alternative, variable length, negation.

SPARQLSelect[
SPARQLPropertyPath[SPARQLVariable["person"],{ex["hasFather"],ex["livesIn"]},SPARQLVariable["city"]]
];

SPARQLSelect[
SPARQLPropertyPath[ex["tom"],{foaf["knows"]..},SPARQLVariable["friend"]]
];

Aggregates

Number of books by author:

SPARQLQuery[
SPARQLSelect[
RDFTriple[SPARQLVariable["book"],ex["author"],SPARQLVariable["author"]]
]/*SPARQLAggregate[
{
SPARQLVariable["author"],
SPARQLVariable["count"]->SPARQLEvaluation["count"][]
},
SPARQLVariable["author"]
]
];

Subqueries

SPARQLSelect[{
RDFTriple[ex["fred"],foaf["knows"],SPARQLVariable["friend"]],
SPARQLSelect[
RDFTriple[SPARQLVariable["friend"],ex["hobby"],SPARQLVariable["hobby"]],
"Limit"->3
]
}];

Federation

SPARQLSelect[{
RDFTriple[SPARQLVariable["item"],ex["wikidataItem"],SPARQLVariable["id"]],
SPARQLService[IRI[(URLArgument["https://query.wikidata.org/sparql"])],{
RDFTriple[SPARQLVariable["id"],wdt["P18"],SPARQLVariable["image"]]
}]
}];

Solution modifiers

SPARQLSelect[{
RDFTriple[SPARQLVariable["person"],foaf["mbox"],SPARQLVariable["mail"]],
RDFTriple[SPARQLVariable["person"],foaf["name"],SPARQLVariable["name"]]
}->SPARQLVariable["mail"],
"Distinct"->True,
"OrderBy"->SPARQLVariable["name"],
"Limit"->100,
"Offset"->1000
];

Query entities

Alpha

Find daughter nuclides of mercury-204:

daughterNuclidesQuery=SPARQLSelect[SPARQLPropertyPath[
(Entity["mercury-204", Entity["Isotope","Mercury204"], "Entity[\"Isotope\", \"Mercury204\"]", "isotope"]),
{(EntityProperty["daughter nuclides", EntityProperty["Isotope","DaughterNuclides"], "EntityProperty[\"Isotope\", \"DaughterNuclides\"]"])..},
SPARQLVariable["daughter"]
]];

daughterNuclidesQuery[Entity["Isotope"]]

Local

Register some “things”:

EntityRegister[EntityStore["thing"-><|
"Entities"->{"banana","bike","orange"},
"EntityClasses"-><|
"fruits"-><|"Entities"->{"banana","orange"},"Label"->"fruits"|>
|>
|>]];

Find things in the class “fruits”:

SPARQLSelect[
RDFTriple[SPARQLVariable["fruit"],rdf["type"],EntityClass["thing","fruits"]]
][Entity["thing"]]

EntityUnregister["thing"];

SQL vs SPARQL

Query:

Find people, their father, and the city in which their father lives.

SQL vs SPARQL

SQL.

Data

peopleDB=CreateFile[];

Create a city and person table:

Needs["DatabaseLink`"]

peopleConn=OpenSQLConnection[JDBC["SQLite",peopleDB]];

SQLExecute[peopleConn,"create table city (
  id int primary key,
  name text
);"];
SQLExecute[peopleConn,"create table person (
  id int primary key,
  name text,
  father int,
  mother int,
  livesin int,
  foreign key(father) references person(id),
  foreign key(mother) references person(id),
  foreign key(livesin) references city(id)
);"];
SQLInsert[peopleConn,"city",{"id","name"},{
{1,"Madrid"},
{2,"München"},
{3,"Somogyvár"}
}];
SQLInsert[peopleConn,"person",{"id","name","father","mother","livesin"},{
{1,"Harald",Null,Null,2},
{2,"Marika",Null,Null,3},
{3,"Toni",1,2,1}
}];

Query

SQLExecute[peopleConn,"
select person.name as person,
  father.name as father,
  city.name as city
from
  person,
  person as father,
  city
where
  person.father = father.id
  and father.livesin = city.id
","ShowColumnHeadings"->True]//TableForm

SQL vs SPARQL

SPARQL.

Data

peopleGraph=ImportString["
prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>
prefix ex: <http://example.org/>

ex:city1 a ex:City; rdfs:label \"Madrid\".
ex:city2 a ex:City; rdfs:label \"MÃ¼nchen\".
ex:city3 a ex:City; rdfs:label \"SomogyvÃ¡r\".

ex:person1 a ex:Person; rdfs:label \"Harald\";
  ex:livesIn ex:city2.
ex:person2 a ex:Person; rdfs:label \"Marika\";
  ex:livesIn ex:city3.
ex:person3 a ex:Person; rdfs:label \"Toni\";
  ex:livesIn ex:city1; 
  ex:hasFather ex:person1; ex:hasMother ex:person2.
","Turtle"];

Query

peopleGraph//SPARQLQuery["
prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>
prefix ex: <http://example.org/>

select ?personLabel ?fatherLabel ?cityLabel where {

  ?person ex:hasFather ?father .
  ?father ex:livesIn ?city .

  ?person rdfs:label ?personLabel .
  ?father rdfs:label ?fatherLabel .
  ?city rdfs:label ?cityLabel .
}
"]

Future

R2RML

relational to RDF mapping language

Workflow

1) look at your relational database

2) choose a vocabulary

3) write a mapping from tables to triples

4a) create an RDF view
or
4b) convert to an RDFStore

R2RML: View

Weather data: Create an RDF view; SPARQL queries are translated to SQL.

DB

weatherDB=CreateFile[];

Create station and observation table and insert data:

weatherConn=OpenSQLConnection[JDBC["SQLite",weatherDB]];
SQLExecute[weatherConn,#]&/@{
"create table station (id int primary key, name text, lat real, lon real)",
"create table observation (id int primary key, station int, date date, temperature real, foreign key(station) references station(id))",

"insert into station values (1, 'Madrid', 40.42, -3.71)",
"insert into station values (2, 'München', 48.14, 11.58)",

"insert into observation values (1, 1, '2018-06-07', 23)",
"insert into observation values (2, 2, '2018-06-07', 25)",
"insert into observation values (3, 1, '2018-06-08', 22)",
"insert into observation values (4, 2, '2018-06-08', 23)"
};
CloseSQLConnection[weatherConn];

Mapping

Map station and observation tables:

weatherMapping=ImportString["
prefix rr: <http://www.w3.org/ns/r2rml#>
prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>
prefix time: <http://www.w3.org/2006/time#>
prefix xsd: <http://www.w3.org/2001/XMLSchema#>
prefix qudt: <http://qudt.org/vocab/unit#>
prefix ex: <http://example.org/>

_:stationMap
 rr:logicalTable [rr:tableName \"station\"];
 rr:subjectMap [
  rr:template \"http://example.org/station/{id}\";
  rr:class ex:Station;
 ];
 rr:predicateObjectMap [
  rr:predicate rdfs:label;
  rr:objectMap [rr:column \"name\"];
 ].

_:observationMap
 rr:logicalTable [rr:tableName \"observation\"];
 rr:subjectMap [
  rr:template \"http://example.org/observation/{id}\";
  rr:class ex:Observation;
 ];
 rr:predicateObjectMap [
  rr:predicate ex:station;
  rr:objectMap [rr:parentTriplesMap _:stationMap; rr:joinCondition [rr:child \"station\"; rr:parent \"id\"]];
 ], [
  rr:predicate time:inXSDDate;
  rr:objectMap [rr:column \"date\"; rr:datatype xsd:date];
 ], [
  rr:predicate ex:temperature;
  rr:objectMap[rr:column \"temperature\"; rr:datatype qudt:DegreeCelsius];
 ].
","Turtle"];

View

weatherStore=DatabaseRDFStore[weatherDB,weatherMapping]

Example queries

Stations:

weatherStore//SPARQLSelect[{
RDFTriple[SPARQLVariable["station"],rdf["type"],ex["Station"]],
RDFTriple[SPARQLVariable["station"],rdfs["label"],SPARQLVariable["stationLabel"]]
}->SPARQLVariable["stationLabel"]]

Weather in Madrid:

weatherStore//SPARQLQuery["
prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>
prefix time: <http://www.w3.org/2006/time#>
prefix ex: <http://example.org/>

select ?date ?temperature where {
  ?station rdfs:label \"Madrid\".
  ?observation ex:station ?station;
    time:inXSDDate ?date;
    ex:temperature ?temperature.
}
"]

Use cases

- large dataset
- frequently changing data

R2RML: Conversion

Conversion

Music data - convert a relation database to an RDF graph.

DB

musicDB=CreateFile[];

Create an artist and a work table and insert data:

musicDBConn=OpenSQLConnection[JDBC["SQLite",musicDB]];
SQLExecute[musicDBConn,#]&/@{
"create table artist (id integer primary key, name text)",
"create table work (id integer, title text, composer integer, foreign key(composer) references artist(id))",

"insert into artist values (17, 'Gerardo Matos Rodríguez')",
"insert into work values (5, 'La Cumparsita', 17)"
};
CloseSQLConnection[musicDBConn];

Mapping

Map artist and work tables.

musicMapping=ImportString["
prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>
prefix rr: <http://www.w3.org/ns/r2rml#>
prefix ex: <http://example.org/>

_:artistMap
 rr:logicalTable [rr:tableName \"artist\"];
 rr:subjectMap [
  rr:template \"http://example.org/artist/{id}\";
  rr:class ex:Artist;
 ];
 rr:predicateObjectMap [
  rr:predicate rdfs:label;
  rr:objectMap [rr:column \"name\"];
 ].

_:workMap
 rr:logicalTable [rr:tableName \"work\"];
 rr:subjectMap [
  rr:template \"http://example.org/work/{id}\";
  rr:class ex:Work;
 ];
 rr:predicateObjectMap [
  rr:predicate rdfs:label;
  rr:objectMap [rr:column \"title\"];
 ];
 rr:predicateObjectMap [
  rr:predicate ex:composer;
  rr:objectMap [
   rr:parentTriplesMap _:artistMap;
   rr:joinCondition [
    rr:child \"composer\";
    rr:parent \"id\";
   ];
  ];
 ].
","Turtle"];

Conversion

musicStore=Normal[DatabaseRDFStore[musicDB,musicMapping]]

View the music graph in Turtle syntax:

ExportString[musicStore,"Turtle","Prefixes"-><|
"rdfs"->"http://www.w3.org/2000/01/rdf-schema#",
"ex"->"http://example.org/"
|>]

Example query

Find all works together with their composer:

musicStore//SPARQLQuery["
prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#>
prefix ex: <http://example.org/>

select * where {
  ?work ^rdfs:label/ex:composer/rdfs:label ?composer.
}
"]

Use cases

- publish dumps
- load into a SPARQL endpoint

Entailment

Infer additional triples from existing ones.

familyGraph=RDFStore[{
RDFTriple[ex["hasFather"],rdfs["domain"],schema["Person"]],
RDFTriple[ex["hasFather"],rdfs["range"],schema["Person"]],

RDFTriple[ex["alfred"],ex["hasFather"],ex["wolfgang"]]
}];

No entailment

Is ex:hasFather a property? No.

familyGraph//SPARQLAsk[RDFTriple[ex["hasFather"],rdf["type"],rdf["Property"]]]

RDF entailment

RDF semantics make ex:hasFather an rdf:Property, because it appears in the predicate position:

familyGraph//SPARQLQuery[
SPARQLAsk[RDFTriple[ex["hasFather"],rdf["type"],rdf["Property"]]],
"EntailmentRegime"->"RDF"
]

RDF can’t infer that Wolfgang is a Person:

familyGraph//SPARQLQuery[
SPARQLAsk[RDFTriple[ex["wolfgang"],rdf["type"],schema["Person"]]],
"EntailmentRegime"->"RDF"
]

RDFS entailment

RDFS is RDF plus additional rules. Now Wolfgang is a Person:

familyGraph//SPARQLQuery[
SPARQLAsk[RDFTriple[ex["wolfgang"],rdf["type"],schema["Person"]]],
"EntailmentRegime"->"RDFS"
]

Find all persons:

familyGraph//SPARQLQuery[
SPARQLSelect[RDFTriple[SPARQLVariable["person"],rdf["type"],schema["Person"]]],
"EntailmentRegime"->"RDFS"
]

If e is in C1 and C1 is a sub-class of C2, then e is also in C2:

RDFStore[{
RDFTriple[ex["e"],rdf["type"],ex["C1"]],
RDFTriple[ex["C1"],rdfs["subClassOf"],ex["C2"]]
}]//SPARQLQuery[
SPARQLSelect[RDFTriple[SPARQLVariable["e"],rdf["type"],ex["C2"]]],
"EntailmentRegime"->"RDFS"
]

OWL

Complex class and property relations.

Thanks.

Toni
2018-10-18
tonis@wolfram.com