Explain

On this page

The explain functionality allows you to extract information about the query plan of a Comunica query engine.

There are three explain modes available:

parsed: The SPARQL Algebra tree as parsed from the input query.
logical: The optimized logical query plan in SPARQL Algebra.
physical: A hierarchical log of which logical operations have been executed by which (physical) actors.

While the parsed and logical explain modes happen before query execution, the physical explain mode requires query execution to be completed. This is because Comunica is an adaptive query engine that alters its query plan dynamically based on the sources it discovers at runtime. This means that query execution must be completed before the final (physical) query plan can be inspected.

If you require more insight into what operations are being executed at runtime, you can make use of the built-in logging functionality.

The output for the physical mode is an experimental feature, which means that the format of it might improve and be changed in the future inbetween major updates.

Explaining on the command line

If you have installed Comunica SPARQL for the command line, then you will have immediate access to the query explain functionality via the --explain option.

Below, you can see examples on how the different explain modes can be invoked.

Explain parsed on the command line

$ comunica-sparql https://fragments.dbpedia.org/2016-04/en \
  -q 'SELECT * { ?s ?p ?o } LIMIT 100' --explain parsed

{
  "type": "slice",
  "input": {
    "type": "project",
    "input": {
      "type": "bgp",
      "patterns": [
        {
          "termType": "Quad",
          "value": "",
          "subject": {
            "termType": "Variable",
            "value": "s"
          },
          "predicate": {
            "termType": "Variable",
            "value": "p"
          },
          "object": {
            "termType": "Variable",
            "value": "o"
          },
          "graph": {
            "termType": "DefaultGraph",
            "value": ""
          },
          "type": "pattern"
        }
      ]
    },
    "variables": [
      {
        "termType": "Variable",
        "value": "s"
      },
      {
        "termType": "Variable",
        "value": "p"
      },
      {
        "termType": "Variable",
        "value": "o"
      }
    ]
  },
  "start": 0,
  "length": 100
}

Explain logical on the command line

$ comunica-sparql https://fragments.dbpedia.org/2016-04/en \
  -q 'SELECT * { ?s ?p ?o } LIMIT 100' --explain logical

{
  "type": "slice",
  "input": {
    "type": "project",
    "input": {
      "type": "join",
      "input": [
        {
          "termType": "Quad",
          "value": "",
          "subject": {
            "termType": "Variable",
            "value": "s"
          },
          "predicate": {
            "termType": "Variable",
            "value": "p"
          },
          "object": {
            "termType": "Variable",
            "value": "o"
          },
          "graph": {
            "termType": "DefaultGraph",
            "value": ""
          },
          "type": "pattern"
        }
      ]
    },
    "variables": [
      {
        "termType": "Variable",
        "value": "s"
      },
      {
        "termType": "Variable",
        "value": "p"
      },
      {
        "termType": "Variable",
        "value": "o"
      }
    ]
  },
  "start": 0,
  "length": 100
}

Explain physical on the command line

$ node bin/query.js https://fragments.dbpedia.org/2016-04/en \
  -q 'SELECT ?movie ?title ?name
WHERE {
  ?movie dbpedia-owl:starring [ rdfs:label "Brad Pitt"@en ];
         rdfs:label ?title;
         dbpedia-owl:director [ rdfs:label ?name ].
  FILTER LANGMATCHES(LANG(?title), "EN")
  FILTER LANGMATCHES(LANG(?name),  "EN")
}' --explain physical

project (movie,title,name)
  join
    join-inner(bind) bindOperation:(?g_0 http://www.w3.org/2000/01/rdf-schema#label "Brad Pitt"@en) bindCardEst:~2 cardReal:43 timeSelf:2.567ms timeLife:667.726ms
      join compacted-occurrences:1
        join-inner(bind) bindOperation:(?movie http://dbpedia.org/ontology/starring http://dbpedia.org/resource/Brad_Pitt) bindCardEst:~40 cardReal:43 timeSelf:6.011ms timeLife:641.139ms
          join compacted-occurrences:38
            join-inner(bind) bindOperation:(http://dbpedia.org/resource/12_Monkeys http://dbpedia.org/ontology/director ?g_1) bindCardEst:~1 cardReal:1 timeSelf:0.647ms timeLife:34.827ms
              filter compacted-occurrences:1
                join
                  join-inner(nested-loop) cardReal:1 timeSelf:0.432ms timeLife:4.024ms
                    pattern (http://dbpedia.org/resource/12_Monkeys http://www.w3.org/2000/01/rdf-schema#label ?title) cardEst:~1 src:0
                    pattern (http://dbpedia.org/resource/Terry_Gilliam http://www.w3.org/2000/01/rdf-schema#label ?name) cardEst:~1 src:0
          join compacted-occurrences:2
            join-inner(multi-empty) timeSelf:0.004ms timeLife:0.053ms
              pattern (http://dbpedia.org/resource/Contact_(1992_film) http://dbpedia.org/ontology/director ?g_1) cardEst:~0 src:0
              filter cardEst:~5,188,789.667
                join
                  join-inner(nested-loop) timeLife:0.6ms
                    pattern (http://dbpedia.org/resource/Contact_(1992_film) http://www.w3.org/2000/01/rdf-schema#label ?title) cardEst:~1 src:0
                    pattern (?g_1 http://www.w3.org/2000/01/rdf-schema#label ?name) cardEst:~20,013,903 src:0
      join compacted-occurrences:1
        join-inner(multi-empty) timeSelf:0.053ms timeLife:0.323ms
          pattern (?movie http://dbpedia.org/ontology/director ?g_1) cardEst:~118,505 src:0
          pattern (?movie http://dbpedia.org/ontology/starring http://wikidata.dbpedia.org/resource/Q35332) cardEst:~0 src:0
          filter cardEst:~242,311,843,844,161
            join
              join-inner(symmetric-hash) timeLife:36.548ms
                pattern (?movie http://www.w3.org/2000/01/rdf-schema#label ?title) cardEst:~20,013,903 src:0
                pattern (?g_1 http://www.w3.org/2000/01/rdf-schema#label ?name) cardEst:~20,013,903 src:0

sources:
  0: QuerySourceHypermedia(https://fragments.dbpedia.org/2016-04/en)(SkolemID:0)

$ comunica-sparql https://fragments.dbpedia.org/2016-04/en \
  -q 'SELECT * { ?s ?p ?o. ?s a ?o } LIMIT 100' --explain physical-json

{
  "logical": "slice",
  "children": [
    {
      "logical": "project",
      "variables": [
        "o",
        "p",
        "s"
      ],
      "children": [
        {
          "logical": "join",
          "children": [
            {
              "logical": "join-inner",
              "physical": "bind",
              "bindIndex": 1,
              "bindOperation": {
                "source": "QuerySourceHypermedia(https://fragments.dbpedia.org/2016-04/en)(SkolemID:0)",
                "pattern": "?s http://www.w3.org/1999/02/22-rdf-syntax-ns#type ?o"
              },
              "bindOperationCardinality": {
                "type": "estimate",
                "value": 100022186,
                "dataset": "https://fragments.dbpedia.org/2016-04/en?predicate=http%3A%2F%2Fwww.w3.org%2F1999%2F02%2F22-rdf-syntax-ns%23type"
              },
              "bindOrder": "depth-first",
              "cardinalities": [
                {
                  "type": "estimate",
                  "value": 1040358853,
                  "dataset": "https://fragments.dbpedia.org/2016-04/en"
                },
                {
                  "type": "estimate",
                  "value": 100022186,
                  "dataset": "https://fragments.dbpedia.org/2016-04/en?predicate=http%3A%2F%2Fwww.w3.org%2F1999%2F02%2F22-rdf-syntax-ns%23type"
                }
              ],
              "joinCoefficients": {
                "iterations": 6404592831613.728,
                "persistedItems": 0,
                "blockingItems": 0,
                "requestTime": 8902477556686.99
              },
              "childrenCompact": [
                {
                  "occurrences": 100,
                  "firstOccurrence": {
                    "logical": "pattern",
                    "source": "QuerySourceHypermedia(https://fragments.dbpedia.org/2016-04/en)(SkolemID:0)",
                    "pattern": "http://commons.wikimedia.org/wiki/Special:FilePath/!!!善福寺.JPG ?p http://dbpedia.org/ontology/Image"
                  }
                }
              ]
            }
          ]
        }
      ]
    }
  ]
}

Explaining in JavaScript

If you have installed Comunica SPARQL in a JavaScript app, then you can invoke the explain method on your query engine with a certain explain mode.

Below, you can see examples on how the different explain modes can be invoked.

Explain parsed in JavaScript

console.log(await engine.explain(`SELECT * WHERE {
      ?s ?p ?o.
    }`, {
  sources: [ 'https://www.rubensworks.net/' ],
}, 'parsed'));

/*
Will print:

{
  explain: true,
  type: 'parsed',
  data: {
    input: {
      patterns: [
        factory.createPattern(
          DF.variable('s'),
          DF.variable('p'),
          DF.variable('o'),
        ),
      ],
      type: 'bgp',
    },
    type: 'project',
    variables: [
      DF.variable('s'),
      DF.variable('p'),
      DF.variable('o'),
    ],
  },
}

with DF being an RDF data factory, and factory being a SPARQL algebra factory.
 */

Explain logical in JavaScript

console.log(await engine.explain(`SELECT * WHERE {
      ?s ?p ?o.
    }`, {
  sources: [ 'https://www.rubensworks.net/' ],
}, 'logical'));

/*
Will print:

{
  explain: true,
  type: 'logical',
  data: {
    input: {
      input: [
        factory.createPattern(
          DF.variable('s'),
          DF.variable('p'),
          DF.variable('o'),
        ),
      ],
      type: 'join',
    },
    type: 'project',
    variables: [
      DF.variable('s'),
      DF.variable('p'),
      DF.variable('o'),
    ],
  },
}

with DF being an RDF data factory, and factory being a SPARQL algebra factory.
 */

Explain physical in JavaScript

console.log(await engine.explain(`SELECT * WHERE {
      ?s ?p ?o.
    }`, {
  sources: [ 'https://www.rubensworks.net/' ],
}, 'physical'));

/*
Will print:

{
  explain: true,
  type: 'physical',
  data: `slice
  project (o,p,s)
    join
      join-inner(bind) bindOperation:(?s http://www.w3.org/1999/02/22-rdf-syntax-ns#type ?o) bindCardEst:~100,022,186
        pattern (http://commons.wikimedia.org/wiki/Special:FilePath/!!!善福寺.JPG ?p http://dbpedia.org/ontology/Image) src:0 compacted-occurrences:100

sources:
  0: QuerySourceHypermedia(https://fragments.dbpedia.org/2016-04/en)(SkolemID:0)
`,
}
 */