Comunica Bus RDF Resolve Hypermedia Links Queue

A bus for creating ILinkQueue instances, which enable different strategies for queueing links.

This module is part of the Comunica framework.

Install

$ yarn add @comunica/bus-rdf-resolve-hypermedia-links-queue

Usage

Bus usage

• Context: "https://linkedsoftwaredependencies.org/bundles/npm/@comunica/bus-rdf-resolve-hypermedia-links-queue/^4.0.0/components/context.jsonld"
• Bus name: ActorRdfResolveHypermediaLinksQueue:_default_bus

Creating actors on this bus

Actors extending ActorRdfResolveHypermediaLinksQueue are automatically subscribed to this bus.

Changelog

All notable changes to this project will be documented in this file.

v4.2.0 - 2025-04-29

Added

• Add HTTP rate limit actor
• Add workaround for DBPedia SPARQL HTTP status 405 with Retry-After
• Add VoID metadata extract actor and cardinality estimation for SPARQL sources
• Allow SPARQL source actor to skip COUNTs for large queries
• Allow sending COUNT queries to be disabled for SPARQL endpoints
• Allow forcing SPARQL source type from config
• Add join sorter based on join selectivities
• Report the url when a fetch fails
• Add distinctConstruct key to IQueryContextCommon

Changed

• Detect SPARQL endpoints with trailing slashes, Closes #1529
• Skip COUNT queries to singular SPARQL source
• Skip unnecessary SPARQL SD requests for a single source
• Update to rdf-streaming-store v2
• Try action.mediaType before doing heuristics for media type
• Avoid unnecessary parse attempts after dereferencing fails
• Avoid printing all metadata in join logging (#1509)
• Avoid creation of intermediary triple objects for in-memory stores

Fixed

• Fix variable names in ActorOptimizeQueryOperationPruneEmptySourceOperations
• Fix incorrect id of actor-query-source-hypermedia-identify-sparql
• Fix GraphQL queries being sent to SPARQL endpoints, Closes #1525
• Fix HTTP request count tracking, Closes #1526
• Fix bad plans sometimes being chosen due to requestTime in files
• Fix regressions of initialBindings with BIND clauses
• Fix queries with complex property paths not terminating
• Avoid flattening joins with input-specific metadata (#1522)
• Fix broken Dockerfile for query-sparql

v4.1.0 - 2025-02-11

v4.1.0 - 2025-02-11

Added

• Add support for x and q regex flags, Closes #1254
• Add ignoreOrder param to toEqualBindings jest helpers
• Add bus to transform intermediary streams
• Extend SPARQL Service Description parsing

Changed

• Performance improvements:
  • Allow Bind Join more for local data sources
  • Shorten code path on empty join operations
  • Remove uncommon variables handling in join entry sort
• Improve warning message on expressions error in left join
• Update dependency htmlparser2 to v10 (#1472)
• Update dependency uuid to v11

Fixed

• Fix link traversal with multiple sources not always working
• Fix floating point error for very low cardinalities
• Fix sd:defaultGraph parsing with QPF endpoints
• Fix query stopping when a triple pattern has estimated cardinality zero
• Fix incorrect test method of ActorQuerySourceIdentifyHypermediaAnnotateSource

v4.0.2 - 2024-10-16

Changed

• Document that algebra inputs should not contain blank nodes, Closes #1234

Fixed

• Fix some zero-or-more path queries producing empty results, Closes #1449

v4.0.1 - 2024-10-15

BREAKING CHANGES

The following breaking changes will only impact users of Node.js < 18 and users that develop custom components or engines with Comunica.

• Remove backwards-compat for Node.js < 18
  • Remove cross-fetch and abort-controller polyfill as unnecessary
  • Update fetch-sparql-endpoint to version 5
  • Remove Node 14 backwards-compat in stats writer
• API changes
  • Refactor Actor.test to not rely on Promise rejections
    • All Actor.test methods must be changed to use passTest or failTest instead of promise rejections, for better performance.
  • Use variable.canBeUndef in metadata instead of canContainUndefs
  • Remove Actor.initialize and .deinitalize
  • Rename noCache CLI argument to invalidateCache
  • Remove hash collisions flag from hash buses
  • Return 32-bit numbers in hash bus based on specific variables
  • Allow inserts and deletes to execute in a single request
• Renamed or changed actors
  • Split expression functions into separate actors on dedicated bus
  • Split aggregators into separate actors on dedicated bus
  • Refactor HTTP fetch and retry logic
    • Retries of failed requests has been improved
    • The User-Agent header has been improved
    • A better approach to detect if Comunica is in a browser environment, compatible with Node.js, Deno, and Bun.
  • Remove SHA1 hash quads actor in favor of Murmur3
  • Remove SHA1 hash actor in favor of Murmur3
  • Abstract actor-rdf-join-minus-hash-undef into actor-rdf-join-minus-hash
• Move utilities
  • Add utils-query-operation and utils-iterator
    • These new packages contain helper functions that previously existed in bus-query-operation and bus-rdf-join.
  • Rename util packages to 'util-' scope
    • data-factory -> utils-data-factory
    • bindings-factory -> utils-bindings-factory
    • jest -> utils-jest
    • metadata -> utils-metadata
  • Move ActorRdfJoin.hashNonClashing to bindingsToCompactString
  • Delegate all join hashing to hash bus
  • Move bindings index from optional join to utils package

Added

• Add default Accept-Encoding header with Brotli
• Add undef support to inner hash join actor
• Forward side data from test to run in actors
• Allow plugging in custom DataFactory through context, Closes #358

Changed

• Remove unneeded false default values on CLI boolean args
• Improve fragment selector shape handling
  • Make doesShapeAcceptOperation recurse into operations
  • Add dedicated wildcard operation type to FragmentSelectorShape
• Improve performance
  • Bump asyncjoin with hash join performance improvement
  • Only consider overlapping vars when testing undef in join actors
• Improve type inference in IActionContext, Closes #933
• Emit planning time from stats writer
• Make stats writer measure time from actual query start
• Improve error reporting when mediators fail

Fixed

• Fix union not setting canContainUndefs to true if child vars differ

v3.3.0 - 2024-10-08

Added

• Add statistics tracking

Changed

• Optimize Bindings merge logic

Fixed

• Fix source attribution for QPF and SPARQL endpoints (#1420)
• Fix internal cardinalities being wrong for SPARQL endpoints with VoID
• Define inner join actor orders for NaN cardinalities

v3.2.3 - 2024-08-22

Fixed

• Fix filter pushdown into patterns losing variable bindings

v3.2.2 - 2024-08-19

Changed

• Optimizations:
  • Splice federated left join with empty right-hand input
  • Add ClosableIterator that bypasses TransformIterator
  • Only enable filter pushdown if criteria are met
  • Push down simple equality filters into triple patterns
  • Support pushing filters into left joins
  • Split conjunctive filters before pushdown
  • Merge nested filters into conjunctive after pushdown

Fixed

• Fix empty optionals in federation producing empty results for full query
• Don't push down FILTER (NOT) EXISTS

v3.2.1 - 2024-08-07

Added

• Add support for deduplication of results in construct queries

Changed

• Remove projection variables missing warning
• Migrate parse-link-header to http-link-header
• Migrate stream-to-string and streamify-string
• Performance improvements:
  • Tweak default BindJoin.minMaxCardinalityRatio value for WatDiv-C1
  • Use Bind Join only if smallest much smaller than largest
  • Assign smallest stream as build side in Hash Join, #884
  • Force NLJ when join entries have no overlapping variables

Fixed

• Fix optimizer incorrectly considering some optionals empty
• Fix SLICE actor not working with ASK queries, Closes #1374
• Make projection variables missing error a warning instead
• Fix some unions in federated queries being pruned incorrectly, Closes #1397
• Avoid single-entry joins in join-connected optimizer, #1397
• Consider unions when determining if operation can produce undefs, #1406
• Fix undefined values from SPARQL endpoints being produced
• Fix unnecessary join being created when binding over filter
• Fix initialBindings failing when done over projected variables

v3.2.0 - 2024-07-05

Added

• Performance improvements:
  • Add optional hash join actor
• Add actors for bindings source attribution
• Add OPT+ operator

Changed

• Performance improvements:
  • Lower default ioWeight to tweak overall performance
  • Make optional hash and bind join only work with common variables
• Debugging and performance tracking:
  • Omit undefined join coefficients in debug logger
  • Emit timings and actual cardinalities in physical explain output
  • Make physical query plan explain output compact and human-readable
  • Compact Bind-Join operations in physical query explain output
  • Add BJ bindOperation to physical query explain output
  • Emit pattern source in physical query explain output

Fixed

• Fix undef sanity check for SPARQL endpoints sometimes going rogue
• Fix CONSTRUCT FROM not being sent to endpoints, Closes #1389
• Fix empty media type not falling back to extension handling
• Fixed the order of count operations influencing the result
• Fix minus-hash using incorrect hash function

v3.1.2 - 2024-05-30

Fixed

• Fix next page requests not being logged anymore
• Fix performance regression of TPF queries

Changed

• Improve performance:
  • Use AsyncIterator.map and .filter instead of expensive .transform
  • Improve performance of hashing during hash joins
  • Optimize usage of bindingsStream transform in PathNps actor
• Use event-emitter-promisify package
• Cleanup stream operations (#1349)
• Maintain backpressure in result serializers
• Maintain backpressure on internal actors
• Don't log if no sources are pruned
• Use cancel in wayback actor when available, Closes #1351

v3.1.1 - 2024-05-11

Fixed

• Fix queries over paged collections always being empty

v3.1.0 - 2024-05-11

Added

• Add @comunica/query-sparql-rdfjs-lite that is optimized for bundle size

Changed

• Migrate from readable-web-to-node-stream to readable-from-web
• Move validateHttpResponse helper to avoid interdependencies

Fixed

• Avoid race conditions during DELETE/INSERT operations
• Catch 404 of non-existing resource when fetching the selector shape (#1324)
• Always add preprocessed context to unidentified query sources (#1337)

v3.0.3 - 2024-04-12

Changed

• Add generic type to LinkQueueWrapper (#1322)

Fixed

• Fix broken LIMITs on CONSTRUCT queries on SPARQL endpoints, Closes #1319

v3.0.2 - 2024-04-10

Fixed

• Lower q-value of application/json in fetch requests

v3.0.1 - 2024-03-19

BREAKING CHANGES

• Rename sparqlee-specific package names
  • @comunica/actor-query-operation-filter-sparqlee -> @comunica/actor-query-operation-filter
  • @comunica/actor-query-operation-orderby-sparqlee -> @comunica/actor-query-operation-orderby
• Generalize sources to accept all query operations
  • The rdf-resolve-quad-pattern bus has been removed in favor of the new query-source-identify bus. This bus enables identification of IQuerySource's which can accept any kind of query operation, not just quad patterns like before.
  • brTPF support is added.
  • Bound joins are supported: pushing bindings into sources (FedX)
  • Several context-preprocess actors are added due to this, which offer:
    • Grouping operations into sources during federated querying (FedX)
    • Pushing down filters into sources
    • Skolemization of blank nodes
• Move sequential query process steps to separate bus and actor
• Move context shortcut handling to preprocess actor

Added

• Allow Bindings to have contexts
• Add timeout for sparql endpoint count queries
• Process source contexts to support shortcuts
• Add option to disable caching (#1297)
• Allow sources to be annotated with traverse flag

Fixed

• Fix GROUP_CONCAT losing languages if equal

Changed

• Update to asynciterator 3.9.0 to make results async iterable
• Ensure QueryEngine instances are fully independent
• Update to Components.js v6

v2.10.2 - 2024-01-09

Fixed

• Migrate to @smessie/readable-web-to-node-stream

v2.10.1 - 2023-11-29

Fixed

• Fix updates with body streams failing in Node >= 18
• Generalize fromBindings in BindingsFactory

v2.10.0 - 2023-10-27

Added

• Convert raw source contexts to ActionContexts
• Support relative and absolute paths for COMUNICA_CONFIG

Changed

• Log expression error messages
• Print log levels in different colors
• Update readable-stream, yargs, lerna, typescript and eslint (#1271)

Fixed

• Fix wrong cardinality for zero-or-one paths with one variable
• Support building with esModuleInterop TS flag
• Skip calling endsWith on mediaType when mediaType is undefined

v2.9.0 - 2023-09-07

Changed

• Early close/destroy in hypermedia iterator when aggregatedStore is unused
• Update dependency uuid to v9 (#1247)
• Migrate sparqlee into Comunica

Fixed

• Enable SPARQL-star for update queries to endpoints

v2.8.3 - 2023-08-21

Changed

• Throw when doing BIND on a pre-bound variable, Closes #1229
• Include final SPARQL endpoint URL in debug logger

Fixed

• Fix hypermedia errors being voided, Closes #1241
• Fix some SPARQL endpoint errors being voided
• Fix missing end event for empty queries with limit
• Fix process halt when re-executing query over erroring source

v2.8.2 - 2023-08-10

Fixed

• Fix aggregated store not containing first source after query re-exec

v2.8.1 - 2023-07-14

Fixed

• Allow multi join actors to handle undefs, Closes #1231
• Fix process halting on many TPF sources

v2.8.0 - 2023-07-04

Added

• Add support for RDF-star and SPARQL-star
  • Support quoted triples in result serializers
  • Add support for processing quoted triples

Changed

• Update to rdf-store-stream v2
• Update dependency lru-cache to v10
• Update to TypeScript 5.x

v2.7.1 - 2023-05-31

Changed

• Update dependency htmlparser2 to v9 (#1211)
• Simplify stringSource actor implementation

Fixed

• Fix existence hook not provided error with OPTIONAL, Closes #1029
• Fix stringSource syntax errors not being caught, Closes #1192
• Fix bind join illegally being used over FILTERs
• Fix NaN selectivity for non-pattern operations

v2.7.0 - 2023-05-24

Added

• Support durations, dates, and times in filters
• Add support for ZeroOrOne path expressions with 2 variables

Changed

• Improving Web support
  • Use process shim (#1186)
• Improve performance with SPARQL endpoints
  • Improve metadata and lazyness in SPARQL hypermedia actor
  • Fix SPARQL endpoint operator always having canContainUndefs: true
  • Fix keepalive not being true in fetch-only environments
  • Cache count queries over SPARQL endpoints
• Enable incremental metadata updates
• Update dependency canonicalize to v2
• Make Bindings#merge and mergeWith accept more generic RDF.Bindings

Fixed

• Fix blank node scoping in CONSTRUCT queries, Closes #1190
• Add missing baseIRI for shaclc
• Destroy unused iterators

v2.6.10 - 2023-03-10

Fixed

• Add missing shaclc dependency in rdfjs engine

v2.6.9 - 2023-03-08

Changed

• Update to Node.js typings v18

Fixed

• Bump rdf-streaming-store with another fix for stream.push() after EOF
• Use globalThis instead of global
• Fix OPTIONAL failing to emit FILTERed results

v2.6.8 - 2023-03-06

Changed

• Remove deprecated calls to lru-cache

Fixed

• Bump rdf-streaming-store with fix for stream.push() after EOF
• Fix aggregated store not ending when linked iterator is destroyed
• Scope aggregated stores in the query context

v2.6.7 - 2023-02-15

Added

• Allow minimum sources check on CLI to be disabled
• Allow generic context type in QueryEngine to omit fields

v2.6.6 - 2023-02-08

Fixed

• Prefer piping in serializers to maintain backpressure, #759

Changed

• Update dependency lru-cache to v7
• Don't lookup default graph when none available in QPF

v2.6.5 - 2023-02-07

Fixed

• Fix group operator only returning stream after completion
• Fix incorrect caching of QPF requests on default graph

v2.6.4 - 2023-02-02

Fixed

• Fix aggregated hypermedia store not closing with LIMIT

v2.6.3 - 2023-02-01

Fixed

• Fix aggregated hypermedia store halting if initial stream is destroyed

v2.6.2 - 2023-01-31

Fixed

• Fix shaclc parser not being able to pipe

v2.6.1 - 2023-01-30

Fixed

• Fix aggregated hypermedia store halting if initial stream not fully consumed
• Cache parsed sources in string source actor

v2.6.0 - 2023-01-26

Added

• Add SHACLC parser and serializer
• Allow hypermedia actor to aggregate link queue in an index

Changed

• Use sparqlee's new type system

Fixed

• Fix invalid baseIRI when querying local files
• Fix errors in Node 19

v2.5.2 - 2022-11-18

Fixed

• Fix broken bindingsStreamToGraphQL export
• Fix incorrect bindingsStreamToGraphQl return type

v2.5.1 - 2022-11-16

Fixed

• Add missing dependencies in file and rdfjs engines
• Rename wayback actor in config, Closes #1086
• Add localizeBlankNodes to IQueryContextCommon
• Fix incorrect files entry in data-factory
• Add recoverBrokenLinks to IQueryContextCommon

v2.5.0 - 2022-11-09

Added

• Enable engines to override context shortcuts and typings
• Support string-based data sources, Closes #639
• Add support for HTTP retries
• Add wayback HTTP actor

Changed

• Add USER to Dockerfile
• Make QueryOperationAsk close the BindingsStream earlier
• Expose localizeBlankNodes on the CLI

Fixed

• Bump to asynciterator 3.8 with ESM packaging fix, Closes #1097
• Include source map files in packed files
• Fix invalid blank node skolemization in DESCRIBE queries

v2.4.3 - 2022-09-06

Fixed

• Add workaround for INSERT queries on Solid pods
• Add polyfill for Buffer in webpack

v2.4.2 - 2022-08-26

Fixed

• Fix typings issue in ActorDereferenceParse#handleDereferenceStreamErrors

v2.4.1 - 2022-08-24

Fixed

• Make global process override conditional

v2.4.0 - 2022-08-24

Added

• Add logging on timeout handlers in HTTP service
• Emit metadata in SPARQL/JSON serializer
• Enable context overriding when running SPARQL endpoint

Changed

• Move away from Node.js built-ins to avoid polyfilling in tools such as Webpack
  • Replace setImmediate by setTimeout
  • Use performance.now instead of process.hrtime
  • Avoid using htmlparser2 stream feature
  • Remove xml dependencies
  • Remove dependency on web-streams-node
• Performance improvements:
  • Update to AsyncIterator 3.7.0
  • Rewrite */+ property paths with proper backpressure handling

Fixed

• Fix propagation of iterator destroying in various places
• Fix blocking operators delaying timeout start in HTTP service
• Fix halting when error occurs in extend operation
• Avoid applying @babel/polyfill on the output code
• Close connections on uncaught exceptions in HTTP service workers
• Kill HTTP service master process if worker was forcefully killed

v2.3.0 - 2022-06-29

Added

• Add httpTimeout option to timeout on HTTP requests
• Add httpTimeoutOnBody option
• Add options to run using union default graph semantics
• Add freshWorker option to HTTP service

Changed

• Improve warning messages in extend actor
• Properly sort named nodes, blank nodes and unbounded values

Fixed

• Fix race conditions when timing out in HTTP server
• Fix responses not always being sent after HTTP server timeout
• Fix closing of streams not always propagating
• Fix N3.js parser not receiving parsing media type
• Fix physical query plan crash when multi-smallest inner join is used

v2.2.1 - 2022-04-13

Fixed

• Fix request with body in browser not working

v2.2.0 - 2022-04-12

Added

• Add getSafe to ActionContext and bus convenience interfaces
• Export deskolimisation functions (#974)

Changed

• Require minimum @rdfjs/types version (#948)
• Update to arrayify-stream 2
• Update to immutable.js 4
• Add missing deps in bus-dereference

Fixed

• Fix invalid JSON in a config file
• Fix hash joins not working on queries with limits
• Fix crash for requests with string body, Closes #969

v2.1.0 - 2022-03-09

Added

• Allow links to be followed in parallel

v2.0.6 - 2022-03-07

Fixed

• Remove original query string from context for initialBindings, Closes #941

v2.0.5 - 2022-03-04

Fixed

• Fix unhandled parse errors, Closes #872

v2.0.4 - 2022-03-04

Fixed

• Disable fetch keepalive when sending a body

v2.0.3 - 2022-03-02

Fixed

• Bump deps to avoid old transitive Comunica deps

v2.0.2 - 2022-03-02

Fixed

• Fix extension-based media type identification not working
• Fix invalid Dockerfile

v2.0.1 - 2022-03-02

BREAKING CHANGES

• Set minimum Node version to 14
  • Update tsconfig to es2020
• Public API
  • Improve names of query result interfaces
  • Implement new RDF/JS Bindings and BindingsFactory
  • Rename update results to void results
  • Implement RDF.Queryable in IQueryEngine
  • Make query context type-safe
  • Represent query formats as RDF.QueryFormat
  • Implement RDF.SparqlQueryable in IQueryEngine
• Internal API
  • Make metadata in query operations mandatory and strictly typed
  • Add Mediate utility type, Closes #902
  • Make ActionContext type-safe
  • Make context object required in actions
  • Use Headers rather than Record<string, string>
  • Rename IQueryableResult to IQueryOperationResult
  • Make IMediatorTypeJoinCoefficients implement RDF.QueryOperationCost
  • Migrate metadata interface to RDF/JS metadata types
  • Move variables field into metadata
  • Make boolean and void results lazy
• Remove deprecated features
  • Remove deprecated string variant in links bus output
  • Make links queue mediator in hypermedia resolver mandatory
  • Remove backwards-compat for source and hypermedia tagged sources
  • Remove deprecated exports
  • Remove unused init actors
  • Remove deprecated packages
  • Remove inefficient query-based metadata extractors
  • Remove utils-datasource package
  • Remove bindings and quads methods in favor of toArray on AsyncIterator
• Restructure packages
  • Move index files to lib directory
  • Rename actor-rdf-dereference-http-parse to actor-rdf-dereference-http
  • Move peerDependencies to dependencies
  • Remove unnecessary entries from package.json files
  • Rename bus-sparql-serialize to bus-query-result-serialize
  • Rename bus-sparql-parse to bus-query-parse
  • Rename actor-http-node-fetch to actor-http-fetch
  • Decouple configs and query engines into separate packages
  • Add generic dereference bus
  • Move sorting of join entries to dedicated bus
  • Remove args_ prefixes in config files when possible
• Refactor join operations:
  • Add logical join type field to join action
  • Add actor to cluster connected join entries
  • Consider multiple coefficients during join mediation
  • Add optimize actor to convert BGPs to joins
  • Add cardinality metadata to all property path actors
  • Rename totalItems metadata to cardinality
  • Remove BGP actors in favor of new join actors
  • Add BGP actor that delegates to join bus
  • Add Bind-Join actor
  • Add bus for providing join selectivities
  • Add variable counting join cardinality heuristic
  • Add minus query operation actor that delegates to join bus
  • Add left join query operation actor that delegates to join bus
  • Add hash-based minus join actor that does not handle undefs
  • Add hash-based minus join actor that handles undefs
  • Add bind-based optional join actor
  • Add nested loop-based optional join actor
  • Change BGP-specific context entries to more general join entries
  • Include original operations into join input interface
  • Migrate to multi-join-based datastructures in sparqlalgebrajs
  • Rename plain join actors to inner join actors
• Other
  • Deskolemise blank nodes before updates, Closes #911

Added

• Public API
  • Allow query plan to be printed
  • Add bindings-factory package
  • Allow full stack trace to be printed on CLI via --showStackTrace
  • Add dedicated explain method on IQueryEngine
• Internal API
  • Expose convenience types in bus packages for simpler actors impls
  • Add hash provider bus
  • Allow packager to output list of dependencies for a config
  • An optional parameter to MediatorCombinePipeline to filter erroring actors
  • Add option to enable ordering strategy in MediatorCombinePipeline
  • Add setSafe method to IActionContext

Changed

• Automatically generate components/ directories:
  • Remove components directory from git
  • Generate components in bulk
• Move index files to lib directory
• Update to sparqlalgebrajs 4
• Mark parsed quad stream as Readable
• Send original SPARQL query string to endpoints
• Update to Webpack 5
• Add sideEffects: false to package.json files for treeshaking
• Prefer joins with early results for queries with limit

v1.22.3 - 2021-10-19

Fixed

• Fix CORS issues when querying in Firefox
• Fix media type overriding not working for HTTP requests
• Fix source type detection not always working in SERVICE

v1.22.2 - 2021-09-30

Changed

• Include HTTP response in update error messages
• Allow patchSparqlUpdate destinations to not exist when forced
• Extract ms-author-via header for SPARQL updates, to support older Solid servers
• Pass optional headers in dereference errors

Fixed

• Fix fetch actor ignoring request bodies in Node.js
• Add temporary workaround for loss of Headers on Solid auth requests

v1.22.1 - 2021-09-22

Fixed

• Fix rare stream interruption before end

v1.22.0 - 2021-08-30

Added

• Add support for SPARQL extension functions
• Allow custom fetch function to be provided via context
• Support AbortControllers in native http actor
• Report FILTER errors as warnings in the logger, Closes #767
• Add SPARQL endpoint-based hypermedia update actor
• Allow resource creation via HTTP PUT:
  • Add RDF update hypermedia actor for LDP PUT destinations
  • Configure RDF update hypermedia actor for LDP PUT destinations
  • Allow RDF dereference actors to return on non-200 responses
  • Add metadata extractor for HTTP Accept-Put header
  • Add metadata extractor for HTTP Allow header

Changed

• Refactor CLI arguments handling using yargs
• Optimize node-fetch HTTP actor and set as default
• Include response body in dereference errors
• Bump fetch-sparql-endpoint with improved error reporting for SPARQL endpoints
• Update dependency htmlparser2 to v7 (#851)
• Migrate to @rdfjs/types
• Bump to sparqlalgebrajs 3
• Notify downstream clients on worker timeout
• Bump fetch-sparql-endpoint with AbortControllers
• Make HTTP accept headers deterministic
• Add URL-based heuristic to determine SPARQL endpoints to top-level actor

Fixed

• Update to AsyncIterator 3.2.0 to fix memory issue
• Fix http proxy not working with immutable node-fetch response
• Fix invalid hash implementation for hash-join
• Fix node-fetch actor not supporting body.cancel
• Do not reset sparql.js blank-node counter (Closes #836)
• Fix SPARQL-based update actors not being able to handle quads

v1.21.3 - 2021-06-18

Fixed

• Fix POST requests with body not working on browsers

v1.21.2 - 2021-06-14

Changed

• Interact with SPARQL endpoints via HTTP POST, Closes #814
• Update dependency @types/node to v14
• Update jsonld-streaming-parser to 2.3.2

v1.21.1 - 2021-04-27

Republish due to npm publishing failure of 1.21.0.

v1.21.0 - 2021-04-27

Added

• Enable hypermedia-driven updates
  • Pass headers to metadata extraction actors
  • Add metadata extractor for detecting SPARQL Update Patch servers
  • Configure hypermedia-based update actors
  • Add actor for handling servers supporting SPARQL Update patches
  • Add update actor for delegating to hypermedia bus
  • Add bus for hypermedia-based update actors
• Support HTTP requests with bodies
• Allow context modification in optimize-query-operation bus

Changed

• Move IQuadDestination to separate file
• Allow multiple graphs to be created/deleted in IQuadDestination
• Use --to option instead of -d to mark destination on CLI
• Allow any parser options to be passed to actor-rdf-parse-jsonld

Fixed

• Emit non-zero exit code on CLI errors, Closes #805
• Remove unnecessary dependencies on utils-datasource to mute Yarn warnings
• Fix types dependencies not being actual dependencies
• Fix unable to federate over multiple GraphDB instances

v1.20.0 - 2021-03-30

Added

• Add support for SPARQL Update queries
  • Add update actors to all init engines
  • Support update queries in SPARQL endpoint
  • Support update results in simple serializer
  • Add readOnly context entry that makes update actors fail
  • Allow single source in context to be inherited as destination
  • Add add, copy, and move update operation actors
  • Add create update operation
  • Add clear and drop update operations
  • Configure load update actor
  • Add load update query operation actor
  • Add fallback rdf-dereference actor
  • Run SPARQL Update test suite
  • Enable passing destination via the CLI
  • Configure INSERT and DELETE actors
  • Add composite update operation actor
  • Add INSERT/DELETE operation actor
  • Add RDF update actor that handles RDF/JS stores
  • Add bus for handling quad updates
  • Add update query operation output type
• Enable worker threads in SPARQL endpoints
• Delegate link queue creation to the new link queue bus

Changed

• Move public query engine interfaces into @comunica/types
• Move all context keys to '@comunica/context-entries'
• Keep track of parent link when pushing into link queue
• Save original query in context

Fixed

• Enable blank node correlation across results, Closes #795
• Fix FROM/FROM NAMED not always restricting views
• Bump asyncjoin with close on undefined fix, Closes #791
• Fix process hanging if a source parsing action rejects, Closes #786
• Fix hypermedia quad pattern resolver not skipping links to itself

v1.19.2 - 2021-02-02

Changed

• Remove disabling of stacktraces when in production

Fixed

• Fix unhandled promise rejection on server errors

v1.19.1 - 2021-01-22

Fixed

• Fix logger not always producing output, Closes #774

v1.19.0 - 2021-01-15

Changed

• Update to Components.js 4
• Update dependency htmlparser2 to v6

v1.18.1 - 2020-12-01

Fixed

• Fix HTTP service not handling conneg correctly with wildcards
• Remove unsafe usages of process
• Fix minor (non-breaking) issues with configs

v1.18.0 - 2020-11-02

Fixed

• Fix eager accept header boundary bug, Closes #534
• Fix incorrect return type for bindings(), Closes #742
• Add missing query result interface exports

Added

• Add Microdata to RDF parser
• Add actor whitelist to LoggerPretty

Changed

• Use fixed hashing functions
• Update sparqlee to achieve smaller bundle sizes
• Bump sparqlee with improved XPath and Unicode support
• Allow quad streams on followed links to be transformed
• Allow hypermedia links to define custom context entries
• Allow JSON-LD strict mode to be enabled via context
• Allow custom JSON-LD document loader to be passed via context

v1.17.0 - 2020-09-25

Added

• Include request headers in logs, Closes #703
• Include user agent in node-fetch HTTP actor

Changed

• Update to asynciterator 3.0.3 to fix hanging queries
• Make metadata retrieval more robust, and fix errors where metadata is emitted before end event
• Make logger data argument lazy
• Tweak priorities to let endpoint default to application/json, Closes #662
• Increase default Turtle priority to 0.6, higher than RDF/XML
• Update to @types/rdf-js 4
• Migrate to rdf-data-factory

Fixed

• Update to JSON-LD context parser with schema.org fix
• Fix redirect not including query parameter

v1.16.2 - 2020-08-24

Fixed

• Fix native HTTP actor broken in browser

v1.16.1 - 2020-08-24

Fixed

• Fix incorrect JSON-LD error code on target script error

v1.16.0 - 2020-08-24

Added

• Implement missing property path cases
• Add HTTP basic authentication support, Closes #613
• Add TSV SPARQL results serializer, #220
• Add CSV SPARQL results serializer, #220

Fixed

• Update sparql parser to fix all syntax spec test failures, #287
• Fix network request without headers, Closes #700 (#701)
• Correctly handle aggregates without group
• Fix subquery aggregates halting the process
• Fix variable graph also matching with default graph
• Fix hypermedia actor failing on * property path queries
• Fix property path queries failing with named graphs
• Fix mainModulePath overriding not being possible in dynamic exec
• Fix process hanging after HTTP error
• Fix HTTP error events being emitted twice
• Fix targeted HTML script tags not throwing error if of unknown type
• Fix missing newline after --listformats

Changed

• Configure nestedloop join actor for handling streams with undefs
• Annotate bindingsStreams with canContainUndefs flag

v1.15.0 - 2020-08-13

Added

• Add convenience methods for getting query results as arrays
• Add option to include credentials in browsers
• Enable min/max over non-numerical literal, Closes #673

Changed

• Improve command signature of comunica-sparql-http
• Remove lodash dependencies, Closes #402

Fixed

• Fix error when parsing webpage with JSON script tag, Closes #707
• Update to asynciterator 3.0.1 to undo webpack hack
• Add missing newline after each pretty log line
• Fix incorrect hash function in symmetric hash join, Closes #608
• Bump to sparqlee 1.4.2 with strtswith and endswith fix

v1.14.0 - 2020-07-24

Added

• Enable cardinality estimates in RDFJS sources
• Redirect default HTTP index to /sparql, Closes #663
• Simplify how RDFJS sources can be passed

Changed

• Migrate to asynciterator to 3 to improve performance in browsers
• Update dependency lru-cache to v6
• Move relevant @types to dependencies, Closes #294
• Replace json-stable-stringify dependency with canonicalize, Closes #590

Fixed

• Fix floating promises
• Fix test failures on Node 10
• Use latest fetch API with cross-fetch to fix fetch-related compatibility issues
• Fix orderby not taking numerical types into account
• Fix invalid RDFJS actor IRI
• Fix newEngine in SPARQL init file exporting browser build, Closes #648
• Fix orderby not properly handing multiple comparators, Closes #598

v1.13.1 - 2020-06-11

Fixed

• Fix crash when using CONSTRUCT queries
• Fix unasigned variables in projection result

v1.13.0 - 2020-06-03

Fixed

• Modify JSON-LD document loader to consider redirects
• Fix table serializer producing inconsistent column orders, Closes #643
• Update sparqlalgebra to fix invalid COUNT queries to endpoints

Changed

• Enable stricter TypeScript compilation, Closes #628

v1.12.1 - 2020-04-27

Fixed

• Fix EXISTS in BIND failing, Closes #650
• Fix project operator removing scoped blank nodes, solid/query-ldflex#64

v1.12.0 - 2020-04-03

Changed

• Update to JSON-LD parser to 1.1

Added

• Handle JSON-LD via JSON extension types as well
• Pass HTTP headers to HTML script parsers
• Allow HTML scripts to be targeted by id via fragments
• Throw error on application/json without valid JSON-LD link header

Fixed

• Fix HTML parser not forwarding errors within listeners

v1.11.1 - 2020-04-02

Added

• Add Runner.collectActors method

Fixed

• Fix initialBindings not applied on full query

Changed

• Output number of HTTP requests in stats serializer
• Remove unneeded action argument in Setup.instantiateComponent

v1.11.0 - 2020-03-30

Added

• Add more efficient optional actor
• Expose SPARQL service description in HTTP tool, Closes #219

Changed

• Optimize default graph handling in QPF sources
• Scope blank nodes to each federated source

Fixed

• Fix application aborting on some optional queries
• Use xsd:dateTime format (ISO 8601) for NOW
• Fix several issues with SPARQL endpoints:
  • Fix crash when slicing construct queries
  • Fix only SELECT queries being handleable by SPARQL endpoints
  • Fix SPARQL endpoint detection failing when SD uses blank node subjects #619
  • Fix SPARQL endpoint detection failing for relative IRIs #619
  • Consider sources ending with '/sparql' as SPARQL endpoints, Closes #535

v1.10.0 - 2019-12-12

Added

• Add actors for joining more than two streams pair-wise
• Allow quad-pattern-level contexts to be defined

Changed

• Manually load sources for federated metadata
• Clarify dereferencing error message.

Fixed

• Fix unresolved promise rejections in SPARQL tree serialization
• Swap node-web-streams package with web-streams-node
• Remove unused jsonld dependency

v1.9.4 - 2019-10-21

Changed

• Bump to Components.js 3.3.0

v1.9.3 - 2019-10-16

Added

• Add lenient mode for HTTP and parsing errors
• Store currently processing operation in context
• Log requesting URLs in node-fetch HTTP actor

Changed

• Tweak priorities of default RDF media types
• Make hypermedia link following lazy
• Allow headers and method to be passed to RDF dereferencers
• Update to Sparqlalgebra.js 2.x.x

Fixed

• Add missing stream-to-string to actor-rdf-parse-jsonld

v1.9.2 - 2019-09-27

Added

• Plug HTTP bus into JSON-LD parser, Closes #478
• Detect JSON-LD contexts in link headers, Closes #486
• Throw error when multiple JSON-LD link headers to contexts were detected

Changed

• Emit missing metadata as null
• Remove deprecated util.inspect calls, Closes #488

Fixed

• Fix QPF sources never going beyond page 2
• Reimplement MINUS operator to handle variables according to spec
• Ensure unique blank nodes across different bindings
• Fix sequence path operator actor missing a join mediator
• Filter quads from quad patterns with shared variables
• Fix SPARQL BASE not propagating through operators
• Fix actor-init-sparql bin being broken, Closes #493
• Fix various small Components.js naming issues
• Update sparqlee to 1.2.0 to fix precision errors in basic arithmetics

v1.9.1 - 2019-07-30

Fixed

• Fix file source not emitting metadata, resulting in slow queries
• Fix crash due to missing version bump in rdf-terms

v1.9.0 - 2019-07-29

Added

• Add HTML script tag parser
• Plug RDFa parse into HTML parsing bus, Closes #138
• Add RDF RDFa parser for XML documents
• Add HTTP proxy actor, Closes #465
• Allow actors to be registered before other actors in buses
• Add abstract GraphQL-LD-based hypermedia extractor
• Add query-based metadata extractors, Closes #115
• Add BusIndexed that indexes actors by type

Fixed

• Fix broken docker images, Closes #469
• Fix hashing breaking with different data factories, #438
• Fix GraphQL-LD query results not singularizing
• Fix RDF dereferencing not taking into account relative response URLs

Changed

• Add RDFJS source actor to default SPARQL config
• Bump sparqlee to 1.1.0, Closes #477
• Allow auto-sources to be defined as strings instead of objects
• Add hypermedia-based SPARQL resolver using service descriptions
• Log identified source
• Improve error message on RDF dereference HTTP errors
• Allow combine union mediator to have an empty bus
• Pass headers to RDF parse actors
• Append '/' in accept header when shortened, Closes #471
• Deprecate triple predicate metadata identifier
• Add config option to allow metadata to also be emitted as data
• Add fallback metadata actor that sees everything as both meta and data
• Make primary topic actor fallback to 'all data is also metadata'
• Remove heuristical source identification
• Emit dataset id in Hydra controls extractor
• Rewrite quad pattern hypermedia resolver to be less restrictive
• Add hypermedia resolver for unknown sources
• Refactor QPF-specific hypermedia resolver, #432
• Remove lodash.map from mediators
• Improve performance of applyInitialBindings
• Index the query operation bus
• Remove unneeded string trimming in SPARQL serialization

v1.8.0 - 2019-06-13

Changed

• Update sparqlee to 1.0.0
• Bump to new GraphQL-LD version

v1.7.4 - 2019-05-29

Fixed

• Fix RDF source identifier joining content types incorrectly
• Allow JSON-LD context to appear out-of-order

v1.7.3 - 2019-05-09

Fixed

• Fix bnodes in CONSTRUCT not working, Closes #449
• Fix CONSTRUCT on empty WHERE clause producing no 1 result, Closes #448

v1.7.2 - 2019-05-03

Fixed

• Remove console.log in HTTP parse RDF dereference actor

v1.7.1 - 2019-05-03

Fixed

• Restrict accept headers to 128 characters in browsers to fix CORS issues
• Fix native HTTP actor not forwarding request errors, Closes #381

Changed

• Rescale default media type priorities
• Remove unneeded spaces and zeros in accept headers
• Add CORS headers for SPARQL interface

v1.7.0 - 2019-04-11

Added

• Allow expressions in EXISTS
• Add support for SPARQL aggragates
• Allow baseIRI to be defined for queries

Fixed

• Fix local file querying not working

v1.6.6 - 2019-04-03

Changed

• Remove jsonld.js in favor of streaming JSON-LD processing, Closes #384
• Use real-world endpoint examples in query help message
• Return error code when config compilation failed, Closes #394
• Make SPARQL endpoint identifier use GET instead of HEAD
• Lower default tree serialization priority

Fixed

• Fix bug in orderby due to multiple transforms on same stream, Closes #434
• Fix URL hashes not being stripped when doing HTTP requests, Closes #277
• Fix invalid default REDUCED cache size, Closes #376
• Fix HEAD requests on HTTP service not working, Closes #417
• Fix patterns with conflicting variables being created in federation

v1.6.5 - 2019-03-15

Added

• Emit bound pattern metadata in BGP subcontexts

Fixed

• Fix invalid expression actor test functions

v1.6.4 - 2019-03-05

Fixed

• Fix HTTP service imports

v1.6.3 - 2019-02-27

Changed

• Fix performance issue due to many Error instantiations
• Disable empty sources check in federated actor

v1.6.2 - 2019-02-26

Added

• Add option to invalidate cache before each query exec

Changed

• Improve help message of HTTP tool

v1.6.1 - 2019-02-25

Fixed

• Fix http tool not working with absolute config paths

v1.6.0 - 2019-02-22

Added

• Expose invalidateHttpCache method on SPARQL init
• Support source identification for single sources
• Pass parent metadata to BGP and pattern actions

Changed

• Update sparqlee to version 0.1.0, improves FILTER expressivity
• Use LRU cache in file resolver
• Abstract SPARQL HTTP service

Fixed

• Fix config compilation issues in file init actor
• Fix parser/serializer priority values not being applied
• Fix not all media types being determined in RDF dereference, Closes #318
• Fix invalid unscoped JSON-LD objects

v1.5.4 - 2019-01-31

Fixed

• Fix newEngineDynamic failing to produce results after multiple calls
• Fix http tool not working with absolute config paths

Changed

• Update jsonld to 1.5.0

v1.5.3 - 2019-01-24

Fixed

• Fix GraphQL-LD queries not producing results anymore
• Fix schema and skos prefix

v1.5.2 - 2019-01-17

Fixed

• Fix extend operator still depending on old sparqlee version

v1.5.1 - 2019-01-17

Fixed

• Fix tslib dependency issue in sparqlee, causing a crash at startup
• Fix JSON-LD serializer producing invalid JSON
• Check HTTP stream type when identifying source
• Remove unneeded filter peer dependency from orderby actor
• Remove unneeded @types/bluebird dependency, Closes #360

v1.5.0 - 2019-01-02

Added

• Add Filter operator (based on Sparqlee)
• Add Extend operator (based on Sparqlee)
• Update LeftJoin NestedLoop actor to use Sparqlee
• Add BGP join optimizer, Closes #90
• Add query operation optimize bus

v1.4.6 - 2018-12-11

Fixed

• Fix missing dependencies between new hypermedia actors

v1.4.5 - 2018-12-11

Fixed

• Fix blank nodes in data being seen as variables
• Fix queries with a variable graph only matching the default graph
• Fix missing base IRI in JSON-LD parser, Closes #342

Changed

• Split resolve quad pattern actor into two separate modules

v1.4.4 - 2018-11-13

Fixed

• Update to Components.js 3.2.0
• Upgrade asynciterator to v2.0.1

v1.4.3 - 2018-11-09

Changed

• Update to generic RDFJS typings
• Update N3 to 1.0.0, Closes #314

v1.4.2 - 2018-11-05

Fixed

• Pass baseIRI to RDF parsers, Closes #311

v1.4.1 - 2018-10-04

Fixed

• Make number mediator pick the first when all are invalid

v1.4.0 - 2018-10-03

Added

• Add MINUS operator actor
• Add preconfigured prefixes, Closes #80
• Also check extensions when no valid media type is present, Closes #286
• Make JSON-LD parser accept application/json
• Update sparqljson-parse to 1.4.0 with fix for Virtuoso, Closes #278

Fixed

• Fix federated actor stringifying sources, Closes #265
• Fix federation actor occasionally failing to produce results
• Fix crash when passing raw context with GraphQL, Closes #293
• Fallback to file-based sources when no content type is present, #286
• Fix parsing errors with datetime on resources without Memento support

1.3.0 - 2018-09-10

Added

• Add RDF/XML parser
• Enable serialization in actor-init-sparql-rdfjs

1.2.2 - 2018-09-05

Fixes

• Bump sparqljson-to-tree to 1.2.2

1.2.1 - 2018-09-05

Fixes

• Bump dependencies with invalid tslib dependency
• Add missing actor-query-operation-service dependencies
• Update peer dependency version

1.2.0 - 2018-09-05

Added

• Path expressions
• GraphQL-LD queries
• Memento support with the -d command line option
• SERVICE clause
• Dockerfile
• Browser script uploads to rdfjs/comunica-browser
• RDFJS sources
• Make browser version of the SPARQL init actor consistent with Node
• Allow initial bindings to be passed to query context
• Dereferenceable config instances
• Modularization of configs into config sets
• Action contexts
• Allow actions to be passively observed
• Optimize single SPARQL endpoint sources
• Set user agent for outgoing HTTP requests
• Add logger
• Add federated actor to default HDT init config

Fixes

• Allow pipeline mediator to have no actors in its bus
• Accept falsy terms in quad pattern RDF resolvers
• Fix falsy term crash when querying QPF interfaces
• Update asynciterator and follow-redirects to fix termination problems, Closes #144
• Make empty BGPs return a single empty bindings

1.1.2 - 2018-06-26

Added

• Created actor to read local files

Fixes

• Fix no client-side filtering of non-matching quads, Closes #152
• Update HDT to 2.x.x

1.1.1 - 2018-06-01

Fixes

• Fix config files missing from some init actor after publication

1.1.0 - 2018-06-01

Added

• Add --version flag to command, Closes #118
• Add Pipeline Combine mediator
• Add context preprocessing bus
• Add File RDF Source Identifier
• Add Hypermedia RDF Source identifier
• Add SPARQL RDF Source Identifier
• Add RDF Source Identifier actor as context preprocessor
• Add separate SPARQL init file actor
• Add separate SPARQL init HDT actor

Fixes

• Fix HTTP service not properly recognizing SPARQL POST content types
• Make spec-incompliant expression evaluator handle lang() more loosely
• Fix ActorHttpNative compilation errors on some platforms
• Don't emit null languages in basic expression evaluator
• Catch engine init errors in HTTP service
• Fix incorrect merge of FROM and FROM NAMED graphs
• Loosen handling of undefineds in expression evaluator
• Convert blank nodes when resolving SPARQL patterns, Closes #104
• Make N3 parser and serializer forward error events, #103
• Delegate stream errors from serializers to root, #103
• Make primary topic actor order-independent and take into account void:subset
• Fix paged RDF dereferencer not forwarding triples flag
• Bump Components.js to 2.2.0 to fix config compilation issues on Windows
• Prevent end event from firing before metadata

1.0.4 - 2018-03-22

Fixes

• Fix static newQuery function failing

1.0.3 - 2018-03-22

Fixes

• Catch failure to resolve federated metadata, #117

1.0.2 - 2018-03-22

Fixes

• Fix file quad resolver actor not included as dependency, Closes #120

1.0.1 - 2018-03-21

Fixes

• Add file quad pattern resolver to default config

[1.0.0] - 2018-03-19

• Initial release