Skip to content

Proxy and Echo

Proxy

The proxy resource allows logged in users to fetch resources from other servers that do not support JSONP.

By default no conversion takes place, the content is fetched from the provided URL and relayed to the client. Redirects are followed.

Anonymous requests

A whitelist consisting of domain names can be specified in the configuration file to allow anonymous user to fetch data from certain domains. The host names are specified as list with the property name entrystore.proxy.whitelist.anonymous.

Whitelist example

entrystore.proxy.whitelist.anonymous.1=wikidata.org
entrystore.proxy.whitelist.anonymous.2=www.wikidata.org
entrystore.proxy.whitelist.anonymous.3=geonames.org

Local sources

Due to security reasons any requests (both authorized or anonymous) to hosts matching any of the following regular expressions are blocked:

  • localhost
  • (.+)\.local
  • ^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$
  • ^\d$

In addition, any hosts that do not resolve into a "regular" public Unicast address, this means that any local, site local, loopback, link local, and multicast addresses are blocked.

Whitelisting local sources

To bypass this restrictive blacklist for local requests, it is possible to specify specific local hostnames in a whitelist:

entrystore.proxy.whitelist.local.1=specialserver.local
entrystore.proxy.whitelist.local.2=...

Usage

GET {base}/proxy?url={url}[&fromFormat={format}&validate={format}]

Supported formats

  • europeana: Europeana ESE will be converted into Europeana EDM RDF/JSON.
  • rdfa: Embedded RDFa will be converted into RDF/JSON.
  • html: If available, the title, description and keywords are extracted from an HTML file and returned as a RDF/JSON graph with DC simple predicates.
  • Various RDF formats which are converted into RDF/JSON
    • application/rdf+xml
    • text/rdf+n3
    • application/x-turtle
    • application/trix
    • application/x-trig
  • application/lom+xml: Converts LOM/XML into RDF/JSON with a LOM/DCAM mapping.

Validation

The parameter validate can be used to supply a MIME type that should be used when checking whether the proxied content is valid RDF. The RDF formats that are supported for validation are found as part of the list above.

If the content is valid RDF the response body contains the RDF data and HTTP status is 200. If the validation failed the response contains the reason for the validation error together with an HTTP status 422.

Cannot be used together with fromFormat.

Examples

  • http://localhost:8181/proxy?url=http%3A%2F%2Foe.confolio.org%2Fscam%2F5%2Fmetadata%2F100%3Fformat%3Dapplication%2Flom%2Bxml&fromFormat=application/lom+xml
  • http://localhost:8181/proxy?url=http://dn.se&fromFormat=html
  • http://localhost:8181/proxy?url=http%3A%2F%2Fdbpedia.org%2Fdata%2FThe_Lord_of_the_Rings&fromFormat=application/rdf+xml
  • http://localhost:8181/proxy?url=http%3A%2F%2Fdbpedia.org%2Fdata%2FThe_Lord_of_the_Rings&validate=application/rdf+xml

Resource DELETE with proxy

For entries with type Link, Reference, or LinkReference, issuing DELETE /{context-id}/resource/{entry-id}?proxy=true not only removes the entry in EntryStore but also propagates a DELETE request to the entry's external resource URI. The typical use case is removing a RowStore dataset that is tracked as a Reference entry — ?proxy=true cleans up both the EntryStore-side reference and the RowStore-side dataset in one call.

SSRF protection

The proxied DELETE goes through the same URL-scheme allowlist (http, https only), private-network blacklist, DNS-resolution check, and DNS-pinned connection as the proxy GET endpoint above. Redirects are re-validated on every hop. Embedded credentials in the URL are rejected.

Whitelisting targets

Unlike the proxy GET whitelist (which is host-only), the DELETE whitelist is origin-based (scheme://host:port). Entries must specify the scheme and explicit port (or rely on the scheme's default port). This prevents an admin from accidentally trusting every port on a host with sensitive co-located services.

Example

entrystore.proxy.remote-resource.delete.whitelist.1=http://rowstore.internal:8282
entrystore.proxy.remote-resource.delete.whitelist.2=https://other-rowstore.example.com

http://x and http://x:80 are treated as the same origin (HTTP default port is normalised); the same holds for https:// and port 443.

Automatic trust for RowStore

When entrystore.rowstore.url is set, its origin is implicitly added to the DELETE trust set — no admin configuration needed to delete RowStore datasets in standard deployments. For split-host deployments where RowStore runs on a different origin than the one in entrystore.rowstore.url, add it explicitly to entrystore.proxy.remote-resource.delete.whitelist.

Echo

Usage

POST {base}/echo[?validate={format}]

Request body must be multipart form data.

The server replies with a <textarea> containing the contents of the submitted file (HTML escaped). The first line is always status:{statusCode} for easier client access to the HTTP status code from within a web browser.