EU Data Portal - Ismena website

info@ismena.com

EU Data Portal Connector

Connector Details

Type

Virtual machines, Single VM , BYOL

Runs on

Google Compute Engine

Last Update

24 October, 2024

Category

Overview

The EU Data Portal Connector is a Flask-based service that acts as a proxy to the EU Data Portal, providing seamless access to metadata search, dataset management, catalogue operations, vocabulary management, and other functionalities. This connector supports various endpoints for searching, retrieving feeds, managing datasets, catalogues, vocabularies, and more.

Integration Overview

This document details the integration points of the EU Data Portal Connector, their purposes, configuration requirements, and supported workflows. The connector forwards requests to the EU Data Portal API, handling authentication via API keys or Bearer tokens where required.

Supported Integration Action Points

searchGet
searchPost
scrollGet
datasetsAtom
datasetsRss
datasetRss
sitemapIndex
sitemap
listDatasets
createDataset
readDataset
createOrUpdateDataset
modifyDataset
deleteDataset
readDatasetRevision
readDatasetRevisionById
createDatasetRevision
deleteDatasetRevision
bulkCreateUpdateDatasets
listCatalogues
createCatalogue
readCatalogue
createOrUpdateCatalogue
modifyCatalogue
deleteCatalogue
readVocabularies
readVocabulary
createOrUpdateVocabulary
deleteVocabulary
createVocable
readVocableByResource
readVocable
createOrUpdateVocable
modifyVocable
deleteVocable
gazetteerAutocomplete
ckanPackageSearch
ckanPackageShow
listResourceTypes
listEditorialContent
readEditorialContent
createOrUpdateEditorialContent
deleteEditorialContent

Detailed Integration Documentation

Search Endpoints

Action	searchGet
Purpose	Performs a metadata search with flexible query parameters to filter and sort results.
Parameters	Required: None Optional: q: The search query (string, e.g., "cat"). filter: Filter by document type (string, deprecated). filters: Array of document types to filter (array of strings). facets: JSON string for facet filtering (e.g., {"catalog":"catalog-x"}). page: Page number of results (integer, default: 0). limit: Maximum datasets per page (integer, default: 10, max: 1000). fields: Fields to filter results (array of strings, e.g., ["title"]). minDate: Minimum date filter (string, format: date-time). maxDate: Maximum date filter (string, format: date-time). boost: Boost document fields (object, e.g., {"title":3.0}). globalAggregation: Count facets globally or locally (boolean, default: true). facetOperator: Combine facets with "AND" or "OR" (string, default: "OR"). facetGroupOperator: Combine facet groups with "AND" or "OR" (string, default: "AND"). bboxMinLon: Bounding box minimum longitude (float, -180 to 180). bboxMaxLon: Bounding box maximum longitude (float, -180 to 180). bboxMaxLat: Bounding box maximum latitude (float, -90 to 90). bboxMinLat: Bounding box minimum latitude (float, -90 to 90). sort: Sorting of results (array of strings, e.g., ["field+asc"]). filterDistributions: Filter distributions in datasets (boolean, default: false). aggregation: Enable facet aggregation (boolean, default: true). includes: Fields to include in response (array of strings). scroll: Enable scroll and return a scroll ID (boolean, default: false). minScoring: Minimum quality scoring value (integer). maxScoring: Maximum quality scoring value (integer). aggregationAllFields: Aggregate all facets (boolean, default: true). aggregationFields: Aggregate selected facets (array of strings). countryData: Filter by country catalogues (boolean). showScore: Include score in results (boolean). vocabulary: Filter by vocabulary ID (array of strings, effective with filter=vocabulary). resource: Filter by resource types (array of strings, effective with filters=resource). dataServices: Show datasets with access services (boolean). autocomplete: Enable autocomplete search via title (boolean, default: false). superCatalogue: Filter by super catalogue (string, deprecated). countOnly: Return only the count of results (boolean, default: false).
Configuration	Ensure the environment variable CONNECTOR_ENV_EU_DATA_PORTAL_API_KEY is set for API key authentication.
Output	Successful: JSON object with search results. Failure: JSON object with error details (e.g., {"success": false, "message": "Invalid request"}).
Workflow Example	Execute searchGet with q=cat and filters=dataset. Process the response to retrieve matching datasets. Use results for data discovery or analysis.

Search Post Endpoint

Action	searchPost
Purpose	Performs a metadata search using a JSON payload, allowing complex query structures.
Parameters	Required: JSON body conforming to the Query schema (see OpenAPI spec). Optional: None
Configuration	Requires API key in CONNECTOR_ENV_EU_DATA_PORTAL_API_KEY.
Output	Successful: JSON object with search results. Failure: JSON object with error details.
Workflow Example	Send a POST request with a JSON body containing query parameters. Process the response to retrieve search results. Use results for metadata analysis.

Scroll Get Endpoint

Action	scrollGet
Purpose	Retrieves large result sets using a scroll ID for pagination.
Parameters	Required: scrollId: The scroll ID from a previous search (string). Optional: None
Configuration	Requires API key in CONNECTOR_ENV_EU_DATA_PORTAL_API_KEY.
Output	Successful: JSON object with paginated results. Failure: JSON object with error details.
Workflow Example	Obtain a scrollId from a searchGet request with scroll=true. Execute scrollGet with the scrollId. Process additional results for large dataset retrieval.

Datasets Atom

Action	datasetsAtom
Purpose	Retrieves dataset feeds in Atom format for a specified language.
Parameters	Required: lang: Language code (string, e.g., "en", valid values: bg, es, cs, da, et, el, en, fr, ga, hr, it, lv, lt, hu, mt, nl, pl, pt, ro, sk, sl, fi, sv, no, de). Optional: Same as searchGet parameters (e.g., q, facets, page, etc.).
Configuration	Requires API key in CONNECTOR_ENV_EU_DATA_PORTAL_API_KEY.
Output	Successful: Atom feed with dataset information. Failure: JSON object with error details.
Workflow Example	Execute datasetsAtom with lang=en and q=hund. Process the Atom feed for dataset updates. Integrate into feed readers or applications.

Datasets RSS

Action	datasetsRss
Purpose	Retrieves dataset feeds in RSS format for a specified language.
Parameters	Required: lang: Language code (string, e.g., "en"). Optional: Same as searchGet parameters.
Configuration	Requires API key in CONNECTOR_ENV_EU_DATA_PORTAL_API_KEY.
Output	Successful: RSS feed with dataset information. Failure: JSON object with error details.
Workflow Example	Execute datasetsRss with lang=de and q=hund. Process the RSS feed for dataset updates. Use in RSS readers or applications.

Dataset RSS

Action	datasetRss
Purpose	Retrieves an RSS feed for a specific dataset by ID.
Parameters	Required: lang: Language code (string, e.g., "en"). id: Dataset ID (string). Optional: None
Configuration	Requires API key in CONNECTOR_ENV_EU_DATA_PORTAL_API_KEY.
Output	Successful: RSS feed for the specified dataset. Failure: JSON object with error details.
Workflow Example	Execute datasetRss with lang=en and id=dataset123. Process the RSS feed for dataset updates. Use for monitoring specific dataset changes.

Sitemap Index

Action	sitemapIndex
Purpose	Retrieves the sitemap index listing all available sitemaps.
Parameters	Required: None Optional: None
Configuration	Requires API key in CONNECTOR_ENV_EU_DATA_PORTAL_API_KEY.
Output	Successful: XML sitemap index. Failure: JSON object with error details.
Workflow Example	Execute sitemapIndex to retrieve the sitemap index. Parse the XML to identify sitemap IDs. Use for SEO or data crawling.

Sitemap Endpoints - Sitemap

Action	sitemap
Purpose	Retrieves a specific sitemap by ID in XML format.
Parameters	Required: id: Sitemap ID (integer, minimum: 1). Optional: None
Configuration	Requires API key in CONNECTOR_ENV_EU_DATA_PORTAL_API_KEY.
Output	Successful: XML sitemap. Failure: JSON object with error details.
Workflow Example	Execute sitemap with id=1. Parse the XML sitemap for dataset URLs. Use for data indexing or crawling.

Datasets Endpoints - List Datasets

Action	listDatasets
Purpose	Lists datasets for a specific catalogue.
Parameters	Required: catalogue: Catalogue ID (string). Optional: alias: Alias for the request (string, default: "read").
Configuration	Requires API key in CONNECTOR_ENV_EU_DATA_PORTAL_API_KEY.
Output	Successful: JSON array of dataset IDs. Failure: JSON object with error details (e.g., 404 for not found).
Workflow Example	Execute listDatasets with catalogue=catalog123. Process the array of dataset IDs. Use for catalogue exploration.

Action	createDataset
Purpose	Creates a new dataset (internal use, requires authentication).
Parameters	Required: JSON body conforming to the Dataset schema. Optional: synchronous: Synchronous execution (boolean, default: true).
Configuration	Requires API key in CONNECTOR_ENV_EU_DATA_PORTAL_API_KEY and Authorization header.
Output	Successful: JSON object with dataset ID (201 or 202 status). Failure: JSON object with error details.
Workflow Example	Send a POST request with a Dataset JSON body and Authorization header. Process the response to confirm dataset creation. Use for internal dataset management.

Action	readDataset
Purpose	Retrieves a specific dataset by ID.
Parameters	Required: id: Dataset ID (string). Optional: None
Configuration	Requires API key in CONNECTOR_ENV_EU_DATA_PORTAL_API_KEY and Authorization header.
Output	Successful: JSON object with dataset details. Failure: JSON object with error details (e.g., 404 for not found).
Workflow Example	Execute readDataset with id=dataset123. Process the dataset details. Use for dataset analysis or display.

Action	createOrUpdateDataset
Purpose	Creates or updates a dataset (internal use, requires authentication).
Parameters	Required: id: Dataset ID (string). JSON body conforming to the Dataset schema. Optional: synchronous: Synchronous execution (boolean, default: true).
Configuration	Requires API key in CONNECTOR_ENV_EU_DATA_PORTAL_API_KEY and Authorization header.
Output	Successful: JSON object with dataset ID (200, 201, or 202 status). Failure: JSON object with error details.
Workflow Example	Send a PUT request with a Dataset JSON body and Authorization header. Process the response to confirm dataset creation or update. Use for internal dataset management.

Action	modifyDataset
Purpose	Modifies an existing dataset (internal use, requires authentication).
Parameters	Required: id: Dataset ID (string). JSON body conforming to the Dataset schema. Optional: synchronous: Synchronous execution (boolean, default: true).
Configuration	Requires API key in CONNECTOR_ENV_EU_DATA_PORTAL_API_KEY and Authorization header.
Output	Successful: JSON object with dataset ID. Failure: JSON object with error details.
Workflow Example	Send a PATCH request with a Dataset JSON body and Authorization header. Process the response to confirm dataset modification. Use for internal dataset updates.

Action	deleteDataset
Purpose	Deletes a dataset (internal use, requires authentication).
Parameters	Required: id: Dataset ID (string). Optional: synchronous: Synchronous execution (boolean, default: true).
Configuration	Requires API key in CONNECTOR_ENV_EU_DATA_PORTAL_API_KEY and Authorization header.
Output	Successful: JSON object confirming deletion. Failure: JSON object with error details.
Workflow Example	Send a DELETE request with Authorization header. Process the response to confirm deletion. Use for internal dataset management.

Action	readDatasetRevision
Purpose	Retrieves a specific dataset revision.
Parameters	Required: id: Dataset ID (string). revision: Revision ID (string). Optional: None
Configuration	Requires API key in CONNECTOR_ENV_EU_DATA_PORTAL_API_KEY.
Output	Successful: JSON object with revision details. Failure: JSON object with error details.
Workflow Example	Execute readDatasetRevision with id=dataset123 and revision=rev1. Process the revision details. Use for version tracking.

Action	readDatasetRevisionById
Purpose	Retrieves a dataset revision by ID.
Parameters	Required: id: Dataset ID (string). revisionId: Revision ID (string). Optional: None
Configuration	Requires API key in CONNECTOR_ENV_EU_DATA_PORTAL_API_KEY.
Output	Successful: JSON object with revision details. Failure: JSON object with error details.
Workflow Example	Execute readDatasetRevisionById with id=dataset123 and revisionId=rev1. Process the revision details. Use for version tracking.

Action	createDatasetRevision
Purpose	Creates a dataset revision (internal use, requires authentication).
Parameters	Required: id: Dataset ID (string). revisionId: Revision ID (string). JSON body conforming to the Dataset schema. Optional: synchronous: Synchronous execution (boolean, default: true).
Configuration	Requires API key in CONNECTOR_ENV_EU_DATA_PORTAL_API_KEY and Authorization header.
Output	Successful: JSON object with revision ID. Failure: JSON object with error details.
Workflow Example	Send a PUT request with a Dataset JSON body and Authorization header. Process the response to confirm revision creation. Use for version management.

Action	deleteDatasetRevision
Purpose	Deletes a dataset revision (internal use, requires authentication).
Parameters	Required: id: Dataset ID (string). revisionId: Revision ID (string). Optional: synchronous: Synchronous execution (boolean, default: true).
Configuration	Requires API key in CONNECTOR_ENV_EU_DATA_PORTAL_API_KEY and Authorization header.
Output	Successful: JSON object confirming deletion. Failure: JSON object with error details.
Workflow Example	Send a DELETE request with Authorization header. Process the response to confirm deletion. Use for version management.

Action	bulkCreateUpdateDatasets
Purpose	Performs bulk creation or update of datasets (internal use, requires authentication).
Parameters	Required: JSON body conforming to the DatasetArray schema. Optional: synchronous: Synchronous execution (boolean, default: true).
Configuration	Requires API key in CONNECTOR_ENV_EU_DATA_PORTAL_API_KEY and Authorization header.
Output	Successful: JSON object with results for each dataset. Failure: JSON object with error details.
Workflow Example	Send a PUT request with a DatasetArray JSON body and Authorization header. Process the response to confirm bulk operations. Use for large-scale dataset management.

Catalogues Endpoints

Action	listCatalogues
Purpose	Lists all catalogues.
Parameters	Required: None Optional: alias: Alias for the request (string, default: "read").
Configuration	Requires API key in CONNECTOR_ENV_EU_DATA_PORTAL_API_KEY.
Output	Successful: JSON array of catalogue IDs. Failure: JSON object with error details.
Workflow Example	Execute listCatalogues. Process the array of catalogue IDs. Use for catalogue exploration.

Action	createCatalogue
Purpose	Creates a new catalogue (internal use, requires authentication).
Parameters	Required: JSON body conforming to the Catalogue schema. Optional: synchronous: Synchronous execution (boolean, default: true).
Configuration	Requires API key in CONNECTOR_ENV_EU_DATA_PORTAL_API_KEY and Authorization header.
Output	Successful: JSON object with catalogue ID. Failure: JSON object with error details.
Workflow Example	Send a POST request with a Catalogue JSON body and Authorization header. Process the response to confirm catalogue creation. Use for internal catalogue management.

Action	readCatalogue
Purpose	Retrieves a specific catalogue by ID.
Parameters	Required: id: Catalogue ID (string). Optional: None
Configuration	Requires API key in CONNECTOR_ENV_EU_DATA_PORTAL_API_KEY.
Output	Successful: JSON object with catalogue details. Failure: JSON object with error details.
Workflow Example	Execute readCatalogue with id=catalog123. Process the catalogue details. Use for catalogue analysis.

Action	createOrUpdateCatalogue
Purpose	Creates or updates a catalogue (internal use, requires authentication).
Parameters	Required: id: Catalogue ID (string). JSON body conforming to the Catalogue schema. Optional: synchronous: Synchronous execution (boolean, default: true).
Configuration	Requires API key in CONNECTOR_ENV_EU_DATA_PORTAL_API_KEY and Authorization header.
Output	Successful: JSON object with catalogue ID. Failure: JSON object with error details.
Workflow Example	Send a PUT request with a Catalogue JSON body and Authorization header. Process the response to confirm catalogue creation or update. Use for internal catalogue management.

Pricing

bill-and-calculator-2023-11-27-05-25-06-utc 1

Request a Quote

Support

For Technical support please contact us on

custom-connectors-support@isolutions.sa

A tech solution company dedicated to providing innovation thus empowering businesses to thrive in the digital age.

Register to IBM x iSolution Event