diff --git a/docs/_config.yml b/docs/_config.yml
index c6fa6c6..612649a 100644
--- a/docs/_config.yml
+++ b/docs/_config.yml
@@ -1,5 +1,6 @@
title: "Maven-Based Data Management"
-remote_theme: pmarsceill/just-the-docs
+#remote_theme: pmarsceill/just-the-docs
+theme: just-the-docs
search_enabled: true
logo: "assets/images/sansa-logo-blue.png"
markdown: kramdown
diff --git a/docs/archive-code.md b/docs/archive-code.md
new file mode 100644
index 0000000..c45de45
--- /dev/null
+++ b/docs/archive-code.md
@@ -0,0 +1,50 @@
+## Archiving Resources (Scripts and Non-Java Code)
+
+Maven can be used bundle any folder up as a JAR archive by declaring that folder as a resource.
+
+For example, assume your project has the following structure:
+
+```
+|- README.md
+|- src
+| |- requirements.txt
+| |- run.sh
+| |- LICENSE.txt
+|
+|- LICENSE.txt # symlink to /src/LICENSE.txt
+```
+
+The following `pom.xml` can be used to package the content of the `src` folder up as a versioned JAR file:
+
+```xml
+
+
+ 4.0.0
+ org.aksw.myproject.mydataset.code
+ data-generation-script
+ 0.0.2
+
+
+
+ org.aksw.data.config
+ aksw-data-deployment
+ 0.0.8
+
+
+
+
+ jar
+
+
+
+
+
+ src
+
+
+
+
+```
+
diff --git a/docs/build-anything-with-docker.md b/docs/build-anything-with-docker.md
index faeecea..c61c335 100644
--- a/docs/build-anything-with-docker.md
+++ b/docs/build-anything-with-docker.md
@@ -13,10 +13,10 @@ In order to package data generation as a self-contained maven build, the two maj
### Example
-The setup at [examples/disasters-rdfizer](examples/disasters-rdfizer) contains the 2 folders
+The setup at [examples/disasters-rdfizer](https://github.com/AKSW/aksw-data-deployment/tree/develop/examples/disasters-rdfizer) contains the 2 folders:
-1. `code`
-2. `generator`
+1. `code`: This folder contains the actual data generation script and a `pom.xml` file in order to package and version it.
+2. `generator`: This is a `pom.xml` file that builds and runs a docker container from the `code`.
#### Code
diff --git a/examples/archive-downloads-generated/README.md b/examples/archive-downloads-generated/README.md
new file mode 100644
index 0000000..cb3715c
--- /dev/null
+++ b/examples/archive-downloads-generated/README.md
@@ -0,0 +1,4 @@
+
+### Files
+* `climatetrace.sources.v2.list`: The list of source URLs
+
diff --git a/examples/archive-downloads-generated/build/climatetrace-agriculture/pom.xml b/examples/archive-downloads-generated/build/climatetrace-agriculture/pom.xml
new file mode 100644
index 0000000..8dbf107
--- /dev/null
+++ b/examples/archive-downloads-generated/build/climatetrace-agriculture/pom.xml
@@ -0,0 +1,127 @@
+
+
+ 4.0.0
+
+ org.aksw.data.configaksw-data-deployment0.0.3
+
+ org.coypu.data.climatetrace
+ climatetrace-agriculture
+ 0.2.0
+ pom
+ Climate TRACE - Agriculture
+ Climate TRACE archive for the sector "Agriculture".
+ https://climatetrace.org
+
+
+
+
+ Creative Commons Attribution 4.0
+ https://creativecommons.org/licenses/by/4.0/
+
+
+
+
+
+
+ https://downloads.climatetrace.org/v02/sector_packages/agriculture.zip
+ agriculture.zip
+ zip
+
+
+
+
+
+
+
+
+
+
+
+ com.googlecode.maven-download-plugin
+ download-maven-plugin
+
+
+ download-dataset
+ process-resources
+
+ wget
+
+
+
+
+ ${input.url}
+ false
+ ${project.build.directory}
+ ${output.filename}
+ true
+
+
+
+
+
+ org.codehaus.mojo
+ build-helper-maven-plugin
+
+
+ attach-artifacts
+ package
+
+ attach-artifact
+
+
+
+
+ ${project.build.directory}/${output.filename}
+ ${output.filetype}
+
+
+
+
+
+
+
+
+
diff --git a/examples/archive-downloads-generated/build/climatetrace-buildings/pom.xml b/examples/archive-downloads-generated/build/climatetrace-buildings/pom.xml
new file mode 100644
index 0000000..ac574a3
--- /dev/null
+++ b/examples/archive-downloads-generated/build/climatetrace-buildings/pom.xml
@@ -0,0 +1,127 @@
+
+
+ 4.0.0
+
+ org.aksw.data.configaksw-data-deployment0.0.3
+
+ org.coypu.data.climatetrace
+ climatetrace-buildings
+ 0.2.0
+ pom
+ Climate TRACE - Buildings
+ Climate TRACE archive for the sector "Buildings".
+ https://climatetrace.org
+
+
+
+
+ Creative Commons Attribution 4.0
+ https://creativecommons.org/licenses/by/4.0/
+
+
+
+
+
+
+ https://downloads.climatetrace.org/v02/sector_packages/buildings.zip
+ buildings.zip
+ zip
+
+
+
+
+
+
+
+
+
+
+
+ com.googlecode.maven-download-plugin
+ download-maven-plugin
+
+
+ download-dataset
+ process-resources
+
+ wget
+
+
+
+
+ ${input.url}
+ false
+ ${project.build.directory}
+ ${output.filename}
+ true
+
+
+
+
+
+ org.codehaus.mojo
+ build-helper-maven-plugin
+
+
+ attach-artifacts
+ package
+
+ attach-artifact
+
+
+
+
+ ${project.build.directory}/${output.filename}
+ ${output.filetype}
+
+
+
+
+
+
+
+
+
diff --git a/examples/archive-downloads-generated/build/climatetrace-fluorinated-gases/pom.xml b/examples/archive-downloads-generated/build/climatetrace-fluorinated-gases/pom.xml
new file mode 100644
index 0000000..8a8ab0e
--- /dev/null
+++ b/examples/archive-downloads-generated/build/climatetrace-fluorinated-gases/pom.xml
@@ -0,0 +1,127 @@
+
+
+ 4.0.0
+
+ org.aksw.data.configaksw-data-deployment0.0.3
+
+ org.coypu.data.climatetrace
+ climatetrace-fluorinated-gases
+ 0.2.0
+ pom
+ Climate TRACE - Fluorinated Gases
+ Climate TRACE archive for the sector "Fluorinated Gases".
+ https://climatetrace.org
+
+
+
+
+ Creative Commons Attribution 4.0
+ https://creativecommons.org/licenses/by/4.0/
+
+
+
+
+
+
+ https://downloads.climatetrace.org/v02/sector_packages/fluorinated_gases.zip
+ fluorinated_gases.zip
+ zip
+
+
+
+
+
+
+
+
+
+
+
+ com.googlecode.maven-download-plugin
+ download-maven-plugin
+
+
+ download-dataset
+ process-resources
+
+ wget
+
+
+
+
+ ${input.url}
+ false
+ ${project.build.directory}
+ ${output.filename}
+ true
+
+
+
+
+
+ org.codehaus.mojo
+ build-helper-maven-plugin
+
+
+ attach-artifacts
+ package
+
+ attach-artifact
+
+
+
+
+ ${project.build.directory}/${output.filename}
+ ${output.filetype}
+
+
+
+
+
+
+
+
+
diff --git a/examples/archive-downloads-generated/build/climatetrace-forestry-and-land-use/pom.xml b/examples/archive-downloads-generated/build/climatetrace-forestry-and-land-use/pom.xml
new file mode 100644
index 0000000..be8ca74
--- /dev/null
+++ b/examples/archive-downloads-generated/build/climatetrace-forestry-and-land-use/pom.xml
@@ -0,0 +1,127 @@
+
+
+ 4.0.0
+
+ org.aksw.data.configaksw-data-deployment0.0.3
+
+ org.coypu.data.climatetrace
+ climatetrace-forestry-and-land-use
+ 0.2.0
+ pom
+ Climate TRACE - Forestry And Land Use
+ Climate TRACE archive for the sector "Forestry And Land Use".
+ https://climatetrace.org
+
+
+
+
+ Creative Commons Attribution 4.0
+ https://creativecommons.org/licenses/by/4.0/
+
+
+
+
+
+
+ https://downloads.climatetrace.org/v02/sector_packages/forestry_and_land_use.zip
+ forestry_and_land_use.zip
+ zip
+
+
+
+
+
+
+
+
+
+
+
+ com.googlecode.maven-download-plugin
+ download-maven-plugin
+
+
+ download-dataset
+ process-resources
+
+ wget
+
+
+
+
+ ${input.url}
+ false
+ ${project.build.directory}
+ ${output.filename}
+ true
+
+
+
+
+
+ org.codehaus.mojo
+ build-helper-maven-plugin
+
+
+ attach-artifacts
+ package
+
+ attach-artifact
+
+
+
+
+ ${project.build.directory}/${output.filename}
+ ${output.filetype}
+
+
+
+
+
+
+
+
+
diff --git a/examples/archive-downloads-generated/build/climatetrace-fossil-fuel-operations/pom.xml b/examples/archive-downloads-generated/build/climatetrace-fossil-fuel-operations/pom.xml
new file mode 100644
index 0000000..63e6526
--- /dev/null
+++ b/examples/archive-downloads-generated/build/climatetrace-fossil-fuel-operations/pom.xml
@@ -0,0 +1,127 @@
+
+
+ 4.0.0
+
+ org.aksw.data.configaksw-data-deployment0.0.3
+
+ org.coypu.data.climatetrace
+ climatetrace-fossil-fuel-operations
+ 0.2.0
+ pom
+ Climate TRACE - Fossil Fuel Operations
+ Climate TRACE archive for the sector "Fossil Fuel Operations".
+ https://climatetrace.org
+
+
+
+
+ Creative Commons Attribution 4.0
+ https://creativecommons.org/licenses/by/4.0/
+
+
+
+
+
+
+ https://downloads.climatetrace.org/v02/sector_packages/fossil_fuel_operations.zip
+ fossil_fuel_operations.zip
+ zip
+
+
+
+
+
+
+
+
+
+
+
+ com.googlecode.maven-download-plugin
+ download-maven-plugin
+
+
+ download-dataset
+ process-resources
+
+ wget
+
+
+
+
+ ${input.url}
+ false
+ ${project.build.directory}
+ ${output.filename}
+ true
+
+
+
+
+
+ org.codehaus.mojo
+ build-helper-maven-plugin
+
+
+ attach-artifacts
+ package
+
+ attach-artifact
+
+
+
+
+ ${project.build.directory}/${output.filename}
+ ${output.filetype}
+
+
+
+
+
+
+
+
+
diff --git a/examples/archive-downloads-generated/build/climatetrace-manufacturing/pom.xml b/examples/archive-downloads-generated/build/climatetrace-manufacturing/pom.xml
new file mode 100644
index 0000000..8b00622
--- /dev/null
+++ b/examples/archive-downloads-generated/build/climatetrace-manufacturing/pom.xml
@@ -0,0 +1,127 @@
+
+
+ 4.0.0
+
+ org.aksw.data.configaksw-data-deployment0.0.3
+
+ org.coypu.data.climatetrace
+ climatetrace-manufacturing
+ 0.2.0
+ pom
+ Climate TRACE - Manufacturing
+ Climate TRACE archive for the sector "Manufacturing".
+ https://climatetrace.org
+
+
+
+
+ Creative Commons Attribution 4.0
+ https://creativecommons.org/licenses/by/4.0/
+
+
+
+
+
+
+ https://downloads.climatetrace.org/v02/sector_packages/manufacturing.zip
+ manufacturing.zip
+ zip
+
+
+
+
+
+
+
+
+
+
+
+ com.googlecode.maven-download-plugin
+ download-maven-plugin
+
+
+ download-dataset
+ process-resources
+
+ wget
+
+
+
+
+ ${input.url}
+ false
+ ${project.build.directory}
+ ${output.filename}
+ true
+
+
+
+
+
+ org.codehaus.mojo
+ build-helper-maven-plugin
+
+
+ attach-artifacts
+ package
+
+ attach-artifact
+
+
+
+
+ ${project.build.directory}/${output.filename}
+ ${output.filetype}
+
+
+
+
+
+
+
+
+
diff --git a/examples/archive-downloads-generated/build/climatetrace-mineral-extraction/pom.xml b/examples/archive-downloads-generated/build/climatetrace-mineral-extraction/pom.xml
new file mode 100644
index 0000000..b808611
--- /dev/null
+++ b/examples/archive-downloads-generated/build/climatetrace-mineral-extraction/pom.xml
@@ -0,0 +1,127 @@
+
+
+ 4.0.0
+
+ org.aksw.data.configaksw-data-deployment0.0.3
+
+ org.coypu.data.climatetrace
+ climatetrace-mineral-extraction
+ 0.2.0
+ pom
+ Climate TRACE - Mineral Extraction
+ Climate TRACE archive for the sector "Mineral Extraction".
+ https://climatetrace.org
+
+
+
+
+ Creative Commons Attribution 4.0
+ https://creativecommons.org/licenses/by/4.0/
+
+
+
+
+
+
+ https://downloads.climatetrace.org/v02/sector_packages/mineral_extraction.zip
+ mineral_extraction.zip
+ zip
+
+
+
+
+
+
+
+
+
+
+
+ com.googlecode.maven-download-plugin
+ download-maven-plugin
+
+
+ download-dataset
+ process-resources
+
+ wget
+
+
+
+
+ ${input.url}
+ false
+ ${project.build.directory}
+ ${output.filename}
+ true
+
+
+
+
+
+ org.codehaus.mojo
+ build-helper-maven-plugin
+
+
+ attach-artifacts
+ package
+
+ attach-artifact
+
+
+
+
+ ${project.build.directory}/${output.filename}
+ ${output.filetype}
+
+
+
+
+
+
+
+
+
diff --git a/examples/archive-downloads-generated/build/climatetrace-power/pom.xml b/examples/archive-downloads-generated/build/climatetrace-power/pom.xml
new file mode 100644
index 0000000..649df9f
--- /dev/null
+++ b/examples/archive-downloads-generated/build/climatetrace-power/pom.xml
@@ -0,0 +1,127 @@
+
+
+ 4.0.0
+
+ org.aksw.data.configaksw-data-deployment0.0.3
+
+ org.coypu.data.climatetrace
+ climatetrace-power
+ 0.2.0
+ pom
+ Climate TRACE - Power
+ Climate TRACE archive for the sector "Power".
+ https://climatetrace.org
+
+
+
+
+ Creative Commons Attribution 4.0
+ https://creativecommons.org/licenses/by/4.0/
+
+
+
+
+
+
+ https://downloads.climatetrace.org/v02/sector_packages/power.zip
+ power.zip
+ zip
+
+
+
+
+
+
+
+
+
+
+
+ com.googlecode.maven-download-plugin
+ download-maven-plugin
+
+
+ download-dataset
+ process-resources
+
+ wget
+
+
+
+
+ ${input.url}
+ false
+ ${project.build.directory}
+ ${output.filename}
+ true
+
+
+
+
+
+ org.codehaus.mojo
+ build-helper-maven-plugin
+
+
+ attach-artifacts
+ package
+
+ attach-artifact
+
+
+
+
+ ${project.build.directory}/${output.filename}
+ ${output.filetype}
+
+
+
+
+
+
+
+
+
diff --git a/examples/archive-downloads-generated/build/climatetrace-transportation/pom.xml b/examples/archive-downloads-generated/build/climatetrace-transportation/pom.xml
new file mode 100644
index 0000000..03a2c73
--- /dev/null
+++ b/examples/archive-downloads-generated/build/climatetrace-transportation/pom.xml
@@ -0,0 +1,127 @@
+
+
+ 4.0.0
+
+ org.aksw.data.configaksw-data-deployment0.0.3
+
+ org.coypu.data.climatetrace
+ climatetrace-transportation
+ 0.2.0
+ pom
+ Climate TRACE - Transportation
+ Climate TRACE archive for the sector "Transportation".
+ https://climatetrace.org
+
+
+
+
+ Creative Commons Attribution 4.0
+ https://creativecommons.org/licenses/by/4.0/
+
+
+
+
+
+
+ https://downloads.climatetrace.org/v02/sector_packages/transportation.zip
+ transportation.zip
+ zip
+
+
+
+
+
+
+
+
+
+
+
+ com.googlecode.maven-download-plugin
+ download-maven-plugin
+
+
+ download-dataset
+ process-resources
+
+ wget
+
+
+
+
+ ${input.url}
+ false
+ ${project.build.directory}
+ ${output.filename}
+ true
+
+
+
+
+
+ org.codehaus.mojo
+ build-helper-maven-plugin
+
+
+ attach-artifacts
+ package
+
+ attach-artifact
+
+
+
+
+ ${project.build.directory}/${output.filename}
+ ${output.filetype}
+
+
+
+
+
+
+
+
+
diff --git a/examples/archive-downloads-generated/build/climatetrace-waste/pom.xml b/examples/archive-downloads-generated/build/climatetrace-waste/pom.xml
new file mode 100644
index 0000000..1cb88f2
--- /dev/null
+++ b/examples/archive-downloads-generated/build/climatetrace-waste/pom.xml
@@ -0,0 +1,127 @@
+
+
+ 4.0.0
+
+ org.aksw.data.configaksw-data-deployment0.0.3
+
+ org.coypu.data.climatetrace
+ climatetrace-waste
+ 0.2.0
+ pom
+ Climate TRACE - Waste
+ Climate TRACE archive for the sector "Waste".
+ https://climatetrace.org
+
+
+
+
+ Creative Commons Attribution 4.0
+ https://creativecommons.org/licenses/by/4.0/
+
+
+
+
+
+
+ https://downloads.climatetrace.org/v02/sector_packages/waste.zip
+ waste.zip
+ zip
+
+
+
+
+
+
+
+
+
+
+
+ com.googlecode.maven-download-plugin
+ download-maven-plugin
+
+
+ download-dataset
+ process-resources
+
+ wget
+
+
+
+
+ ${input.url}
+ false
+ ${project.build.directory}
+ ${output.filename}
+ true
+
+
+
+
+
+ org.codehaus.mojo
+ build-helper-maven-plugin
+
+
+ attach-artifacts
+ package
+
+ attach-artifact
+
+
+
+
+ ${project.build.directory}/${output.filename}
+ ${output.filetype}
+
+
+
+
+
+
+
+
+
diff --git a/examples/archive-downloads-generated/build/pom.xml b/examples/archive-downloads-generated/build/pom.xml
new file mode 100644
index 0000000..127e33a
--- /dev/null
+++ b/examples/archive-downloads-generated/build/pom.xml
@@ -0,0 +1,51 @@
+
+
+ 4.0.0
+ org.coypu.data.climatetrace
+ climatetrace-parent
+ 0.2.0
+ pom
+
+ org.aksw.data.configaksw-data-deployment0.0.3
+
+
+
+ climatetrace-fluorinated-gases
+
+ climatetrace-agriculture
+
+ climatetrace-buildings
+
+ climatetrace-fossil-fuel-operations
+
+ climatetrace-forestry-and-land-use
+
+ climatetrace-manufacturing
+
+ climatetrace-mineral-extraction
+
+ climatetrace-power
+
+ climatetrace-transportation
+
+ climatetrace-waste
+
+
+
+
+
+ maven-deploy-plugin
+
+ true
+
+
+
+ maven-install-plugin
+
+ true
+
+
+
+
+
diff --git a/examples/archive-downloads-generated/climatetrace.sources.v2.list b/examples/archive-downloads-generated/climatetrace.sources.v2.list
new file mode 100644
index 0000000..c82f6a1
--- /dev/null
+++ b/examples/archive-downloads-generated/climatetrace.sources.v2.list
@@ -0,0 +1,11 @@
+https://downloads.climatetrace.org/v02/sector_packages/fluorinated_gases.zip
+https://downloads.climatetrace.org/v02/sector_packages/agriculture.zip
+https://downloads.climatetrace.org/v02/sector_packages/buildings.zip
+https://downloads.climatetrace.org/v02/sector_packages/fossil_fuel_operations.zip
+https://downloads.climatetrace.org/v02/sector_packages/forestry_and_land_use.zip
+https://downloads.climatetrace.org/v02/sector_packages/manufacturing.zip
+https://downloads.climatetrace.org/v02/sector_packages/mineral_extraction.zip
+https://downloads.climatetrace.org/v02/sector_packages/power.zip
+https://downloads.climatetrace.org/v02/sector_packages/transportation.zip
+https://downloads.climatetrace.org/v02/sector_packages/waste.zip
+
diff --git a/examples/archive-downloads-generated/consumer/pom.xml b/examples/archive-downloads-generated/consumer/pom.xml
new file mode 100644
index 0000000..27d0f2e
--- /dev/null
+++ b/examples/archive-downloads-generated/consumer/pom.xml
@@ -0,0 +1,137 @@
+
+
+ 4.0.0
+ org.coypu.data.climatetrace
+ climatetrace-consumer
+ 0.2.0
+
+ org.aksw.data.configaksw-data-deployment0.0.3
+
+ pom
+
+
+ ${project.build.directory}/datasets
+
+
+
+
+
+ ${project.build.directory}/datasets
+
+
+
+
+
+ org.apache.maven.plugins
+ maven-dependency-plugin
+
+
+ resource-dependencies
+ process-resources
+
+ copy
+
+
+ true
+
+
+
+ org.coypu.data.climatetrace
+ climatetrace-fluorinated-gases
+ 0.2.0
+ zip
+ true
+ ${datasetDirectory}
+
+
+
+ org.coypu.data.climatetrace
+ climatetrace-agriculture
+ 0.2.0
+ zip
+ true
+ ${datasetDirectory}
+
+
+
+ org.coypu.data.climatetrace
+ climatetrace-buildings
+ 0.2.0
+ zip
+ true
+ ${datasetDirectory}
+
+
+
+ org.coypu.data.climatetrace
+ climatetrace-fossil-fuel-operations
+ 0.2.0
+ zip
+ true
+ ${datasetDirectory}
+
+
+
+ org.coypu.data.climatetrace
+ climatetrace-forestry-and-land-use
+ 0.2.0
+ zip
+ true
+ ${datasetDirectory}
+
+
+
+ org.coypu.data.climatetrace
+ climatetrace-manufacturing
+ 0.2.0
+ zip
+ true
+ ${datasetDirectory}
+
+
+
+ org.coypu.data.climatetrace
+ climatetrace-mineral-extraction
+ 0.2.0
+ zip
+ true
+ ${datasetDirectory}
+
+
+
+ org.coypu.data.climatetrace
+ climatetrace-power
+ 0.2.0
+ zip
+ true
+ ${datasetDirectory}
+
+
+
+ org.coypu.data.climatetrace
+ climatetrace-transportation
+ 0.2.0
+ zip
+ true
+ ${datasetDirectory}
+
+
+
+ org.coypu.data.climatetrace
+ climatetrace-waste
+ 0.2.0
+ zip
+ true
+ ${datasetDirectory}
+
+
+
+
+
+
+
+
+
+
\ No newline at end of file
diff --git a/examples/archive-downloads-generated/create-maven-model.py b/examples/archive-downloads-generated/create-maven-model.py
new file mode 100755
index 0000000..79a2c8c
--- /dev/null
+++ b/examples/archive-downloads-generated/create-maven-model.py
@@ -0,0 +1,111 @@
+#!/bin/python
+
+import os
+
+from urllib.parse import urlparse
+from pathlib import Path
+from string import Template
+from jinja2 import Environment, FileSystemLoader
+import xml.etree.ElementTree as ET
+
+file_path = 'climatetrace.sources.v2.list'
+
+group_id = 'org.coypu.data.climatetrace'
+version = '0.2.0'
+project_prefix = "climatetrace-"
+
+parent_str = "org.aksw.data.configaksw-data-deployment0.0.3"
+
+def capitalize_words(s):
+ return ' '.join(word.capitalize() for word in s.split())
+
+def prettify_xml(s):
+ return s
+ #xml = ET.fromstring(s)
+ #ET.indent(xml)
+ #result = ET.tostring(xml, encoding='unicode')
+ #return result
+
+
+build_path = 'build'
+consumer_path = 'consumer'
+os.makedirs(build_path, exist_ok=True)
+os.makedirs(consumer_path, exist_ok=True)
+
+env = Environment(loader=FileSystemLoader('resources'))
+artifact_template = env.get_template('artifact.template.pom.xml')
+parent_template = env.get_template('parent.template.pom.xml')
+consumer_template = env.get_template('consumer.template.pom.xml')
+
+# Open the file and process each line
+with open(file_path, 'r') as file:
+ modules = []
+ for line in file:
+ line = line.strip()
+ if line and not line.startswith('#'):
+ parsed_url = urlparse(line)
+ # Extracting the local name (last part of the path)
+ input_file_fullname = parsed_url.path.split('/')[-1]
+ input_file_parts = input_file_fullname.partition('.') # os.path.splitext(input_file_fullname)
+ input_file_basename = input_file_parts[0]
+ input_file_ext = input_file_parts[2]
+ base_name = input_file_basename.replace('_', '-')
+ artifact_id = f'{project_prefix}{base_name}'
+ output_file_fullname_alt = f'{artifact_id}.{input_file_ext}'
+ output_file_fullname = input_file_fullname
+ module_name = artifact_id
+ tmp_name = base_name.replace('-', ' ')
+ sector_name = capitalize_words(tmp_name)
+ # print(f'{line} -> {module_name}')
+ module = {
+ "moduleName": module_name,
+ "parentStr": parent_str,
+ "groupId": group_id,
+ "artifactId": artifact_id,
+ "version": version,
+ "description": f'Climate TRACE archive for the sector "{sector_name}".',
+ "name": f'Climate TRACE - {sector_name}',
+ "url": "https://climatetrace.org",
+ "licenses": [{
+ "name": "Creative Commons Attribution 4.0",
+ "url": "https://creativecommons.org/licenses/by/4.0/"
+ }],
+ "input": {
+ "url": line
+ },
+ "output": {
+ "filename": output_file_fullname,
+ "filetype": input_file_ext
+ }
+ }
+ rendered = artifact_template.render(this=module)
+ rendered = prettify_xml(rendered)
+ modules.append(module)
+ module_folder=f'{build_path}/{module_name}'
+ os.makedirs(module_folder, exist_ok=True)
+ with open(f'{module_folder}/pom.xml', "w") as file:
+ file.write(rendered)
+ parent = {
+ "parentStr": parent_str,
+ "groupId": group_id,
+ "artifactId": f'{project_prefix}parent',
+ "version": version,
+ "modules": modules
+ }
+ rendered = parent_template.render(this=parent)
+ rendered = prettify_xml(rendered)
+ with open(f'{build_path}/pom.xml', "w") as file:
+ file.write(rendered)
+ consumer = {
+ "parentStr": parent_str,
+ "groupId": group_id,
+ "artifactId": f'{project_prefix}consumer',
+ "version": version,
+ "modules": modules
+ }
+ rendered = consumer_template.render(this=consumer)
+ rendered = prettify_xml(rendered)
+ with open(f'{consumer_path}/pom.xml', "w") as file:
+ file.write(rendered)
+
+
diff --git a/examples/archive-downloads-generated/resources/artifact.template.pom.xml b/examples/archive-downloads-generated/resources/artifact.template.pom.xml
new file mode 100644
index 0000000..88f24ee
--- /dev/null
+++ b/examples/archive-downloads-generated/resources/artifact.template.pom.xml
@@ -0,0 +1,128 @@
+
+
+ 4.0.0
+
+ {{this.parentStr}}
+
+ {{this.groupId}}
+ {{this.artifactId}}
+ {{this.version}}
+ pom
+ {{this.name}}
+ {{this.description}}
+ {{this.url}}
+
+
+ {% for license in this.licenses %}
+
+ {{license.name}}
+ {{license.url}}
+
+ {% endfor %}
+
+
+
+
+ {{this.input.url}}
+ {{this.output.filename}}
+ {{this.output.filetype}}
+
+
+
+
+
+
+
+
+
+
+
+ com.googlecode.maven-download-plugin
+ download-maven-plugin
+
+
+ download-dataset
+ process-resources
+
+ wget
+
+
+
+
+ ${input.url}
+ false
+ ${project.build.directory}
+ ${output.filename}
+ true
+
+
+
+
+
+ org.codehaus.mojo
+ build-helper-maven-plugin
+
+
+ attach-artifacts
+ package
+
+ attach-artifact
+
+
+
+
+ ${project.build.directory}/${output.filename}
+ ${output.filetype}
+
+
+
+
+
+
+
+
+
+
diff --git a/examples/archive-downloads-generated/resources/consumer.template.pom.xml b/examples/archive-downloads-generated/resources/consumer.template.pom.xml
new file mode 100644
index 0000000..20e68a7
--- /dev/null
+++ b/examples/archive-downloads-generated/resources/consumer.template.pom.xml
@@ -0,0 +1,56 @@
+
+
+ 4.0.0
+ {{this.groupId}}
+ {{this.artifactId}}
+ {{this.version}}
+
+ {{this.parentStr}}
+
+ pom
+
+
+ ${project.build.directory}/datasets
+
+
+
+
+
+ ${project.build.directory}/datasets
+
+
+
+
+
+ org.apache.maven.plugins
+ maven-dependency-plugin
+
+
+ resource-dependencies
+ process-resources
+
+ copy
+
+
+ true
+
+ {% for item in this.modules %}
+
+ {{item.groupId}}
+ {{item.artifactId}}
+ {{item.version}}
+ {{item.output.filetype}}
+ true
+ ${datasetDirectory}
+
+ {% endfor %}
+
+
+
+
+
+
+
+
diff --git a/examples/archive-downloads-generated/resources/parent.template.pom.xml b/examples/archive-downloads-generated/resources/parent.template.pom.xml
new file mode 100644
index 0000000..1e2cef3
--- /dev/null
+++ b/examples/archive-downloads-generated/resources/parent.template.pom.xml
@@ -0,0 +1,34 @@
+
+
+ 4.0.0
+ {{this.groupId}}
+ {{this.artifactId}}
+ {{this.version}}
+ pom
+
+ {{this.parentStr}}
+
+
+ {% for module in this.modules %}
+ {{module.moduleName}}
+ {% endfor %}
+
+
+
+
+ maven-deploy-plugin
+
+ true
+
+
+
+ maven-install-plugin
+
+ true
+
+
+
+
+
+