diff --git a/docs/images/2024-03-01-mvn-sync-architecture.png b/docs/images/2024-03-01-mvn-sync-architecture.png new file mode 100644 index 0000000..454bbcd Binary files /dev/null and b/docs/images/2024-03-01-mvn-sync-architecture.png differ diff --git a/docs/sync/change-detection.md b/docs/sync/change-detection.md index f73cc6f..7e98478 100644 --- a/docs/sync/change-detection.md +++ b/docs/sync/change-detection.md @@ -1,8 +1,8 @@ --- title: Maven Repository Events -layout: home +layout: default parent: Maven-RDF Sync -nav_order: 10 +nav_order: 20 --- diff --git a/docs/sync/messaging.md b/docs/sync/messaging.md index 592a367..359fc17 100644 --- a/docs/sync/messaging.md +++ b/docs/sync/messaging.md @@ -1,8 +1,8 @@ --- title: Messaging -layout: home +layout: default parent: Maven-RDF Sync -nav_order: 20 +nav_order: 30 --- # Messaging diff --git a/docs/sync/sync-architecture.md b/docs/sync/sync-architecture.md index c212642..fcdcd4d 100644 --- a/docs/sync/sync-architecture.md +++ b/docs/sync/sync-architecture.md @@ -1,41 +1,21 @@ --- -title: Maven-RDF Sync +title: Mvn-Sync Architecture layout: default -nav_order: 60 -has_children: true +parent: Maven-RDF Sync +nav_order: 10 --- -# Synchronizing a Triple Store with Maven Repository Data +# Mvn Sync Architecture ## Synopsis -* This chapter presents a lightweight trigger-based approach to realize "build actions" (or "bots") over local maven repositories. A local maven repository is simply a certain directory structure. - These bots can be used to automatically create new maven projects for producing RDF metadata. - ## Purpose -* Metadata artifacts are just plain maven artifacts whose content describes another artifact. -* A large part of RDF metadata generation is agnostic of the content of a dataset and can be fully automated. In those cases, a user should not need to manually set up metadata projects. - ## Abstract Approach -The `mvn-rdf-sync` approach comprises two separate processes: - - -1. A *file system watch* on a (local) maven repository notifies raises events whenever the repository content changes. -2. The event is transmitted to an appropriate receiver. -3. A *filter* discards irrelevant events, such as changes to non-dataset artifacts. -4. If a dataset is deployed (which is not a metadata dataset), then automatically create an instance of a template maven project, and run the build. - -TODO architecture chart with the addition of prepublish and triple store loading - - ### Concrete Approach -1. A simple way to watch a directory recursively for changes is `inotifywait`. This commands even works from within a docker container on a mounted host folder. -2. A relatively straight forward way to transmit events is with Apache Kafka. The advantage of a messaging protocol is that it is possible to send custom events besides the `inotifyway` source. - This is useful when one wants to trigger recreation of metadata after changing the routine that produced the matdata.