Try using gumtree-simple matcher in Spork instead of gumtree-classic

[jrfaller]

Since GumTree's version 3.0 we have a new faster and more accurate matcher, called gumtree-simple. I think it could be a nice thing to try using it inside Spork instead of gumtree-classic as the base matcher and perhaps also the left-right matcher? I am not sure where in the code the matchers are instantiated, but I think it should be easy to do.

WDYT?

Cheers!

[monperrus]

it's a great idea!

[jrfaller]

OK found the code here -> https://github.com/ASSERT-KTH/spork/blob/master/src/main/kotlin/se/kth/spork/spoon/Spoon3dmMerge.kt#L355-L365 therefore I can easily make a patch to update to gumtree-simple. However, it would be nice to know whether or not it improves merging time and/or textual similarity to the gold set compared to using classic and XY. Do you have any idea how to do that?

[slarse]

Hey @jrfaller,

That's an interesting idea. Swapping out left/right matcher is almost guaranteed to cause trouble failures as everything is very finely tuned for the X/Y matcher. I chose the X/Y matcher because it was less prone to spurious left/right matchings (which cause massive trouble). The default matcher caused way too many spurious matchings and was very impractical to use for that.

The base matcher might be OK to swap out, but it also depends on how Gumtree-Spoon has changed since Spork was created. Spork makes heavy use of Gumtree-Spoon's internal behavior, I think the most blatant example of this is where it infers additional mappings (see

spork/src/main/kotlin/se/kth/spork/spoon/matching/SpoonMapping.kt

Lines 50 to 61 in 7672b76

	private fun inferAdditionalMappings(matches: List<Pair<CtElement, CtElement>>) {
	var mutableMatches = matches
	while (mutableMatches.isNotEmpty()) {
	val newMatches: MutableList<Pair<CtElement, CtElement>> = ArrayList()
	for (match in mutableMatches) {
	val src = match.first
	val dst = match.second
	newMatches.addAll(inferAdditionalMappings(src, dst))
	}
	mutableMatches = newMatches
	}
	}

).

The benchmark suite should be viable to do some quick and dirty evaluations. The latest version is in the replication package (download https://github.com/ASSERT-KTH/spork/releases/download/v0.5.1/replication_package.tar.gz, the benchmark suite is in replication_package/software/benchmark-scripts, README explains from there). I'm not sure why I never checked that into the repository.

There's also a very outdated version of the benchmark suite that for reasons along the lines of "I never got around to it" still runs in CI, see e.g. https://github.com/ASSERT-KTH/spork/actions/runs/7715093311/job/21028824279. But I would go with the one in the replication package.

For a comprehensive evaluation I think just running the experiments again would be the least hands-on thing to do (see https://github.com/ASSERT-KTH/spork/tree/master/replication#setting-up-for-the-experiments).

[monperrus]

OK found the code here -> master/src/main/kotlin/se/kth/spork/spoon/Spoon3dmMerge.kt#L355-L365 therefore I can easily make a patch to update to gumtree-simple

give it a try @jrfaller !

[jrfaller]

OK I am preparing a pull request, we will see how to run a benchmark to see whether or not it is useful! Cheers!

[jrfaller]

Started the WIP in #534 you can take a look