RPM packages for RHEL/CentOS 7

[tschoonj]

Hi Bruce,

I created RPM packages of Demeter, Ifeffit and about a dozen of Perl modules for RHEL/CentOS 7. The spec files can be found at https://github.com/tschoonj/demeter-linux-packages, where instructions can be found to install the RPMs with minimal effort.

As far as I can tell most of it works properly, with the notable exception of Hephaestus, which segfaults on startup after I close a Warning dialog.

If I have some time I will look into creating similar packages for Debian/Ubuntu.

Feel free to give it a try, and let me know what you think.

Best,

Tom

[bruceravel]

Holy shit! Tom, you are my hero! I'll update the web site when I get back to my desk.

If you show me what's going on with the Heph crash, maybe I can help diagnose it.

[tschoonj]

Thanks for the kind words, but I am not confident that everything works properly at this point. The Hephaestus crash is most likely not the only issue. I am attaching a screenshot of how things look after running dhephaestus. Interestingly, Hephaestus segfaults immediately after the screenshot is taken, even without me interacting with it!

[newville]

Please do not use or encourage the use ifeffit for this. While it is open source and you are free to use it however you see fit, it has several problems and has not been supported or maintained for many years. A complete replacement exists and is supported.

…

On Apr 6, 2017 8:51 AM, "Bruce Ravel" ***@***.***> wrote: Holy shit! Tom, you are my hero! I'll update the web site when I get back to my desk. If you show me what's going on with the Heph crash, maybe I can help diagnose it. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#50 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AASke8ScqBtMHH6WoNsJGn1Pnch3q5OCks5rtO35gaJpZM4M1npJ> .

[tschoonj]

By the way, it should be rather easy to create packages for Fedora too, especially because several of the Perl modules I had to create RPMs for, have already been packaged for that distribution.

[tschoonj]

@newville No problem for me. I could look into packaging larch if that is the preferred backend.

FWIW it is currently not possible to build Demeter without having ifeffit installed as Build returns with the following error:

* WARNING: Configuration was initially created with Module::Build
  version '0.4005' but we are now using version '0.4211'.
  If errors occur, you must re-run the Build.PL or Makefile.PL script.
Can't exec "ifeffit": No such file or directory at DemeterBuilder.pm line 201.
Use of uninitialized value $iffdir in substitution (s///) at DemeterBuilder.pm line 202.
Use of uninitialized value $iffdir in substitution (s///) at DemeterBuilder.pm line 202.
Could not open /config/Config.mak file for reading
Ifeffit's installations directory is 
	(found by capturing `ifeffit -i`)

[bruceravel]

@tschoonj : The error message suggests that you do not have https://metacpan.org/pod/Chemistry::Elements on your machine. That wouldn't explain the crash, but would explain the warning dialog.

@newville Tom is right. The build script does not properly allow someone to opt out of using Ifeffit from the get-go. That's an oversight. Even before this conversation started, I was planning on updating to the latest Larch today and verifying the Demeter test suite. I'll work on the build script while I am doing that.

[tschoonj]

Chemistry::Elements is installed though: it's one of the perl modules I had to package myself.

perl-Chemistry-Elements-1.071-1.el7.centos.noarch already installed and latest version.

That package does not contain much, is that correct? Just one module named Chemistry/Elements.pm...

There are no messages in the terminal.

[newville]

@tschoonj ah, ok. I hadn't realized that ifeffit was needed for the build.

I haven't thought about rpms for Larch. I use Centos and Fedora, but have tended to not require system python and the official packages for Larch and nornally use virtual environments or Anaconda python which are orthogonal to the yum/dnf system. I think we could make rpms work with the official Centos or Fedora systems but it might take some iteration. I'll look into what is needed to run with the official Centos7 packages.

[bruceravel]

@tschoonj @newville :
I think that 74ca326 fixes the issue that Ifeffit must be present in order to build Demeter. I also think that Demeter will then run correctly if only Larch and not Ifeffit is present. Not fully tested, but I am confident-ish.

[bruceravel]

@tschoonj Regarding the warning message when you fire up Heph ... could you click the details button, then save what it says. I am a bit flummoxed, but I am now wondering if it has something to do with the very heavy elements. It might help to know how many times that warning message was triggered. Thanks.

[newville]

@bruceravel

I think that 74ca326 fixes the issue that Ifeffit must be present in order to build Demeter.
I also think that Demeter will then run correctly if only Larch and not Ifeffit is present.
Not fully tested, but I am confident-ish.

Awesome!

[newville]

@tschoonj OK, after a little more investigation, I am less optimistic about being able to support Larch with the standard system Python stack included with Centos 7, as the packages are fairly old:

• numpy is at 1.7 (current 1.11). This would be OK, but might cause one or two patches.

• scipy is at 1.12 (current 1.18). This will definitely cause problems. The optimization code in scipy has progressed. Currently, parts of Larch still use the older approach and parts rely on LMFIT - I'm working towards using LMFIT throughout. LMFIT now requires scipy 0.15, for good reasons.

• matplotlib is at 1.2 (current 1.5 or 2.0). This may cause problems, but could worked around.

• wxPython is not available in the main repo. The common "EPEL" repo has wxPython 2.8.12 (current is 3.something), which may cause problems, but could be worked around.

Basically, these represent versions that are 4 years old. That's not ridiculously old, but scientific python has been moving fast. By "could be worked around", I definitely mean getting in there and patching existing code to work with older versions, so real work.

I think relying on Anaconda Python for Larch on Centos7 is the way to go -- don't bother messing with system Python, and let each user install their own local version.

[tschoonj]

@bruceravel Clicking the Details button triggers the segfault unfortunately 😢
It feels to me that this is coming from wxPerl/wxWidgets, but not sure what. I will investigate when I have the time. Also, thanks for removing the Ifeffit build time dependency!

@newville I agree completely with your assessment. On older systems like Centos 7, Anaconda is definitely the best solution. The latest Fedora release should be able to run larch even with the dependencies offered by the official repositories, as they are famous/notorious for using the latest versions. Not sure if it would be worth investigating RPM building for this distribution though.

[newville]

@tschoonj I have no strong opinion on rpms for Fedora. I use Fedora and Centos and it is definitely a challenge keeping up with Fedora releases. But, I believe that Larch works fine (for XAFS + XRF) using only standard packages for Fedora 24 and later.

I guess it depends on your goals. If DLS is trying to deploy on many similar Linuxes, rpms would be a fine way to go. For Python tools like Larch though, using Anaconda Python and their conda packaging system seems to be the way to go.

[tschoonj]

@bruceravel It looks like you were right: it definitely has to do with the heavy elements. I removed all data in @elements beyond Pu and Hephaestus works fine!

[bruceravel]

@newville : So ... the problem that occurred to me this morning about removing the dependence on Ifeffit for building Demeter is that, without installing Ifeffit, there is not currently an obvious path for having Feff in place. That is, Demeter tacitly expects that Feff6 will get installed when Ifeffit gets installed. Obviously the Feff8L thing will eventually fix that. But for now, installing Larch, then installing Demeter could leave things in a state where Artemis won't do anything. Obviously that's Demeter's problem, not Larch's problem. But it does suggest that @tschoonj should make an rpm package for Ifeffit which includes feff -- at least for now.

[tschoonj]

That's already the case: the ifeffit RPM contains all executables, so including feff6, that are produced by ifeffit.

[newville]

@bruceravel Ah, it is complete reasonable for Larch to include a feff6l binary. It doesn't currently do that, but it could. I'll work on that for next release.

Perhaps it suggests for @tschoonj that an rpm for Demeter should include an executable of feff6l.

[bruceravel]

@tschoonj : I updated my Ubuntu machine this week to 17.04 and am now seeing the "missing argument in sprintf" error message. I am assuming it has something to do with perl 5.24, which is relatively new. Anyway, now that I see the problem, I imagine it'll get fixed when I find a few minutes. Thanks, as always, for the report -- I was just a bit slow this time 😄

[bruceravel]

@tschoonj : Turns out that an upstream module is broken. I am having trouble finding a graceful solution. Will work on it some more tomorrow.