Adding new script "poisson_err.py"

[jlothringer]

Adding a new script "poisson_err.py" to add the function poisson_err.poisson_err(), which calculates Poisson confidence intervals for NUV-MAMA and FUV-MAMA 1D extracted spectra using astropy's stats.poisson_confidence_interval function. These Poisson confidence intervals are the more statistically robust way to calculate errors for data with low numbers of event counts, like in NUV and FUV data, compared to the root-N approximation used by the pipeline. This is related to the "Low_Count_Uncertainties" notebook in development here.

[sean-lockwood]

@jlothringer -
Do you have a copy of the test output dataset (obgh07020_PCI_x1d.fits) you want to use as a standard? We'll need to figure out how to upload it to Artifactory.

Does it use the current standard pipeline x1d file as input?

[sean-lockwood]

Was this breaking the install? I recall @stscirij talking about removing teal code...

[jlothringer]

No, I think install was fine. I just re-installed the latest version of stenv and everything went smoothly. When importing stistools, I get the usual NMPFIT and GFIT deprectation warnings and then the usual TEAL package note ("The following tasks...").

[sean-lockwood]

We're going to want to add a file for the Sphinx documentation here:
https://github.com/jlothringer/stistools/tree/master/doc/source

Most of these files are pretty minimal, pointing to the docstring. We can iterate on the contents/format of the docstring we want to use for this purpose.

[jlothringer]

@jlothringer - Do you have a copy of the test output dataset (obgh07020_PCI_x1d.fits) you want to use as a standard? We'll need to figure out how to upload it to Artifactory.

Does it use the current standard pipeline x1d file as input?

Yes, I've uploaded both test input and output datasets to Box here: https://stsci.box.com/s/md0i61m1hdapbuse90vgtg0agr7km88u

And they indeed used a standard x1d file as input (which I've also uploaded there).

[sean-lockwood]

I've "deployed" the test input and truth files to our artifactory instance.

[sean-lockwood]

@jlothringer -
To get the docs building and the tests working (locally), you'll need to rebase with the Master branch from the spacetelescope fork.

First, confirm we have the same terminology for remotes:

git remote -vv

If origin refers to your fork, it should look like this:

origin	[email protected]:jlothringer/stistools.git (fetch)
origin	[email protected]:jlothringer/stistools.git (push)

You should then add an upstream remote if it doesn't exist pointing to the spacetelescope fork:

git remote add upstream [email protected]:spacetelescope/stistools.git

Now you can fetch changes to your system:

git fetch upstream

And then rebase, which will re-run your commits on top of the current Master branch:

git rebase upstream/master

There may be merge conflicts to resolve at this point. After dealing with these, you'll then push to your fork:

git push --force origin master

(Again, this assumes origin refers to jlothringer's fork above).

This will update the pull request. Let me know if you have any questions or encounter any issues!

We'll then be able to install and run tests "locally" on a server:
<conda activate ...>
<clone, checkout, etc>

cd stistools
pip install '.[test]'
pytest --bigdata --slow tests/test_poisson_error.py

[sean-lockwood]

Additionally, you'll need to modify the doc/source/index.rst file with your new routine.

[jlothringer]

Great, I think I've succeeded in doing this. The RTD build looks like it succeeded as well. I've run the test (note: tests/test_poisson_err.py, not tests/test_poisson_error.py) on my local machine and on plhstins4 at /home/jlothringer/STIS/stistools/ in a fresh stenv environment and I get the following, none of which concerns poisson_err.py: