strauss_dotAstro13.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "b05ae3e1-1744-4674-9b6a-64a12536412d",
   "metadata": {},
   "source": [
    "<a target=\"_blank\" href=\"https://colab.research.google.com/github/james-trayford/dotAstro13/blob/main/strauss_dotAstro13.ipynb\">\n",
    "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5014193-c2b7-4c3e-93a1-0b040e036e7e",
   "metadata": {},
   "source": [
    "Notebook prepared by **Dr James Trayford** - for queries please email [`james.trayford@port.ac.uk`](mailto:james.trayford@port.ac.uk)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6df294e0-bbce-4654-916c-3e1b0ef6893a",
   "metadata": {},
   "source": [
    "To use this notebook (if you haven't already) you can first save a copy to your local drive by clicking `File > Save a Copy in Drive` and run on that copy. `Edit > Clear all outputs` on that copy should also ensure you have a clean version to start from."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c982dd6-334b-4dc7-ba7c-9ab8724ea8bc",
   "metadata": {},
   "source": [
    "# **.Astronomy 13** - 🔊🎶🎵 Sonification of data with `strauss` "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f166839e-8300-405a-8699-7ea368b9a01c",
   "metadata": {},
   "source": [
    "In this notebook, I've tried to make the different examples stand-alone so you can run and re-run cells out of order once you've got the data. This means repeating code between cells. To give some hints at what you might want to edit in these examples, I've added comments and the \"✏️\" symbol \"\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c79a43c-3aeb-4afd-8db8-c159535dc794",
   "metadata": {},
   "source": [
    "first, let's install the `strauss` code!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "60330dfe-93c7-40ad-a4ac-c502b18299e3",
   "metadata": {},
   "outputs": [],
   "source": [
    " %pip install --quiet strauss"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66276559-7de7-493a-b406-ad5f21d2fe88",
   "metadata": {},
   "source": [
    "*We will use the `spectraliser` development branch for this notebook. Install can take a while - but you should only need to run it once!*\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "49e64c89-2946-4d6c-bfb3-6e35684fe327",
   "metadata": {},
   "source": [
    "We'll also grab the repository from _GitHub_ for easy access to some files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "27502abf-acfc-4cef-aba0-2796a0e4a6c3",
   "metadata": {},
   "outputs": [],
   "source": [
    " !git clone https://github.com/james-trayford/strauss.git"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b08ddb1-d6a4-474f-9f7a-f752e58c6845",
   "metadata": {},
   "source": [
    "Now, let's import the modules we need..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5425800d-ca2f-47a1-977a-273e41234c54",
   "metadata": {},
   "outputs": [],
   "source": [
    "%reload_ext autoreload \n",
    "%autoreload 2\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "from strauss.sonification import Sonification\n",
    "from strauss.sources import Objects, Events\n",
    "from strauss import channels\n",
    "from strauss.score import Score\n",
    "from strauss.generator import Synthesizer, Spectralizer, Sampler\n",
    "from strauss import sources as Sources\n",
    "\n",
    "import IPython.display as ipd\n",
    "import os\n",
    "from scipy.interpolate import interp1d\n",
    "import numpy as np\n",
    "import requests\n",
    "\n",
    "# modules to display in-notebook\n",
    "import IPython.display as ipd\n",
    "from IPython.display import display, Markdown, Latex, Image"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "756e6562-7554-4e76-9be1-5cb10c14f294",
   "metadata": {},
   "source": [
    "## Section 1 - Getting Data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3d4bdad-0099-4c7b-94a5-948b56dbec13",
   "metadata": {},
   "source": [
    "⚠  ***The cells in this section are to organise the data we need for the session, don't worry too much about the details of the code here!*** ⚠"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aa0fdcee-49d2-4a4a-b79f-f975a9603876",
   "metadata": {},
   "outputs": [],
   "source": [
    "import glob\n",
    "import urllib\n",
    "import zipfile\n",
    "import os"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c71dbbf-9a9e-4e6f-af76-a27db09d212f",
   "metadata": {},
   "source": [
    "We download and unpack the data used in this notebook:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7c780bd9-edbf-4972-bd8f-b739b7af7a54",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Don't bother if the directory already exists (i.e. Section 0 was run)\n",
    "if not os.path.isdir('prepared_data'):\n",
    "    outdir = \"./prepared_data\"\n",
    "\n",
    "    path = os.path.realpath(outdir)\n",
    "    if not glob.glob(outdir):\n",
    "      os.mkdir(path)\n",
    "\n",
    "    fname = \"dotAstro13_STRAUSS_tutorial.zip\"\n",
    "    url = \"https://drive.google.com/uc?export=download&id=1sXesBtj2DE1fyThay8EX6l87H9cDz0-h\"\n",
    "    print(f\"Downloading files...\")\n",
    "\n",
    "    with urllib.request.urlopen(url) as response, open(f\"{path}/{fname}\", 'wb') as out_file:\n",
    "      data = response.read() # a `bytes` object\n",
    "      out_file.write(data)\n",
    "\n",
    "    print(f\"Unzipping files to {outdir}/ ...\")\n",
    "    with zipfile.ZipFile(f\"{outdir}/{fname}\", 'r') as zip_ref:\n",
    "        zip_ref.extractall(f\"{outdir}\")\n",
    "\n",
    "    print(f\"Clearing up...\")\n",
    "    os.remove(f\"{path}/{fname}\")\n",
    "\n",
    "    print(\"Done.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dbed37aa-4fcd-4f35-8407-f408564f726a",
   "metadata": {},
   "source": [
    "First, we organise the data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7e8b7a8c-92d2-465d-b451-269052d067df",
   "metadata": {},
   "outputs": [],
   "source": [
    "lc = {}\n",
    "psd = {}\n",
    "\n",
    "for f in glob.glob('prepared_data/variable_stars/*_lc.dat'):\n",
    "    tag = f.split(os.sep)[-1].split('_')[0]\n",
    "    lc[tag] = np.genfromtxt(f)\n",
    "    psd[tag] = np.genfromtxt(f'prepared_data/variable_stars/{tag}_psd.dat')  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "519d367d-4f2f-496d-ac70-bd1cb76f95d4",
   "metadata": {},
   "source": [
    "Define a useful function for plotting these lightcurves as we deal with them..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "516a9974-ad00-4afb-b65f-15efe856d9d0",
   "metadata": {},
   "outputs": [],
   "source": [
    "def show_lightcurve(name, plot_type='scatter', truncate_index=None): \n",
    "    x = lc[name][:N,0]-lc[name][:N,0].min()\n",
    "    y = lc[name][:N,1]\n",
    "\n",
    "    if plot_type == 'scatter':\n",
    "        plt.scatter(x, y, s=4)\n",
    "    if plot_type == 'line':\n",
    "        plt.plot(x, y)\n",
    "        \n",
    "    plt.ylabel('Flux')\n",
    "    plt.xlabel('Time [days]')\n",
    "    plt.title(name)\n",
    "    plt.show()\n",
    "    return x, y "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7e739c7-b829-47b2-a697-2857f758bce4",
   "metadata": {},
   "source": [
    "Show what targets we've got..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4ddbb919-3db7-4bc9-a6fa-79823918be20",
   "metadata": {},
   "outputs": [],
   "source": [
    "targets = list(psd.keys())\n",
    "print(targets)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b4da8405-6740-41f8-a900-9b8cacc01e99",
   "metadata": {},
   "source": [
    "...and pick which we're going to sonify using the different variations below"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "84279391-1161-4859-b2ab-5820375b8842",
   "metadata": {},
   "outputs": [],
   "source": [
    "targlist = ['55Cancri'] # Pick an object or iterate through all 'targets'  ✏️\n",
    "#targlist = targets"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "67174ec3-d9a9-472b-9a59-f991caf86dae",
   "metadata": {},
   "source": [
    "Now, we are ready iterate through target light-curves to sonify them, alongside their plots!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6c36fe5b-25c3-4f86-ae65-0ca472e2d802",
   "metadata": {},
   "source": [
    "## Section 2 - One-dimensional data series"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "323e690c-ab4d-40b4-b157-63a86329b0bf",
   "metadata": {},
   "source": [
    "Let's consider the simple case of one-dimensional data series. In these examples, we have light-curves and periodograms from a number of diverse variable stars. How can we go about sonifying them?\n",
    "\n",
    "Here we will sonify light-curves as a ***one-dimensional time series*** , where some ***sound property*** is is varied with ***time*** in the ***sonification***, in the same way that **flux**, varies with ***observation time*** in a ***light-curve*** (early in the sonification is shorter wavelengths, later is longer wavelengths)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db965de7-bc03-41bf-aa51-becb9bc03532",
   "metadata": {},
   "source": [
    "### Section 2.1 - Event sonification vs Scatter Plots"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98f1edb3-5ab8-473e-ad60-2bf6bfeb13cc",
   "metadata": {},
   "source": [
    "In `strauss` we could treat each ***flux density*** data point in the spectra as separate audio `Events`, with an occurence `time` mapped from their ***wavelength***.\n",
    "\n",
    "However, articulating each data point as a separate ***note*** for ***many thousands*** of data points can require long and drawn-out sonifications.\n",
    "\n",
    "Here we demonstrate this approach with just a portion of the data points (staying within `Colab`'s RAM limitations for unpaid users 🤫). We use the `Synthesizer` object with the `pitch_mapper` preset by default - this has a default pitch range of ***two octaves*** (a factor of 4 in frequency) and we pick an `E3` note (165 Hz) as the base (lowest) frequency.\n",
    "\n",
    "We will hear the **flux** of each point as `pitch`, with their observation mapped to the occurence `time` (moving from low to high wavelength). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d62e738f-bee3-40ae-a422-aebd4f8139a3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# set an index to trunate the lightcurve (if it's too long...)\n",
    "N = 150\n",
    "#N = None\n",
    "\n",
    "\n",
    "for trg in targlist: # iterate, if you like\n",
    "    \n",
    "    x,y = show_lightcurve(trg, plot_type='scatter', truncate_index=N)\n",
    "    \n",
    "    # specify the base notes used. In this example we use a single E3 note and\n",
    "    # freely vary the pitch via the 'pitch_shift' parameter\n",
    "    notes = [[\"E3\"]]\n",
    "    \n",
    "    # we could also just specify a particular frequency... ✏️\n",
    "    # notes = [[150.]]\n",
    "    \n",
    "    score =  Score(notes, 15)\n",
    "    \n",
    "    maps = {'pitch':np.ones(x.size),\n",
    "            'time': x,\n",
    "            'pitch_shift':y}\n",
    "    \n",
    "    # specify audio system (e.g. mono, stereo, 5.1, ...)\n",
    "    system = \"mono\"\n",
    "    \n",
    "    # set up synth (this generates the sound using mathematical waveforms)\n",
    "    generator = Synthesizer()\n",
    "    generator.load_preset('pitch_mapper')\n",
    "    \n",
    "    # or maybe the sampler instead by uncommenting this block (this uses recorded audio clips)\n",
    "    # generator = Sampler(sampfiles=\"./strauss/data/samples/mallets\")\n",
    "    \n",
    "    generator.modify_preset({'note_length':0.1,\n",
    "                             'volume_envelope': {'use':'on',\n",
    "                                                 # A,D,R values in seconds, S sustain fraction from 0-1 that note\n",
    "                                                 # will 'decay' to (after time A+D)\n",
    "                                                 'A':0.02,    # ✏️ for such a fast sequence, using ~10 ms values\n",
    "                                                 'D':0.02,    # ✏️ for such a fast sequence, using ~10 ms values\n",
    "                                                 'S':0.,      # ✏️ decay to volume 0\n",
    "                                                 'R':0.001}}) # ✏️ for such a fast sequence, using ~10 ms values\n",
    "    \n",
    "    # set 0 to 100 percentile limits so the full pitch range is used...\n",
    "    # setting 0 to 101 for pitch means the sonification is 1% longer than the final note position\n",
    "    lims = {'time': ('0','101'),\n",
    "            'pitch_shift': ('0','100')}\n",
    "    \n",
    "    # set up source\n",
    "    sources = Events(maps.keys())\n",
    "    sources.fromdict(maps)\n",
    "    sources.apply_mapping_functions(map_lims=lims)\n",
    "    \n",
    "    soni = Sonification(score, sources, generator, system)\n",
    "    soni.render()\n",
    "    # soni.save('pitchpm.wav')\n",
    "    dobj = soni.notebook_display(show_waveform=0);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9fde33ff-328e-4e57-be9a-741b2e01eda6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# set an index to trunate the lightcurve (if it's too long...)\n",
    "N = 950\n",
    "#N = None\n",
    "\n",
    "for trg in targlist: # iterate, if you like\n",
    "    \n",
    "    x,y = show_lightcurve(trg, plot_type='scatter', truncate_index=N)\n",
    "    \n",
    "    # specify the notes used.\n",
    "    # C major pentatonic\n",
    "    notes = [[\"C3\",\"E3\",\"F3\",\"G3\",\"B3\",\"C4\",\"E4\",\"F4\",\"G4\",\"B4\",\"C5\",\"E5\",\"F5\",\"G5\",\"B5\"]]\n",
    "    \n",
    "    # or a whole-tone scale, (was it just a dream... a dream... a dream...)\n",
    "    #\n",
    "    # This is a \"symmetrical\" scale and here also demonstrates using raw frequencies in Hz.\n",
    "    # The whole tone scale uses 2 semitone jumps, what about minor thirds (3) fourths (5)\n",
    "    # or fifths (7)? Specify `semitones` to change this.\n",
    "    # `nint` specifies the number of intervals in the scale. Note a bigger semitone interval\n",
    "    # will reach higher pitches (perhaps inaudible ones...) ✏️\n",
    "    \n",
    "    # semitones = 2.\n",
    "    # nint = 10\n",
    "    # notes = [100*2**(np.arange(nint)*(semitones/12))]\n",
    "    \n",
    "    # ------\n",
    "    # In this context, we also consider the `Score` optional parameter `pitch_binning`.\n",
    "    #\n",
    "    # This determines how we bin the 'pitch' quantity, which can be either:\n",
    "    # - 'uniform':  the range of mapped values for 'pitch' are split into even bins\n",
    "    #               representing  each of the scored notes\n",
    "    # - 'adaptive': the range of mapped values for 'pitch' split by percentile.\n",
    "    #               such that each interval is played approximately the same number of\n",
    "    #               times\n",
    "    # without specifying,  'adaptive' is used by default, but we specify 'uniform' here\n",
    "    #\n",
    "    # which is better for representing this data?\n",
    "    \n",
    "    # we specify 'uniform' pitch binning in the primary example... ✏️\n",
    "    score =  Score(notes, 15, pitch_binning='uniform')\n",
    "    \n",
    "    # what about adaptive?\n",
    "    #score =  Score(notes, 15, pitch_binning='adaptive')\n",
    "    \n",
    "    maps = {'pitch':y,\n",
    "            'time': x}\n",
    "    \n",
    "    # specify audio system (e.g. mono, stereo, 5.1, ...)\n",
    "    system = \"mono\"\n",
    "    \n",
    "    # set up synth (this generates the sound using mathematical waveforms)\n",
    "    generator = Synthesizer()\n",
    "    generator.load_preset('pitch_mapper')\n",
    "    \n",
    "    generator.modify_preset({'note_length':0.15,\n",
    "                             'volume_envelope': {'use':'on',\n",
    "                                                 # A,D,R values in seconds, S sustain fraction from 0-1 that note\n",
    "                                                 # will 'decay' to (after time A+D)\n",
    "                                                 'A':0.02,    # ✏️ for such a fast sequence, using ~10 ms values\n",
    "                                                 'D':0.02,    # ✏️ for such a fast sequence, using ~10 ms values\n",
    "                                                 'S':1.,      # ✏️ decay to volume 0\n",
    "                                                 'R':0.001}}) # ✏️ for such a fast sequence, using ~10 ms values\n",
    "    \n",
    "    # set 0 to 100 percentile limits so the full pitch range is used...\n",
    "    # setting 0 to 101 for pitch means the sonification is 1% longer than\n",
    "    # the time needed to trigger each note - by making this more than 100%\n",
    "    # we give all the notes time to ring out (setting this at 100% means\n",
    "    # the final note is triggered at the momement the sonification ends)\n",
    "    lims = {'time': ('0','101'),\n",
    "            'pitch_shift': ('0','100'),\n",
    "            'pitch': ('0','100')}\n",
    "    \n",
    "    # set up source\n",
    "    sources = Events(maps.keys())\n",
    "    sources.fromdict(maps)\n",
    "    sources.apply_mapping_functions(map_lims=lims)\n",
    "    \n",
    "    soni = Sonification(score, sources, generator, system)\n",
    "    soni.render()\n",
    "    dobj = soni.notebook_display(show_waveform=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a6cd57e-3b93-4cba-9f32-24a32ef4b13a",
   "metadata": {},
   "source": [
    "The articulation of each note lets you hear each data point separately, but over many noisy data points can lead to a confusing result - it's hard to keep track of the fluxes and our pitch memory is challenged.\n",
    "\n",
    "Instead we can try ***smoothly evolving*** parameters, designating the spectra as an ***evolving `Object`*** source class. We demonstrate this in the following subsection `3.1`."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d072603-8e70-4f71-90d0-60d5fb9b8d39",
   "metadata": {},
   "source": [
    "### Section 2.2: Object Sonification vs Line Plots"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "383b87c9-919d-4050-8816-0a195ee2d1ff",
   "metadata": {},
   "source": [
    "Let's first try the evolving Object approach without truncating the raw data in any way. In analogy to visual display, the Event representation is like plotting the spectrum as a scatter plot, while an Object representation is like plotting the spectrum as a continuous line.\n",
    "\n",
    "let's hear the 1D time-series data sonification, mapping flux density to pitch for the spectrum.\n",
    "\n",
    "*NB: If you were wondering why `strauss` refers to the varying pitch mapping as `pitch_shift` and not just `pitch`, this is because all sources in `strauss` also need a base `pitch` which is chosen from the `'notes'` structure (here always `'A2'`) by the `Score`. This is because `strauss` can play many sources at the same time! Again, you can read more about this [in the docs](https://strauss.readthedocs.io/en/latest/).*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "35280f38-553e-42a2-8f99-d00ae4f12dad",
   "metadata": {},
   "outputs": [],
   "source": [
    "# # show spectrum again, for reference\n",
    "# display(Image(spectra_plots[stype][slabel]))\n",
    "\n",
    "for trg in targlist: # iterate, if you like  \n",
    "    # set an index to truncate the lightcurve (if it's too long...)\n",
    "    N = None\n",
    "    x,y = show_lightcurve(trg, plot_type='line')\n",
    "\n",
    "    notes = [[\"A2\"]]\n",
    "    score =  Score(notes, 15)\n",
    "    \n",
    "    # set up synth (this generates the sound using mathematical waveforms)\n",
    "    generator = Synthesizer()\n",
    "    generator.preset_details('pitch_mapper')\n",
    "    generator.load_preset('pitch_mapper')\n",
    "    \n",
    "    \n",
    "    data = {'pitch':1.,\n",
    "            'time_evo':x,\n",
    "            'pitch_shift':y}\n",
    "    \n",
    "    # set 0 to 100 percentile limits so the full pitch and time range is used...\n",
    "    lims = {'time_evo': ('0','100'),\n",
    "            'pitch_shift': ('0','100')}\n",
    "    \n",
    "    # set up source\n",
    "    sources = Objects(data.keys())\n",
    "    sources.fromdict(data)\n",
    "    sources.apply_mapping_functions(map_lims=lims)\n",
    "    \n",
    "    soni = Sonification(score, sources, generator, system)\n",
    "    soni.render()\n",
    "    dobj = soni.notebook_display(show_waveform=0);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0ef52d6c-ce73-4c5a-a33a-4500baedb4ea",
   "metadata": {},
   "source": [
    "How about mapping a low-pass filter to flux density?\n",
    "\n",
    "*using a 'tonal' carrier, uncomment line for a 'textural' (or white noise) carrier*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e0a41bd4-67ed-4b97-8ffb-88848d5359dd",
   "metadata": {},
   "outputs": [],
   "source": [
    "for trg in targlist: # iterate, if you like\n",
    "\n",
    "    # set an index to trucnate the lightcurve (if it's too long...)\n",
    "    N = None\n",
    "    x,y = show_lightcurve(trg, plot_type='line', truncate_index=N)\n",
    "    \n",
    "    generator = Synthesizer()\n",
    "    generator.modify_preset({'filter':'on'})\n",
    "    \n",
    "    # uncomment these lines to try a 'textural' sonification using white noise! ✏️\n",
    "    # generator.load_preset('windy')\n",
    "    # generator.preset_details('windy')\n",
    "    \n",
    "    # we use a 'chord' here to create more harmonic richness (stacking fifths)...\n",
    "    notes = [[\"A2\", \"E3\", 'B3', 'F#4']]\n",
    "    score =  Score(notes, 15)\n",
    "    \n",
    "    data = {'pitch':[0,1,2,3],\n",
    "            'time_evo':[x]*4,\n",
    "            'cutoff':[y]*4}\n",
    "    \n",
    "    lims = {'time_evo': ('0','100'),\n",
    "            'cutoff': ('0','100')}\n",
    "    \n",
    "    # set up source\n",
    "    sources = Objects(data.keys())\n",
    "    sources.fromdict(data)\n",
    "    plims = {'cutoff': (0.15,0.95)}\n",
    "    sources.apply_mapping_functions(map_lims=lims, param_lims=plims)\n",
    "    \n",
    "    soni = Sonification(score, sources, generator, system)\n",
    "    soni.render()\n",
    "    dobj = soni.notebook_display(show_waveform=0);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f9ff41c-108f-4e21-84cd-2a83d75a45b8",
   "metadata": {},
   "source": [
    "In fact, there are many expressive properties of sound we could use to represent the data in a similar way. In `strauss` these are referred to as `mappable` properties.\n",
    "\n",
    "A subset of these can be used as an evolving property with the `Object` source class. These are referred to as `evolvable` properties.\n",
    "\n",
    "Lets show what's available:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "75277149-0ce4-483a-b8d3-8f22416cd267",
   "metadata": {},
   "outputs": [],
   "source": [
    "display(Markdown(f\"### ***'Mappable'*** properties:\"))\n",
    "for m in Sources.mappable:\n",
    "  display(Markdown(f' * `{m}` '))\n",
    "\n",
    "display(Markdown(f\"### ***'Evolvable'*** properties:\"))\n",
    "for m in Sources.evolvable:\n",
    "  display(Markdown(f' * `{m}` '))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14e1aec6-7949-4228-adb0-618292c0d8e1",
   "metadata": {},
   "source": [
    "We can play around with some of these `evolvable` properties here. Are all of these effective for this data? Could they be effective for other types of data representations?\n",
    "\n",
    "The `idx` variable below controls which evolvable property is selected from the `some_mappings` list. `idx` can be changed to a number from 0 to 5 inclusive to choose a certain sound parameter from the list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e8fa0d51-79a1-4db9-badf-fdb3f18e3bd0",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "for trg in targlist: # iterate, if you like\n",
    "\n",
    "    # set an index to trunate the lightcurve (if it's too long...)\n",
    "    N = None\n",
    "    x,y = show_lightcurve(trg, plot_type='line', truncate_index=N)\n",
    "    \n",
    "    # A list of some 'evolvable' mappings\n",
    "    some_mappings = [\"volume\",\n",
    "                     \"azimuth\",\n",
    "                     \"volume_lfo/amount\",\n",
    "                     \"volume_lfo/freq_shift\",\n",
    "                     \"pitch_lfo/amount\",\n",
    "                     \"pitch_lfo/freq_shift\"]\n",
    "    \n",
    "    # change this (between 0 and 5) to select a different property to map...\n",
    "    idx = 3\n",
    "    \n",
    "    # use a stereo system to allow 'phi' mapping (low pan left and high pan right)\n",
    "    system = \"stereo\"\n",
    "    \n",
    "    display(Markdown(f\"### Sonifying in 1D using `evolvable` property - ***`{some_mappings[idx]}`***:\"))\n",
    "    \n",
    "    generator = Synthesizer()\n",
    "    generator.modify_preset({'filter':'on',\n",
    "                             \"pitch_hi\":-1, \"pitch_lo\": 1,\n",
    "                             \"pitch_lfo\": {\"use\": \"on\",\n",
    "                                           \"amount\":1*(\"pitch_lfo/freq_shift\" == some_mappings[idx]),\n",
    "                                           \"freq\":3*5**(\"pitch_lfo/freq_shift\" != some_mappings[idx]),\n",
    "                                           \"phase\":0.25},\n",
    "                             \"volume_lfo\": {\"use\": \"on\",\n",
    "                                            \"amount\":1*(\"volume_lfo/freq_shift\" == some_mappings[idx]),\n",
    "                                            \"freq\":3*5**(\"volume_lfo/freq_shift\" != some_mappings[idx]),\n",
    "                                            \"phase\":0}\n",
    "                            })\n",
    "    \n",
    "    # try a different chord (stacking fifths)...\n",
    "    notes = [[\"A2\",\"E3\",\"B4\",\"F#4\"]]\n",
    "    \n",
    "    # chords and can also be specified via chord names and base octave....\n",
    "    # notes = \"Em6_3\"\n",
    "    \n",
    "    \n",
    "    score =  Score(notes, 15)\n",
    "    \n",
    "    data = {'pitch':[0,1,2,3],\n",
    "            'time_evo':[x]*4,\n",
    "            'cutoff':[0.9]*4,\n",
    "            'polar':[0.5]*4,\n",
    "            some_mappings[idx]:[(y-y.min())/(y.max()-y.min())]*4}\n",
    "    \n",
    "    lims = {'time_evo': ('0','100'),\n",
    "            \"volume\": ('0','100'),\n",
    "            \"azimuth\": (-0.5,1.5),\n",
    "            \"volume_lfo/amount\": ('0','100'),\n",
    "            \"volume_lfo/freq_shift\": ('0','100'),\n",
    "            \"pitch_lfo/amount\": ('0','100'),\n",
    "            \"pitch_lfo/freq_shift\": ('0','100')}\n",
    "    \n",
    "    # set up source\n",
    "    sources = Objects(data.keys())\n",
    "    sources.fromdict(data)\n",
    "    plims = {'cutoff': (0.25,0.9)}\n",
    "    sources.apply_mapping_functions(map_lims=lims, param_lims=plims)\n",
    "    \n",
    "    soni = Sonification(score, sources, generator, system)\n",
    "    soni.render()\n",
    "    dobj = soni.notebook_display(show_waveform=0.);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5928384e-7883-4a48-bd70-3d5372abdae4",
   "metadata": {},
   "source": [
    "## 4. Sonification: Spectral Representation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6dcd5691-b832-433c-b646-428d5a79bafe",
   "metadata": {},
   "source": [
    "Having covered some of these more abstract approaches, let's consider a direct _Spectral Audification_ or _\"Spectralisation\"_"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "caeb1992-b516-40fb-b411-94c62dc6abd7",
   "metadata": {},
   "source": [
    "We will try converting the ***periodogram*** into a ***sound spectrum*** directly, i.e. the sound has the same frequency features as the observed light curves, albeit at ***audible frequencies***"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f3a03638-39ab-4f65-8e03-ade650f8d2fd",
   "metadata": {},
   "outputs": [],
   "source": [
    "for c in targlist:\n",
    "\n",
    "    fmin = 0.2\n",
    "    fmax = 20  \n",
    "    \n",
    "    freqs = psd[c][:,0].copy()\n",
    "    fsel = np.logical_and(freqs < fmax, freqs > fmin) \n",
    "    powers = psd[c][:,1][fsel]\n",
    "    freqs = freqs[fsel]\n",
    "\n",
    "    plt.plot(freqs, powers)\n",
    "    plt.xlabel(r'Frequency [days$^{-1}$]')\n",
    "    plt.ylabel('Power')\n",
    "    plt.title(c)\n",
    "    plt.show() \n",
    "\n",
    "    dynrange = freqs.max()/freqs.min()\n",
    "    \n",
    "    # set up spectralizer generator\n",
    "    generator = Spectralizer()\n",
    "    \n",
    "    # Lets pick the mapping frequency range for the spectrum... ✏️\n",
    "    generator.modify_preset({'min_freq':100, 'max_freq':100*dynrange})\n",
    "    \n",
    "    score =  Score([['A2']], 10)\n",
    "    \n",
    "    # set up spectrum and choose some envelope parameters for fade-in and fade-out\n",
    "    maps = {'spectrum': [powers], 'pitch':[1],\n",
    "            'volume_envelope/D':[0.8],\n",
    "            'volume_envelope/S':[0.],\n",
    "            'volume_envelope/A':[0.01]}\n",
    "    \n",
    "    # again, use maximal range for the mapped parameters\n",
    "    lims = {'spectrum': ('0','100')}\n",
    "\n",
    "\n",
    "    # set up source\n",
    "    sources = Events(maps.keys())\n",
    "    sources.fromdict(maps)\n",
    "    sources.apply_mapping_functions(map_lims=lims)\n",
    "    \n",
    "    # render and play sonification!\n",
    "    soni = Sonification(score, sources, generator, system)\n",
    "    soni.render()\n",
    "    dobj = soni.notebook_display(show_waveform=0);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9e861a6-0aac-4ce8-a3c2-d3faf46e8f5c",
   "metadata": {},
   "source": [
    "Can you hear how all these lightcurves give a different _\"timbre\"_ or _\"formant\"_?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0fd07798-af29-40dd-96b8-8e82f7127001",
   "metadata": {},
   "source": [
    "For some more context, we take the _\"55-Cancri\"_ curve in particular.\n",
    "\n",
    "Let's isolate the fundamental frequency of the short period variation (rapid dips) and some of it's overtones:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8ab07e1a-3146-43cd-91ad-96d019d1c483",
   "metadata": {},
   "outputs": [],
   "source": [
    "targ = '55Cancri'\n",
    "\n",
    "fmin = 0.2\n",
    "fmax = 20 \n",
    "\n",
    "freqs = psd[targ][:,0].copy()\n",
    "fsel = np.logical_and(freqs < fmax, freqs > fmin) \n",
    "powers = psd[targ][:,1][fsel]\n",
    "freqs = freqs[fsel]\n",
    "    \n",
    "plt.plot(freqs*(100/freqs.min()), powers)\n",
    "plt.title(targ)\n",
    "plt.xlim(100, 100*dynrange)\n",
    "# plt.axvline(115)\n",
    "\n",
    "harmonics = [660]\n",
    "cdx = 0\n",
    "for h in harmonics:\n",
    "    cdx +=1\n",
    "    for k in range(1,12):\n",
    "        plt.axvline(h*k, c=f\"C{cdx}\",ls=':')\n",
    "plt.xlabel(\"Spectral Sonification Frequency distribution [Hz]\")\n",
    "plt.ylabel(\"Power\")\n",
    "# plt.semilogx()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "489a5edc-fe86-4d36-b9b9-495f83304e44",
   "metadata": {},
   "source": [
    "Turns out the dominant fundamental in _55 Cancri's_ periodogram is sonified at around 660 Hz, or an 'E5' in musical notation, lets hear that for comparison... "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7aca1e58-e9cf-4f48-8b28-dfa6ed8be991",
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "score =  Score([['E5']], 2)\n",
    "#score =  Score([[660.]], 2)\n",
    "\n",
    "maps = {'pitch': [1]}\n",
    "\n",
    "# specify audio system (e.g. mono, stereo, 5.1, ...)\n",
    "system = \"mono\"\n",
    "\n",
    "# set up synth (this generates the sound using mathematical waveforms)\n",
    "generator = Synthesizer()\n",
    "\n",
    "generator.load_preset('pitch_mapper')\n",
    "generator.modify_preset({\"volume_envelope\": {\"use\": \"on\", \"S\": 0., \"D\": 2}\n",
    "                        })\n",
    "\n",
    "# set up source\n",
    "sources = Objects(maps.keys())\n",
    "sources.fromdict(maps)\n",
    "sources.apply_mapping_functions(map_lims=lims)\n",
    "\n",
    "# render and play sonification!\n",
    "soni = Sonification(score, sources, generator, system)\n",
    "soni.render()\n",
    "\n",
    "soni.notebook_display(show_waveform=False);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0fd07798-af29-40dd-96b8-8e82f7127001",
   "metadata": {},
   "source": [
    "More Spectralisation can be found in the _Bonus Material_."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8f93a2ce-70a5-4d7e-994a-a8c5386bef49",
   "metadata": {},
   "source": [
    "# Bonus Material!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "79b14208-caa5-4dcc-867d-4c5ee0a937f5",
   "metadata": {},
   "source": [
    "Here's some additional examples to explore if you're interested!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b9291b21-10ac-473e-8171-93b9608ce02b",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "## A. \"Stars Appearing\" from the _Tour of the Universe_ planetarium show"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e2a74bc6-5c4a-4c7f-810c-cec590f6e7be",
   "metadata": {},
   "source": [
    "<u> __The Score:__ </u>\n",
    "\n",
    "We set up the ***Score***; this is analagous to a musical score and controls what notes can be played over the course of the sonification. We can specify a chord sequence as a single string (`str`) where chord names are separated by a `|` character. The root octave of the chord may also be specified by adding `_X` where `X` is the octave number. <span style=\"color:gray\">_(Note: for now, each chord occupies an equal lenth in the sonification, in the future chord change times can be directly specified and optionally related to events in the data)_</span>\n",
    "\n",
    "We can directly specify the ___chord voicing___ as \n",
    "a list of lists containing the notes from low to high in each chord.\n",
    "\n",
    "Here, we  are directly specifying a single __`Db6/9`__ chord voicing. These notes will later be played by stars of different colours!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cd1ef56e-4571-4ebb-9f41-cd5b671e3502",
   "metadata": {},
   "outputs": [],
   "source": [
    "chords = [['Db3','Gb3', 'Ab3', 'Eb4','F4']]\n",
    "length = \"1m 30s\"\n",
    "score =  Score(chords, length)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e7447211-2fd4-4e5a-b537-8af2094635fe",
   "metadata": {},
   "source": [
    "<u> __The Sources:__ </u>\n",
    "\n",
    "Next, we import the data that will represent ___sources___ of sound. The data is the sky positions, brightness and colour of stars from the _Paranal_ observatory site in _Chile_. This data is contained in an `ascii` file (specified by the `datafile` variable) and organised where each __row__ is a star and each __column__ is a property of that star.\n",
    "\n",
    "The idea here is that as night draws in we see the brightet stars first. As it gets darker, and our eyes adjust, we see more dim stars. We \"***sonify***\" this by having a note play when each star appears. The \"***panning***\" (a.k.a stereo imaging) of the note is controlled by the ***altitude*** and ***azimuth*** of the stars, as if we were facing south. The ***colour*** of the star contols the note within the chord we've chosen, where notes low to high (short to long wavelength) represent fixed-number bins in colour from blue to red (again, short to long wavelength). Finally, the volume of the note is also related to the brightness of the star (dimmer stars are quieter). This is chosen to give a relatively even volume throught the sonification, as dim stars are much more numerous than bright ones <span style=\"color:gray\">_(Note: in the future we will have the option to scale volumes in this way automatically)_</span>\n",
    "\n",
    "We speciify the sound preperty to star property mappimg as as three `dict` objects with keys representing each sound property we dapat in the sonification:\n",
    "- **`mapcols`**: entries are the data file columns used to map each property\n",
    "- **`mapvals`**: entries are function objects that manipulate each columns values to yield the linear mapping\n",
    "- **`maplims`**: entries are `tuple`s representing the (`low`,`high`) limits of each mapping.numerical values represent absolute limits (used here for the angles in degrees to correctly limit the `azimuth` and `polar` mappings to 360° (2π) and 180° (π) respectively. `str` values are taken to be percentiles from 0 to 100. string values > 100 can also be used, where e.g. 104 is 4% larger than the 100th percentile value. This is used for the time here, so that the last sample doesnt trigger at exactly the end of the sonification, giving time for the sound to die away slowly."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "89ff0188-7ab5-40a8-96bc-bc34c477fd04",
   "metadata": {},
   "outputs": [],
   "source": [
    "datafile = \"./strauss/data/datasets/stars_paranal.txt\"\n",
    "mapcols =  {'azimuth':1, 'polar':0, 'volume':2, 'time':2, 'pitch':3}\n",
    "\n",
    "mapvals =  {'azimuth': lambda x : x,\n",
    "            'polar': lambda x : 90.-x,\n",
    "            'time': lambda x : x,\n",
    "            'pitch' : lambda x: -x,\n",
    "            'volume' : lambda x : (1+np.argsort(x).astype(float))**-0.2}\n",
    "\n",
    "maplims =  {'azimuth': (0, 360),\n",
    "            'polar': (0, 180), \n",
    "            'time': ('0', '104'),\n",
    "            'pitch' : ('0', '100'),\n",
    "            'volume' : ('0', '100')}\n",
    "\n",
    "events = Events(mapcols.keys())\n",
    "events.fromfile(datafile, mapcols)\n",
    "events.apply_mapping_functions(mapvals, maplims)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ac1979d-bd86-4a9c-a672-a9c4ec7eac63",
   "metadata": {},
   "source": [
    "<u> __The Generator:__ </u>\n",
    "\n",
    "The final element we need is a ***Generator*** that actually generates the audio given the ***Score*** and ***Sources***. Here, we use a ***Sampler***-type generator that plays an audio sample for each note. The samples and other parameters (not specified here) control the sound for each note. These can be specified in `dict` format note-by-note (keys are note name strings, entries are strings pointing to the `WAV` format audio sample to load) or just using a string that points to a sample directory (each sample filename in that directory ends with `_XX.wav` where `XX` is the note name) we use the example sample back in `./data/samples/glockenspiels` <span style=\"color:gray\">_(Note: rendering can take a while with the long audio samples we use here, shorter samples can be used to render faster, such as those in `./data/samples/mallets`. This is also useful if you want to try different notes or chords, as only the 5 notes specified above are provided in the glockenspiel sample folder.)_</span>\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "67680b16-0520-4f33-81b8-b65184e77d2d",
   "metadata": {},
   "outputs": [],
   "source": [
    "sampler = Sampler(\"./strauss/data/samples/glockenspiels\")\n",
    "sampler.preset_details(\"default\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6009fdca-8945-4978-870c-6899f2fc9e49",
   "metadata": {},
   "source": [
    "<u> __The Sonification:__ </u>\n",
    "\n",
    "We consolidate the three elements above in to a sonification object to generate the sound, specifying the audio setup (here `'stereo'` as opposed to `'mono'`, `'5.1'`, etc). <span style=\"color:gray\">_(Note: you can generate the audio in any specified audio setup, but following cells assume stereo and only mono and stereo formats are supported by the jupyter audio player in the final cell)_</span>\n",
    "\n",
    "We then `render` the sonification to generate the audio track (may take some time with the glockenspiel samples)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b27e215f-a511-4b27-a36b-f815eda0dc5b",
   "metadata": {},
   "outputs": [],
   "source": [
    "system = \"stereo\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2af40400-484b-4b1d-8d6d-be5292b87ca2",
   "metadata": {},
   "outputs": [],
   "source": [
    "soni = Sonification(score, events, sampler, system)\n",
    "soni.render()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92138d06-6191-4627-b64b-29f10406e2fe",
   "metadata": {},
   "source": [
    "Finally, let's visualise the waveform, and preview the audio in-notebook*!\n",
    "\n",
    "<span style=\"color:gray\">_*if using a surround sound format (i.e > 2 channels) the preview is stereo, with the first two channels mapped left and right, due to the limitations of the notebook audio player_</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4c2efdcc-fe71-4e02-a1c6-499204618d82",
   "metadata": {},
   "outputs": [],
   "source": [
    "soni.notebook_display()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9df5cd97-4724-4d19-9d17-967542d56ea1",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "## B. Bivariate Sonification with EAGLE star formation histories"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "416088d6-efc9-40fd-a28e-55600094d165",
   "metadata": {},
   "source": [
    "Lets look at these multivariate data sets. These are star-formation and metal-enrichment histories of 5 galaxies from the [EAGLE simulations](https://eagle.strw.leidenuniv.nl/) (virtual galaxies, modelled on a supercomputer) \n",
    "\n",
    "We see that we have two time-dependent quantities: The rate at which stars form (aka ***SFR***, in solar masses per year) and the mass fraction of those stars in elements heavier than Helium (aka ***Metallicity*** or $Z$) \n",
    "\n",
    "Let's take a look..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e0f17d33-9ec9-4df6-bdf4-62742271ce10",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "for f in glob.glob('./prepared_data/eagle_galaxies/*.txt'):\n",
    "  data_table = pd.read_csv(f)\n",
    "  display(Markdown(f\"### File: <mark>`{f}`</mark>\"))\n",
    "  xlab = data_table.columns[0]\n",
    "  ylab = data_table.columns[1]\n",
    "  zlab = data_table.columns[2]\n",
    "\n",
    "# show the last-loaded SFHs\n",
    "plt.plot(data_table[xlab], data_table[ylab])\n",
    "plt.xlabel(f'{xlab}')\n",
    "plt.ylabel(f'{ylab}')\n",
    "plt.show()\n",
    "plt.plot(data_table[xlab], data_table[zlab])\n",
    "plt.xlabel(f'{xlab}')\n",
    "plt.ylabel(f'{zlab}')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c9fab20c-223c-4754-8180-f51b0d07dd2e",
   "metadata": {},
   "source": [
    "Now let's choose some combination of evolvable parameters!\n",
    "\n",
    "Now there are 2 properties being sonified at once (not including the time variable), there are 28 possible combinations of these mappings $\\left(\\frac{8!}{2!(8-2)!} = 28\\right)$ \n",
    "\n",
    "Some combinations might work better than others..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c613ba9c-cfc0-48e4-9b7a-c7358e6475dc",
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "# A list of some 'evolvable' mappings\n",
    "some_mappings = [\"pitch_shift\",\n",
    "                 \"cutoff\",\n",
    "                 \"volume\",\n",
    "                 \"phi\",\n",
    "                 \"volume_lfo/amount\", \n",
    "                 \"volume_lfo/freq_shift\",\n",
    "                 \"pitch_lfo/amount\", \n",
    "                 \"pitch_lfo/freq_shift\"]\n",
    "\n",
    "# !! change these (between 0 and 7) to select a different property to map... \n",
    "idx_sfr = 1\n",
    "idx_Z = 5\n",
    "\n",
    "chosen = [some_mappings[idx_Z],some_mappings[idx_sfr]]\n",
    "\n",
    "# use a stereo system to allow 'phi' mapping (low pan left and high pan right)\n",
    "system = \"stereo\"\n",
    "\n",
    "# some strauss setup can happen outside the loop...\n",
    "generator = Synthesizer()\n",
    "\n",
    "generator = Synthesizer()\n",
    "generator.modify_preset({'filter':'on',\n",
    "                         \"pitch_hi\":-1, \"pitch_lo\": 1,\n",
    "                         \"pitch_lfo\": {\"use\": \"on\", \n",
    "                                       \"amount\":1*(\"pitch_lfo/freq_shift\" in chosen), \n",
    "                                       \"freq\":3*5**(\"pitch_lfo/freq_shift\" not in chosen), \n",
    "                                       \"phase\":0.25},\n",
    "                         \"volume_lfo\": {\"use\": \"on\", \n",
    "                                        \"amount\":1*(\"volume_lfo/freq_shift\" in chosen), \n",
    "                                        \"freq\":3*5**(\"volume_lfo/freq_shift\" not in chosen), \n",
    "                                        \"phase\":0}\n",
    "                        })\n",
    "\n",
    "\n",
    "notes = [[\"A2\", \"E3\"]]\n",
    "score =  Score(notes, 30)\n",
    "\n",
    "lims = {'time_evo': ('0','100'),\n",
    "        'pitch_shift': ('0','100'),\n",
    "        'cutoff': ('0','100'),\n",
    "        \"volume\": ('0','100'), \n",
    "        \"phi\": (-0.5,1.5),\n",
    "        \"volume_lfo/amount\": ('0','100'), \n",
    "        \"volume_lfo/freq_shift\": ('0','100'),\n",
    "        \"pitch_lfo/amount\": ('0','100'), \n",
    "        \"pitch_lfo/freq_shift\": ('0','100')}\n",
    "\n",
    "display(Markdown(f\"## Mapping SFR to <mark>***`{some_mappings[idx_sfr]}`***</mark> and Metallicity to <mark>***`{some_mappings[idx_Z]}`***</mark>:\"))\n",
    "\n",
    "for f in glob.glob('./prepared_data/eagle_galaxies/*.txt'):\n",
    "\n",
    "  display(Markdown(f\"### EAGLE Galaxy <mark>***`{f.split('.')[1].split('_')[-1]}`</mark>***:\"))\n",
    "\n",
    "  # get data\n",
    "  data_table = np.genfromtxt(f, delimiter=',')[1:]\n",
    "  time = data_table[:,0]\n",
    "  sfr = data_table[:,1]\n",
    "  Z = data_table[:,2]\n",
    "\n",
    "  # plot multivariate data ------------------------ \n",
    "  plt.plot(time,sfr)\n",
    "  plt.xlabel(f'Time [Gyr]')\n",
    "  plt.ylabel(f'Star Formation Rate [Msun/year]')\n",
    "  plt.gca().twinx()\n",
    "  plt.plot([0],[0], label='Star Formation Rate') # dummy plot to make legend work\n",
    "  plt.plot(time,Z,c='C1',label=\"Stellar Metallicity\")\n",
    "  plt.xlabel(f'Time [Gyr]')\n",
    "  plt.ylabel(f'Metallicity')\n",
    "  plt.legend(frameon=0)\n",
    "  plt.show();\n",
    "  # ------------------------------------------------ \n",
    "\n",
    "  data = {'pitch':[0,1],\n",
    "          'time_evo':[time]*2,\n",
    "          'theta':[0.5]*2,\n",
    "          some_mappings[idx_sfr]:[(sfr - sfr.min())/(sfr.max()-sfr.min())]*2,\n",
    "          some_mappings[idx_Z]:[(Z - Z.min())/(Z.max()-Z.min())]*2}\n",
    "\n",
    "  # set up source\n",
    "  sources = Objects(data.keys())\n",
    "  sources.fromdict(data)\n",
    "  plims = {'cutoff': (0.4,1)}\n",
    "  sources.apply_mapping_functions(map_lims=lims, param_lims=plims)\n",
    "\n",
    "  soni = Sonification(score, sources, generator, system)\n",
    "  soni.render()\n",
    "  dobj = soni.notebook_display(show_waveform=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8a744c1e-ef70-4b2d-be70-a1031a27bfad",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true
   },
   "source": [
    "## C. More _Spectralising_ : Planetary Nebulae and the old STRAUSS logo"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "acf9f005-c9fa-467e-91f1-d652aba76ce4",
   "metadata": {},
   "source": [
    "In other examples we use a 'parameter mapping' approach for one-dimensional data series, where we map _y_ as a function of _x_ using the change in some expressive property of sound (e.g. `pitch_shift`) as a function of time.\n",
    "\n",
    "We consider a direct spectralisation approach where the sopund is generated by treating th 1D data as a sound spectrum! This uses a direct inverse Fourier transform.This seems relatively intuitive for spectral data, particular those with spectral features similar to what we can identify in sound.\n",
    "\n",
    "We will use Planetary Nebulae (PNe) data to demonstrate this, objects that are dominated by strong emission lines."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7c93a86-cfa5-4ba5-ba35-1ec0c9461934",
   "metadata": {},
   "source": [
    "**First, let's grab some data...**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "62b9dbcc-97e5-4743-81fd-5a5e974bf710",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "spectral_data1 = np.genfromtxt('./strauss/data/datasets/NGC1535.csv', delimiter=',')\n",
    "wlen1 = spectral_data1[:,0]\n",
    "fluxdens1 = spectral_data1[:,1]\n",
    "\n",
    "spectral_data2 = np.genfromtxt('./strauss/data/datasets/NGC6302.csv', delimiter=',')\n",
    "wlen2 = spectral_data2[:,0]\n",
    "fluxdens2 = spectral_data2[:,1]\n",
    "\n",
    "# spectrum needs to be provided to the Spectraliser in frequency order (i.e. low to high), \n",
    "# so we ensure it is sorted that way... \n",
    "spec1 = fluxdens1[np.argsort(1/wlen1)]\n",
    "spec2 = fluxdens2[np.argsort(1/wlen2)]\n",
    "wlen1 = wlen1[np.argsort(1/wlen1)]\n",
    "wlen2 = wlen2[np.argsort(1/wlen2)]\n",
    "\n",
    "# plot the spectra vs wavlength\n",
    "plt.plot(wlen1, spec1/spec1.max(), label= 'NGC1535')\n",
    "plt.plot(wlen2, spec2/spec2.max(), alpha=0.6, label= 'NGC6302')\n",
    "plt.xlabel('Wavelength [Angstrom]')\n",
    "plt.ylabel('Flux')\n",
    "plt.legend(frameon=False)\n",
    "plt.show()\n",
    "\n",
    "# plot the spectra vs frequency\n",
    "plt.plot(1e-12*3e8/(wlen1*1e-10), spec1/spec1.max(),label= 'NGC1535')\n",
    "plt.plot(1e-12*3e8/(wlen2*1e-10), spec2/spec2.max(), alpha=0.6,label= 'NGC1535')\n",
    "plt.xlabel('Frequency [THz]')\n",
    "plt.ylabel('Flux')\n",
    "plt.legend(frameon=False)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed54c716-efa0-4d32-8f1d-6e2a6568e985",
   "metadata": {},
   "source": [
    "**Set up some universal sonification parameters and classes for the examples below**\n",
    "\n",
    "For all examples we use the `Synthesizer` generator to create a 30 second, mono sonification."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c313727a-b3fe-408c-a61a-a80a71bf7a89",
   "metadata": {},
   "outputs": [],
   "source": [
    "# specify audio system (e.g. mono, stereo, 5.1, ...)\n",
    "system = \"stereo\"\n",
    "\n",
    "# length of the sonification in s\n",
    "length = 10."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "859560ff-77da-4932-a225-4d01131f762d",
   "metadata": {},
   "source": [
    "### <u>Example 1</u> &nbsp; **Comparing the NGC 1535 & NGC 6302 Spectra**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7b833189-7bb3-4dc2-8ef7-a4e9d5ca4649",
   "metadata": {},
   "source": [
    "Lets compare the _Spectraliser_ representations of these two spectra"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6c7e1d8d-6af4-464b-98be-208c85f18f82",
   "metadata": {},
   "outputs": [],
   "source": [
    "notes = [[\"A2\"]]\n",
    "\n",
    "score =  Score(notes, length)\n",
    "\n",
    "spectra = [spec1, spec2]\n",
    "wlens = [wlen1, wlen2]\n",
    "names = ['NGC 1535', 'NGC 6302']\n",
    "\n",
    "for i in range(2):\n",
    "\n",
    "    #set up spectralizer generator\n",
    "    generator = Spectralizer()\n",
    "\n",
    "    # Lets pick the mapping frequency range for the spectrum...\n",
    "    generator.modify_preset({'min_freq':100, 'max_freq':1000})\n",
    "\n",
    "    s = np.zeros(spec1.size)\n",
    "    s[-1] = 1\n",
    "    # set up spectrum and choose some envelope parameters for fade-in and fade-out\n",
    "    data = {'spectrum':[spectra[i]], 'pitch':[1],\n",
    "            'volume_envelope/D':[0.9], \n",
    "            'volume_envelope/S':[0.], \n",
    "            'volume_envelope/A':[0.05]}\n",
    "    \n",
    "    # again, use maximal range for the mapped parameters\n",
    "    lims = {'spectrum': ('0','100')}\n",
    "    \n",
    "    # set up source\n",
    "    sources = Events(data.keys())\n",
    "    sources.fromdict(data)\n",
    "    sources.apply_mapping_functions(map_lims=lims)\n",
    "    \n",
    "    # render and play sonification!\n",
    "    soni = Sonification(score, sources, generator, system)\n",
    "    soni.render()\n",
    "    print(f\"Spectralising {names[i]}...\")\n",
    "    plt.plot(1e-12*3e8/(wlens[i]*1e-10), spectra[i]/spectra[i].max(), alpha=0.7,label=names[i])\n",
    "    soni.notebook_display(show_waveform=0)\n",
    "plt.xlabel('Frequency [THz]')\n",
    "plt.ylabel('Flux')\n",
    "plt.legend(frameon=False)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e58e6c8c-33b9-40d3-88d6-78a870accdf2",
   "metadata": {},
   "source": [
    "What differences do you notice about the sounds? Can you here the presence/absence of spectral lines, and their relative pitches?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd259ade-d486-40b4-b077-f07ec58819b7",
   "metadata": {},
   "source": [
    "### <u>Example 2</u> &nbsp; **Evolving Spectra and Image Sonification**"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8860ef8b-e2dd-4409-ad4d-e057717084d9",
   "metadata": {},
   "source": [
    "We could also perform a `Object` type sonification with an evolving Spectrum. \n",
    "\n",
    "An evolving spectrum can be represented as a 2D array, similar to a regular image. Using this similarity, the `Spectraliser` provides a neat way to sonify images!\n",
    "\n",
    "Here we sonify the `strauss` logo, lets grab it..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "62cdae98-5321-45a0-8bbd-68c5c81a8c2a",
   "metadata": {},
   "outputs": [],
   "source": [
    "image = plt.imread('./strauss/misc/strauss_logo.png')\n",
    "image = image[:,:,:-1].sum(axis=-1)\n",
    "image_inv = 1-image\n",
    "plt.imshow(image_inv, cmap='gray_r')\n",
    "plt.axis('off')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4e3dac2f-a6b5-4be0-a46a-b0ef12283c71",
   "metadata": {},
   "source": [
    "in `strauss` each row represents a spectrum, ordered from first to last.\n",
    "\n",
    "Convention to represent the image is to evolve from left to right, with higher features in the _y_-axis sounding higher pitch. Due to image formatting conventions being different, we need transpse and flip the image array to get the right format for `strauss`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "20e0f6a2-d752-426f-af83-26f6075d0819",
   "metadata": {},
   "outputs": [],
   "source": [
    "spec_stack = image_inv[::-1].T\n",
    "plt.imshow(spec_stack,  cmap='gray_r')\n",
    "plt.axis('off')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cf7edd31-6e83-4a71-8075-5adf49f24917",
   "metadata": {},
   "source": [
    "Now let's _Spectralise_!  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9a960f85-65a7-4a32-9a3e-3aa5724080ba",
   "metadata": {},
   "outputs": [],
   "source": [
    "#show the image again...\n",
    "plt.imshow(image_inv, cmap='gray_r')\n",
    "plt.axis('off')\n",
    "plt.show()\n",
    "\n",
    "score =  Score(notes, 15)\n",
    "\n",
    "#set up spectralizer generator\n",
    "generator = Spectralizer()\n",
    "\n",
    "# Lets pick the mapping frequency range for the spectrum...\n",
    "generator.modify_preset({'min_freq':20, 'max_freq':10000})\n",
    "\n",
    "# set up spectrum\n",
    "\n",
    "# I'm also adding a linear pan from left (azimuth=0.25) to right (azimuth=0.75)\n",
    "# in the horizontal plane (polar=0.5) to give a sense of scrolling left to right...\n",
    "data = {'spectrum':[spec_stack], 'pitch':[1], \n",
    "        'time_evo': [np.linspace(0,1,1920)],\n",
    "        'azimuth':[np.linspace(0.25,0.75,1920)],\n",
    "        'polar':[0.5]}\n",
    "\n",
    "# again, use maximal range for the mapped parameters\n",
    "lims = {'spectrum': ('0','100')}\n",
    "\n",
    "# set up source\n",
    "sources = Events(data.keys())\n",
    "sources.fromdict(data)\n",
    "sources.apply_mapping_functions(map_lims=lims)\n",
    "\n",
    "# render and play sonification!\n",
    "soni = Sonification(score, sources, generator, system)\n",
    "soni.render()\n",
    "print(f\"Spectralising Image...\")\n",
    "soni.notebook_display(show_waveform=0)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}