{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Finding GBM Data\n", "\n", "A natural question may be: \"Where do I find the data I need?\" Well, you're in luck, because this will show you how to find the data you seek. GBM Data is hosted publicly on the HEASARC FTP server via the Fermi Science Support Center, and the data are stored in a consistent directory structure. But instead of having to navigate a winding maze of FTP directories, we provide a couple of classes built to retrieve the data you want. First, you need to decide if you want trigger data (say from a GRB) or continuous data. Let's start with trigger data, and assume you know the trigger number you're interested in:" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "122" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# the datafinder class for triggers\n", "from gbm.finder import TriggerFtp\n", "\n", "# initialize the Trigger data finder with a trigger number\n", "trig_finder = TriggerFtp('190114873')\n", "trig_finder.num_files" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We don't really care about the directory structure, we just want the data. So this quickly gets us to the directory we need. There are 122 files associated with this trigger. Say we want CSPEC data. is there CSPEC available?" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['glg_cspec_b0_bn190114873_v00.pha',\n", " 'glg_cspec_b1_bn190114873_v00.pha',\n", " 'glg_cspec_n0_bn190114873_v00.pha',\n", " 'glg_cspec_n1_bn190114873_v00.pha',\n", " 'glg_cspec_n2_bn190114873_v00.pha',\n", " 'glg_cspec_n3_bn190114873_v00.pha',\n", " 'glg_cspec_n4_bn190114873_v00.pha',\n", " 'glg_cspec_n5_bn190114873_v00.pha',\n", " 'glg_cspec_n6_bn190114873_v00.pha',\n", " 'glg_cspec_n7_bn190114873_v00.pha',\n", " 'glg_cspec_n8_bn190114873_v00.pha',\n", " 'glg_cspec_n9_bn190114873_v00.pha',\n", " 'glg_cspec_na_bn190114873_v00.pha',\n", " 'glg_cspec_nb_bn190114873_v00.pha']" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "trig_finder.ls_cspec()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Great! There's a full complement of CSPEC data. How about responses for the CSPEC data?" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['glg_cspec_b0_bn190114873_v02.rsp',\n", " 'glg_cspec_b1_bn190114873_v02.rsp',\n", " 'glg_cspec_n0_bn190114873_v02.rsp',\n", " 'glg_cspec_n1_bn190114873_v02.rsp',\n", " 'glg_cspec_n2_bn190114873_v02.rsp',\n", " 'glg_cspec_n3_bn190114873_v02.rsp',\n", " 'glg_cspec_n4_bn190114873_v02.rsp',\n", " 'glg_cspec_n5_bn190114873_v02.rsp',\n", " 'glg_cspec_n6_bn190114873_v02.rsp',\n", " 'glg_cspec_n7_bn190114873_v02.rsp',\n", " 'glg_cspec_n8_bn190114873_v02.rsp',\n", " 'glg_cspec_n9_bn190114873_v02.rsp',\n", " 'glg_cspec_na_bn190114873_v02.rsp',\n", " 'glg_cspec_nb_bn190114873_v02.rsp']" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "trig_finder.ls_rsp(cspec=True, ctime=False)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Again, we can list all of the relevant files. Are there any quicklook lightcurve plots?" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['glg_lc_chan12_bn190114873_v00.pdf',\n", " 'glg_lc_chan34_bn190114873_v00.pdf',\n", " 'glg_lc_chan567_bn190114873_v00.pdf',\n", " 'glg_lc_chantot_bn190114873_v00.pdf',\n", " 'glg_lc_tot_bn190114873_v00.pdf']" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "trig_finder.ls_lightcurve()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "What if we want to move on to another trigger? You don't have to create a new `TriggerFTP` object, you can just `set_trigger()`:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "128" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# change trigger\n", "trig_finder.set_trigger('170817529')\n", "trig_finder.num_files" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Of course, you don't want to just list the files in a directory, you want to download them. Let's download all the catalog files for GRB 170817A:" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "glg_bcat_all_bn170817529_v01.fit [==============================] 100.00%\n", "glg_scat_all_bn170817529_flnc_band_v00.fit [==============================] 100.00%\n", "glg_scat_all_bn170817529_flnc_comp_v00.fit [==============================] 100.00%\n", "glg_scat_all_bn170817529_flnc_plaw_v00.fit [==============================] 100.00%\n", "glg_scat_all_bn170817529_flnc_sbpl_v00.fit [==============================] 100.00%\n", "glg_scat_all_bn170817529_pflx_band_v00.fit [==============================] 100.00%\n", "glg_scat_all_bn170817529_pflx_comp_v00.fit [==============================] 100.00%\n", "glg_scat_all_bn170817529_pflx_plaw_v00.fit [==============================] 100.00%\n", "glg_scat_all_bn170817529_pflx_sbpl_v00.fit [==============================] 100.00%\n", "glg_tcat_all_bn170817529_v03.fit [==============================] 100.00%\n" ] } ], "source": [ "trig_finder.get_cat_files('./')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we want some continuous data. There aren't any trigger numbers for continuous data. Continuous CTIME and CSPEC are available in files that cover a whole day (in UTC) and TTE are offered in hourly files. To find the data you need, instead of a trigger number, you need to specify a time" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "379" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# the datafinder class for continuous data\n", "from gbm.finder import ContinuousFtp\n", "\n", "# initialize the continuous data finder with a time (Fermi MET, UTC, or GPS)\n", "cont_finder = ContinuousFtp(met=587683338.0)\n", "cont_finder.num_files" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "That's a whole lotta files in this directory. Most of them are TTE; remember that each hour has a TTE file (since the end of 2012) for each detector. Let's just list the CTIME that's available:" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['glg_ctime_b0_190816_v00.pha',\n", " 'glg_ctime_b1_190816_v00.pha',\n", " 'glg_ctime_n0_190816_v00.pha',\n", " 'glg_ctime_n1_190816_v00.pha',\n", " 'glg_ctime_n2_190816_v00.pha',\n", " 'glg_ctime_n3_190816_v00.pha',\n", " 'glg_ctime_n4_190816_v00.pha',\n", " 'glg_ctime_n5_190816_v00.pha',\n", " 'glg_ctime_n6_190816_v00.pha',\n", " 'glg_ctime_n7_190816_v00.pha',\n", " 'glg_ctime_n8_190816_v00.pha',\n", " 'glg_ctime_n9_190816_v00.pha',\n", " 'glg_ctime_na_190816_v00.pha',\n", " 'glg_ctime_nb_190816_v00.pha']" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# list ctime data covering this time\n", "cont_finder.ls_ctime()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now let's list the available TTE for this time. This will only list the TTE files in the directory that cover the relevant time:" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['glg_tte_b0_190816_21z_v00.fit.gz',\n", " 'glg_tte_b1_190816_21z_v00.fit.gz',\n", " 'glg_tte_n0_190816_21z_v00.fit.gz',\n", " 'glg_tte_n1_190816_21z_v00.fit.gz',\n", " 'glg_tte_n2_190816_21z_v00.fit.gz',\n", " 'glg_tte_n3_190816_21z_v00.fit.gz',\n", " 'glg_tte_n4_190816_21z_v00.fit.gz',\n", " 'glg_tte_n5_190816_21z_v00.fit.gz',\n", " 'glg_tte_n6_190816_21z_v00.fit.gz',\n", " 'glg_tte_n7_190816_21z_v00.fit.gz',\n", " 'glg_tte_n8_190816_21z_v00.fit.gz',\n", " 'glg_tte_n9_190816_21z_v00.fit.gz',\n", " 'glg_tte_na_190816_21z_v00.fit.gz',\n", " 'glg_tte_nb_190816_21z_v00.fit.gz']" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# list hourly TTE data covering this time\n", "cont_finder.ls_tte()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Similar to the trigger finder, you can use the same object to search at different times." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "# change the time of interest\n", "cont_finder.set_time(utc='2017-08-17T12:41:06.47')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now how about downloading the position history file for this time:" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "glg_poshist_all_170817_v01.fit [==============================] 100.00%\n" ] } ], "source": [ "cont_finder.get_poshist('./')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "---\n", "## Searching the GBM Catalogs" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The HEASARC also hosts two catalogs that are of interest here: a Trigger Catalog that contains information about every GBM trigger, and a Burst Catalog that contains standard analysis of every triggered GRB. HEASARC provides a way to search these catalogs online through their Browse interface, but we offer a way to do it in Python through the Data Tools.\n", "\n", "Let's look at the trigger catalog first:" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Downloading Catalog from HEASARC via w3query.pl...\n", "Finished in 8 s\n" ] }, { "data": { "text/plain": [ "8283" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from gbm.finder import TriggerCatalog\n", "\n", "trigcat = TriggerCatalog()\n", "trigcat.num_rows" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Depending on your connection, initialization may take a few seconds. You can see what columns are available in the catalog:" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array(['version', 'trigger_name', 'name', 'ra', 'dec', 'trigger_time',\n", " 'trigger_type', 'reliability', 'adc_high', 'adc_low', 'bii',\n", " 'channel_high', 'channel_low', 'dec_scx', 'dec_scz',\n", " 'detector_mask', 'end_time', 'error_radius', 'geo_lat', 'geo_long',\n", " 'lii', 'localization_source', 'phi', 'ra_scx', 'ra_scz', 'theta',\n", " 'time', 'trigger_algorithm', 'trigger_timescale'], dtype='