pyValEIA.io.load

Load supported data files.

Functions

`load_cdf_data`(file_path, variable_names)	Load a CDF file.
`extract_cdf_time`(cdf_data[, time_var])	Extract common coordinate data from a CDF file object.
`load_swarm`(start_date, end_date, sat_id, file_dir[, ...])	Load Swarm data, downloading any missing files.
`load_madrigal`(stime, fdir)	Load Madrigal TEC data from given time.
`load_nimo`(stime, file_dir[, name_format, ne_var, ...])	Load daily NIMO model files.
`load_daily_stats`(stime, model, obs, file_dir, **kwargs)	Load the daily statistics file with model-data comparisons.
`multiday_states_report`(date_range, daily_dir, ...[, ...])	Create a state report for a range of dates with a skill check.

Module Contents

pyValEIA.io.load.load_cdf_data(file_path, variable_names)[source]

Load a CDF file.

Parameters:

file_pathstr: file path
variable_namesarray-like: variable names for file to be extracted

Returns:

var_dictdict: CDF file data with desired variables extracted
cdf_datacdflib.cdfread.CDF: Loaded CDF data from cdflib

See also

cdflib.CDF

pyValEIA.io.load.extract_cdf_time(cdf_data, time_var='Timestamp')[source]

Extract common coordinate data from a CDF file object.

Parameters:

cdf_datacdflib.cdfread.CDF: CDF data object from loaded file
time_varstr: Time variable (default=’Timestamp’)

Returns:

epocharray-like: UT as datetime objects

Raises:

ValueError: If an incorrect variable name is requested

pyValEIA.io.load.load_swarm(start_date, end_date, sat_id, file_dir, instrument='EFI', dataset='LP', f_end='0602')[source]

Load Swarm data, downloading any missing files.

Parameters:

start_datedt.datetime: Starting time
end_datedt.datetime: Ending time
sat_idstr: Swarm satellite ID, one of ‘A’, ‘B’, or ‘C’
file_dirstr: File directory where the instrument directory is located. Files will be located in a directory tree specified by download_and_unzip_swarm
instrumentstr: Swarm instrument (default=’EFI’)
datasetstr: Desired dataset acronym from instrument, e.g. ‘LP’ is Langmuir Probe (default=’LP’)
f_endstr: For different data products there are different numbers at the end The most common for EFIxLP is ‘0602’ where ‘0602’ represents the file version. Other datasets may also have a string that represents the record type (default=’0602’)

Returns:

swarm_datapd.DataFrame: DataFrame of Swarm data for the desired instrument and satellite

Raises:

ValueError: If an unknown dataset is requested (currently only supports ‘LP’)

pyValEIA.io.load.load_madrigal(stime, fdir)[source]

Load Madrigal TEC data from given time.

Parameters:

stime: datetime object: Universal time for the desired madrigal output
fdirstr kwarg: directory where file is located

Returns:

mad_dictdict: Dictionary of the madrigal data including: TEC, geographic latitude, geographic longitude, TEC error (dTEC), timestamp, and date in the datetime format

Raises:

ValueError: If no file was found for the desired date

Notes

This takes in madrgial files of format gps%y%m%dg.002.netCDF4 5 minute cadence

pyValEIA.io.load.load_nimo(stime, file_dir, name_format='NIMO_AQ_%Y%j', ne_var='dene', lon_var='lon', lat_var='lat', alt_var='alt', hr_var='hour', min_var='minute', tec_var='tec', hmf2_var='hmf2', nmf2_var='nmf2', time_cadence=15)[source]

Load daily NIMO model files.

Parameters:

stimedt.datetime: Day of desired NIMO run
file_dirstr: File directory, wildcards will be resolved but should only result in one file per day for the specified name_format
name_formatstr: Format of NIMO file name including date format before .nc (default=’NIMO_AQ_%Y%j’)
ne_varstr: Electron density variable name (default=’dene’)
lon_varstr: Geographic longitude variable name (default=’lon’)
lat_varstr: Geodetic latitude variable name (default=’lat’)
alt_varstr: Altitude variable name (default=’alt’)
hr_varstr: UT hour variable name (default=’hour’)
min_varstr: UT minute variable name, or ‘’ if not present (default=’minute’)
tec_varstr: TEC variable name (default=’tec’)
hmf2_varstr: hmF2 variable name (default=’hmf2’)
nmf2_varstr: NmF2 variable name (default=’nmf2’)
time_cadenceint: Model UT output time cadence of data in minutes (default=15)

Returns:

nimo_dcdict: Dictionary with variables dene, glon, glat, alt, hour, minute, date, tec, nmf2, and hmf2

Raises:

ValueError: If no NIMO file could be found at the specified location and time
KeyError: If an unexpected variable is supplied

pyValEIA.io.load.load_daily_stats(stime, model, obs, file_dir, **kwargs)[source]

Load the daily statistics file with model-data comparisons.

Parameters:

stimedatetime: day of desired file
modelstr: Case-sensitive name of model requested (e.g., ‘NIMO’, ‘PyIRI’).
obsstr: Name of data set requested (e.g., ‘SWARM’, ‘MADRIGAL’)
file_dirstr: File directory
kwargsdict: Optional kwargs by data type. Includes ‘mad_lon’, which expects longitudes of either -90 deg E or 60 deg E for Madrigal data.

Returns:

stat_datapd.DataFrame: Dataframe that includes all information from type file

Raises:

ValueError: If expected file does not exist

pyValEIA.io.load.multiday_states_report(date_range, daily_dir, model_name, obs_name, obs_constraint, comp_type='eia')[source]

Create a state report for a range of dates with a skill check.

Parameters:

date_rangepd.DateRange: Date range of desired states files
daily_dirstr: Directory containing the daily files
model_namestr: Case-sensitive model name to load, expects the capitalization used in the daily stats file header. (e.g., ‘Nimo’, ‘PyIRI’)
obs_namestr: Observation name to load (e.g., ‘SWARM’, ‘MADRIGAL’)
obs_constraintstr or int: Additional constraint for observation type. For Madrigal, this is the longitude as a integer. For Swarm, this is the altitude string, which currently accepts ‘swarm’, ‘hmf2’, or ‘100’.
comp_typestr: Desired type to check against for orientation or EIA state, expecting one of ‘eia’, ‘peak’, ‘flat’, or ‘trough’ for EIA state comparison or one of ‘north’, ‘south’, or ‘neither’ for an orientation comparison (default=’eia’)

Returns:

model_framepd.DataFrame: Model longitude, local time, states, directions, and EIA types. If Swarm observations are requested, will also contain a satellite list.
obs_framepd.DataFrame: Data longitude, local time, states, directions, and EIA types. If Swarm observations are requested, will also contain a satellite list.

Raises:

ValueError: For incompatible input combinations