.. highlight:: rst .. _scqcalert: ######### scqcalert ######### .. _SCQCALERT: ========= SCQCALERT ========= *scqcalert* is a tool that, based on the evaluation of some parameters, allows monitoring and quality control of station acquisition, signal quality, and the automatic processing system. .. note:: *scqcalert* uses a system of **alerts** produced after evaluating two types of parameters; first, those already calculated by other active tools such as the SeisComP module *scqc* and second, new parameters that are not available in the system. Alert types ----------- **Imported from scqc** scqcalert uses a direct connection to the messaging system and collects all alerts calculated and sent by the **scqc** module. Thresholds for sending these alerts and the list of parameters involved must be configured directly in the .cfg file and following the documentation: ref: `scqc`. Currently, the following parameters are available: - real-time data availability [%] - delay [s] - gaps (conteo [counts], intervalo [s], duración [s]) - latencia [s] - offset [counts] - overlaps (count [counts], interval [s], length [s]) - rms [counts] - spikes (count [counts], interval [s], amplitude [counts]) - timing [\%] **Calculated with scqcalert** new custom alerts: - Trigger rate (triggers): Count picks in a time range (1 day by default) using a connection with the messaging groups configured for ref: `scautopick` and its aliases. If picks exceed a threshold (configurable), an alert will be sent. - Pick residual (pickres): Evaluate pick time residuals (including non-preferred origins) using a connection with the messaging groups configured for localization modules ref: `scautoloc`. If residual exceeds a threshold (configurable), an alert will be sent. - Magnitude residual (magres): Evaluate magnitude residuals (including non-preferred magnitudes) using a connection with the messaging groups configured for magnitude calculation modules ref: `scmag`. If residual exceeds a threshold (configurable), an alert will be sent. - Noisy station alerts using PSD (developing): Analyze the noise of each station using a PSDs database with ObsPy npz files. Alert will send when noise is outside of configurable thresholds (-200 and -60 dB by default). In addition, a detection system was implemented following the STA / LTA logic where the ratio between the PSDs value means calculated with the last N days (configurable) against the estimated with the last hour is compared with a threshold (configurable) to send alerts. .. math: ratio = average(\frac{PSDlasthour}{PSDlastNdays}) - Archieved data availability (developing): The module queries the fdsnws "availability" service to check the availability of data stored in the archive for each station. If the availability percentage is lower than a configurable threshold, an alert will be sent. - Flat alerts from scqc parameters (developing): In addition to the alerts sent by scqc generated by sudden changes in the parameters shown in section 4.2. Scqcalert includes configurable thresholds to send alerts if a parameter report values below or above these. For example: delay < 0 seconds. Outputs ------- The outputs of this module are addressed to be used by the networks' technical staff to evaluate quality parameters in data acquisition, seismic noise, and automatic processing. Email alerts ------------ An email including all alerts collected from scqc and produced by scqcalert will be automatically sent to a customizable mailing list every reporting period (by default 24 hours). This email includes a table grouping the alerts for all the channels of each instrument. In the column `` mseedid '', the name of the station is presented without including the component. The `` Alerts '' column includes the parameter's name, the worst value of the collected alerts, and additional information, which could be the number of alerts and/or relevant configuration values ​. The table is sorted by the network in descending order, starting with the network with the highest number of instruments with alerts. Instruments for each network are sorted in descending order, starting with the instrument with the most types of alerts. Here is an example of the table format sent automatically by email: .. _fig-email: .. figure:: media/email.png :width: 16cm :align: center Email send by scqcalert Database with alerts -------------------- Alerts are stored in a database organized with the archive structure: .. code-block:: sh /year/network/station/channel.location/network.station.location.channel.year.day By default the database is stored in the path: .. code-block:: sh @DATADIR@/scqcalert/ Files are stored in *JSON* format. Dashboard --------- A dashboard that incorporates all the information provided by scqcalert is included to make easy quality control .. _fig1-dashboard: .. figure:: media/dash_1.png :width: 16 cm :align: center Drop-down menus included in the Dashboard: Left: the period in days. Center: parameter to analyze (metric). Right: Station sort order method. Drop-down menus available in the dashboard: - Select Time (by default 10): It allows choosing from 1 to 30 days. Users can modify these periods in the cfg file, however, when selecting more than 30 days, the dashboard may take more than 5 seconds to read the information from the database and generate the plots. - Select metric (by default "alert_types"): Metric or type of alert to analyze. Allows choosing Parameters presented in the Alert types section of this document in addition to a metric called` "alert \ _types" which stores the number of alert types for each station. The user can modify the list of metrics in the cfg file. - Select sort order (by default "default"): Select the sort order of the stations in the plots. Currently, the following options are available: - default: Order in which the data was imported from the database with JSON files - net-sta: The stations are sorted by the network in descending order, starting with the network with the highest number of instruments with alerts. The instruments of each network are sorted in descending order, starting with the instrument with the highest number of types of alerts. - total ascending: The stations are arranged in descending order, starting with the station with the highest number of accumulated alerts in the selected period. - max ascending: The stations are arranged in descending order, starting with the station with the highest number of alerts in a reported period despite the accumulated value in the entire selected period. - total descending: The stations are arranged in ascending order, starting with the station with the lowest number of accumulated alerts in the entire selected period. .. note:: To modify the selection of the drop-down menus "Select Time" and "Select metric", please select the option "default" in the menu "Select sort order". This is because ordering the stations must be done after extracting the information from the database. This clarification is just above the "Select sort order" menu. The heat map on the left presents the number of alerts per station and per day based on the metric and period selected from the drop-down menus. The horizontal axis shows the day, the vertical axis shows the station, and the color on the graph shows the number of alerts. Here are some comments about this graph: .. figure:: media/dash_2.png :width: 16cm :align: center The heat map on the left presents the number of alerts per station and per day based on the metric and period selected from the drop-down menus. The horizontal axis shows the day, the vertical axis shows the station, and the color on the graph shows the number of alerts. Here are some comments about this graph: - When selecting the metric "alert_types" the colors mean the number of types of alerts. When selecting any other metric, the colors mean the number of alerts of the selected metric. - Only stations with alerts stored by scqcalert are included - The graph will be reordered according to the selection in the "Select sort order '' drop-down menu. - The graph is interactive with dynamic options shown in the upper right corner. - Hovering the pointer over any cell in the graph will show associated information with the cell: station, day, number of alerts, and types of alerts (dark gray box in the Figure). The histogram on the right shows the number of accumulated alerts per station for the metric and period selected from the drop-down menus. The horizontal axis indicates the number of alerts, and the vertical axis indicates the station name. Here are some comments about this graph: - When selecting the metric "alert_types" the horizontal axis shows the number of accumulated alert types. When selecting any other metric, the horizontal axis indicates the number of accumulated alerts of the selected type. - Only stations with alerts stored by scqcalert are included - The graph will be reordered according to the selection in the "Select sort order '' drop-down menu. - The graph is interactive with dynamic options shown in the upper right corner. - Each histogram bar is divided by cells representing each day during the reporting period. - Hovering the pointer over any cell in the graph will show the information associated with this cell: station, number of alerts/types of alerts, and day (blue box in the Figure). Configuration ============= | :file:`etc/defaults/global.cfg` | :file:`etc/defaults/scqcalert.cfg` | :file:`etc/global.cfg` | :file:`etc/scqcalert.cfg` | :file:`~/.seiscomp/global.cfg` | :file:`~/.seiscomp/scqcalert.cfg` scqcalert inherits `global options `_. .. confval:: APPSpath Type: *path* Path to save\/update config for dashboard Default is ``@DATADIR@/scqcalert/``. .. confval:: jsonPath Type: *path* Path of JSON database to use in dashbaord Default is ``@DATADIR@/scqcalert/``. .. confval:: qcFreqStore Type: *int* Unit: *minutes* Interval in minutes to send and stored alerts and notifications. Default is ``1435``. .. confval:: qcAlertDir Type: *path* Path to store email content, json database and config files. Default is ``@DATADIR@/scqcalert``. .. confval:: qcMsgtype Type: *String* Message type imported from scqc to send new messages to scqcalert \(report\/alert\). Default is ``alert``. .. confval:: listMsg Type: *list:String* Messaging groups to connect. \(Not working, to change messaging groups go to source code\) Default is ``[QC,CONFIG,INVENTORY,PICK,MAGNITUDE,LOCATION]``. .. confval:: netWhitelist Type: *list:String* List with networks allowed to process Default is ``[MX,GI,SV,SN,NU,TC,OV,AM]``. .. confval:: netBlacklist Type: *list:String* List with networks not allowed to process .. confval:: locBlacklist Type: *list:String* List with location codes not allowed to process Default is ``99``. .. confval:: chanWhitelist Type: *list:String* List with channels allowed to process Default is ``[HN,EN,SN,HH,EH,SH]``. .. confval:: chanBlacklist Type: *list:String* List with channels not allowed to process .. confval:: dbURIcmpuQC Type: *String* db URI to query QC parameters when using \-\-QCquery mode .. confval:: qcMetNames Type: *list:String* QC metric names to query when using \-\-QCquery mode Default is ``[delay,latency]``. .. confval:: qcMetThrs Type: *list:double* QC metric thresholds to send alerts when using \-\-QCquery mode Default is ``[5.0,5.0]``. .. confval:: psdDir Type: *path* Path with npz files for psd evaluation .. confval:: trig_max Type: *int* Maximum number of triggers Default is ``30``. .. confval:: pickName Type: *String* scautopick alias name to extract config values \(trigOn\) Default is ``scautopick``. .. confval:: mag_max Type: *double* Maximum magnitude value Default is ``8.0``. .. confval:: pick_res_max Type: *double* Unit: *seconds* Maximum pick residual Default is ``1.5``. .. confval:: dbURIcmpuDB Type: *String* db URI to query arrivals and magnitudes when using \-\-DBquery mode .. confval:: arrRes Type: *double* Maximum arrival residual to send alerts when using \-\-DBquery mode .. confval:: magRes Type: *double* Maximum magnitude residual to send alerts when using \-\-DBquery mode .. confval:: qcMargin Type: *int* Period in seconds to search for qc values \(timing quality\) Default is ``86400``. .. confval:: mag_res_max Type: *double* Maximum magnitude residual Default is ``1.0``. .. confval:: min_res_sample Type: *int* Minimum number of pick and magnitude residuals Default is ``3``. .. confval:: fdsnwsResp Type: *String* fdsnws service to import response Default is ``http://localhost:8080``. .. confval:: psdPeriods Type: *list:double* Unit: *seconds* Periods to evaluate psd values Default is ``[1,0.2,0.1,0.066]``. .. confval:: psdmin Type: *double* Unit: *db* Minimum PSD value Default is ``-130.0``. .. confval:: psdmax Type: *double* Unit: *db* Mimimum ratio STA\/LTA logic to evaluate changes on PSDs Default is ``-90.0``. .. confval:: psdslmax Type: *double* Maximum ratio STA\/LTA logic to evaluate changes on PSDs Default is ``1.04``. .. confval:: psd_numdays Type: *int* Number of days to calculate LTA using PSDs average values Default is ``5``. .. confval:: psdSource Type: *String* Source of PSD files in .npz format. Only url or file available. Default is ``file``. .. confval:: psdNets Type: *list:String* Network codes to query PSD files when url set as psdSourse. .. confval:: psdUrl Type: *String* URL to query PSD files when url set as psdSource .. confval:: psdTempDir Type: *path* Path to save PSDs downloaded from url when url set as psdSourse. .. confval:: availThr Type: *int* Percentage threshold to send alert on SDS data available Default is ``70``. .. confval:: dbURIcmpuAVAIL Type: *String* db URI to query availability information using \-\-AVAILcomp mode