Signal detection¶
1650 words Estimated reading time 7 minutes
The controller cannot distinguish between electromagnetic signals which have been sent out by lightning discharges on the one hand, and which are a result of local noise on the other hand. There is a variety of such sources which can generate interference, i.e. power-supplies, energy-saving lamps and almost all electric home equipment.
The system uses a combination of hardware and software filters to distinguish between these two types of signals.
Hardware Filters¶
The amplifiers already have fixed low and high pass filters on board which can not be adjusted.
High Pass (BLUE only)¶
System BLUE has a switchable high pass filter.
Further description still missing. Contributions/examples from community are welcome.
Adjustable Digital Filter (BLUE only)¶
See here.
Further description still missing. Contributions/examples from community are welcome.
Software Filters¶
Info
Theoretically a notch filter implemented with DSP functions would be sufficient to mask local noise. However, they alter the signal shape and also our hardware isn't fast enough for that so we have some very simple and unusual filters which don't need much processing power. So only if a channel doesn't get filtered by them, it may get analyzed by DSP functions.
Software filters are used to limit the number of data sent to the computing servers. They are applied on each channel individually and they decide wether the signal is good or not. As a result the whole signal is filtered (nothing will be sent) or just specific channels. They do not alter the signal shape sent to the servers. Some filters are always enabled, some have to be enabled manually or will be enabled by the servers. The following table lists all available filters. The letter in brackets is used to identify the filter. When such a letter appears on the signal page, then this particular channel has been filtered out.
Filter Name | Letter | Description / Use Case |
---|---|---|
Threshold (by ADC) | L | Signal is below the threshold (ADC hardware watchdog) |
Threshold Filter | T | Avoid transmit of low power signal (traffic saving) |
Peak (Voltage) Band | P | Filters voltage band in case of special disturbers |
Spike | S | Spike detection from on board electrical noise |
Max Amplitude | M | Avoid high power signals and focus on lower amplitudes |
DSP | D | Check frequencies, filter specific noise |
Channel Interference | I | Too much interference on one channel |
Interference Mode | - | Signal sending temporarily disabled |
Threshold (by ADC)¶
This is not a filter like the other ones as it is implemented in hardware, but it is included here for the sake of completeness. An upper and lower threshold are set for each channel. Only if a signal exceeds either of these thresholds, the CPU will read the data of all channels and continue with the further signal processing and filtering. While doing this the CPU can not handle any other tasks.
Threshold Filter¶
If one channel has been triggered then always all other channels are processed too, even if their amplitudes stay below the threshold. These channel can still be sent, as long as they stay over a adjustable threshold percentage. This may give some additional (useful) data to the computing servers, but also avoids unnecessary further data processing an saves traffic on very low powered signals.
The only parameter of this filter is the minimum percentage of the threshold. For example, if the minimum percentage is set to 50%, then only channels with an amplitude of at least 50% of the threshold will be included in the signal packet.
Peak Band Filter¶
Here the number of peak amplitudes which occur in a specific voltage span (band) are counted over some time. In case there are too much of peaks in one band, then the signal of this channel will not be sent. This filter is off by default and should only be enabled on noise from specific devices which can not be filtered otherwise.
The purpose of this filter is very limited and only useful for a few special cases. For example robotic lawn mowers can produce single peaks at a constant rate with constant voltage levels, i.e. 100 peaks per second at 1V. Lowering the gain here is not an option, because it would reduce the overall sensitivity of the sensor. But the beak band filter will detect that there are a lot of peaks around 1V and then ignore this voltage band. Thus, sferics with much lower amplitudes can still be detected.
For historic reasons this filter can be manually enabled on the web interface, but the parameters can not be adjusted there. The parameters are:
- Number of Bands: The number of voltage bands which are checked for peaks.
- Measure Interval: Peaks are added up within this period.
- Minimum Count: The minimum number of signals on a channel within measure period.
- Filter Count Min.: The minimum number of signals within measure period in a band so the band gets masked.
- Filter Count Min. Percentage: The above condition is only true if count is above this percentage.
- Filter Count Max.: Band will be masked if count of peaks is above this parameter.
Spike Filter¶
This filter is used to detect spikes in the signal, i.e. sudden changes in the amplitude of the signal between two very close samples. Only the trigger point will be checked for a spike, so it takes very few CPU time. Spikes can be a result from errors during the sampling process or from on board electrical noise. Very close lightning discharges can produce similar spikes, so they might be filtered too.
The spike filter is enabled by default on RED stations. If a spike has been detected, the whole signal will be ignored.
Parameters are spike width and minimum size.
Max Amplitude Filter¶
This filter may be activated by the servers on close thunderstorms. It avoids sending signals with very high amplitudes from strong lightning discharges as they can easily be detected by stations a bit farther away. The resources then can be used for processing and sending signals from signals with lower amplitudes.
Parameters are minimum and maximum amplitude voltage. So far it isn't used and it is unclear whether this filter is really useful.
DSP Filter¶
Work in progress
Filtering not implemented yet
When a channel hasn't been filtered before it will be processed by DSP functions. This is very CPU intensive workload although the STM32F4 processor has support for signal processing, like fast fourier transformation (FFT). The FFT will be made with fewer samples (i.e. divided by 4). With default settings and three channels it is able to process about 1000 signals per second. However, in this case almost no processing power is left for other tasks, like sending the signals. Thus the previous filters are important to keep the signal rate low. The results of the DSP processing, the frequency spectrum, won't be transmitted to the servers, as it would produce unneeded traffic. Within the determined frequency spectrum, the frequency bands with the highest energy are determined. Just like the other filters a yes/no decision will be made.
Parameters:
- Divisor: The divisor for the FFT calculation.
- Bands: Amount of frequency bands to check.
- Buffers: Amount of buffers, so that results can be seen on signals page.
Interference Filter¶
The moving average of triggered signals per second of the particular channel is computed. If this value is higher than a defined maximum threshold then the channel will be disabled. It will be enabled again if the rate drops below a minimum threshold. If a toggle from off to on happens to fast within a certain time span, the sensitivity will be raised. Higher sensitivity value result in even lower max/min thresholds. The sensitivity will be slightly lowered again if the off/on toggle time span is long enough. The default thresholds and time spans are derived from the slow interference mode value (see below).
Parameters:
- Max Rate: Defaults to 70% of the slow interference mode value.
- Min Rate: Defaults to 50% of the slow interference mode value.
- Time Gap: 5 times of the "smooth" value (moving average time)
Please note that the interference filter might not always work as intended and while active may cause some additional CPU load. That's because the ADC watchdog stays active even for the channel which has been "disabled" by the interference filter, because it needs to count the triggers. In extreme cases it triggers all the time which leads to ~1000 signals per second. This may have influence on the other channels. While interference filter is active on a channel, no DSP checks are made on this channel.
Interference Mode¶
The interference mode checks the amount of signals which are already queued ready to send. Such a signal has already passed all filters. However, that might still be too much to process on our computing servers. So the interference mode will be enabled, when the amount of signals is higher than a defined value. The threshold value for disabling the interference mode is several percents lower, which avoids conditions where the mode toggles between on and off all the time. The thresholds cannot be changed by the user.
In detail, there are three interference modes implemented:
- Burst: It is very fast and checks the signal rates of a very short time period, i.e. of one second. This is very useful to protect the server against short bursts, which take only some seconds but produces a lot of interference signals.
- Slow: Checks the amount of signals during a longer time span, i.e. 60s. That means, it will be enabled on medium amount of interference signals over a longer time span.
- Repeated Bursts: Counts the number of signals before the burst mode is triggered and enables the slow mode in case the threshold is reached. This mode is used to detect repeated bursts of signals over a longer time span, which are not detected by the burst mode alone.