Skip to content

Home

Jump to bottom
Matthias Prinke edited this page Feb 8, 2023 · 56 revisions

Welcome to the BresserWeatherSensorReceiver wiki!

Adding Support of multiple Sensors

e.g. Weather Sensor and Soil Moisture Sensor

Option for skipping unwanted sensors (e.g. neighbours') - currently messages are received and decoded regardless of type or ID of sensor!

3 Different solutions have been considered:

Solution 1

  • decode radio messages in order as received
  • save decoded data into temporary buffer
  • copy data into final structure

Pros:

  • easy to implement
  • clean code structure

Cons:

  • memory wasted for temporary buffer
  • time for copying data

Solution 2

  • handle radio messages in predefined order
  • save decoded data into final structure

Pros:

  • easy to implement
  • clean code structure
  • efficient memory utilization

Cons:

  • inefficient usage of CPU time waiting for desired message

Solution 3

  • decode radio messages in order as received
  • save decoded data into final buffer in reception order; check for data already received (either complete or incomplete)
  • map data buffer (order of reception) to predefined order or search for desired sensor ID/type for presentation

Pros:

  • efficient memory utilization
  • efficient usage of CPU time

Cons:

  • somewhat difficult to implement
  • potentially fuzzy code structure

Implemented: Solution #3

Solution #3 has been implemented and the code does not look too bad, IMHO!

Implementing Rain Gauge Statistics

Currently, the rain gauge value is provided as-is, 0...999.9 mm for the "5-in-1" protocol, 0...19999.9 for the "6-in-1" protocol. When an overflow occurred in the former, my main and only customer (Dad!) thought that he had found a bug in my code! (Whaaat? 😮) After some explanation, he stated that it would be nice to have the same kind of data on the smartphone's MQTT Panel as on the Bresser weather station display. Here comes the feature request...

The following problems have to be addressed:

  • Which values are wanted/needed?

    • Rate - Current rainfall rate (base on 10 min rain data) [not implemented]

    • Hourly - the total rainfall in the past hour

    • Daily - the total rainfall from midnight (default)

    • Weekly - the total rainfall of the current week

    • Monthly - the total rainfall of the current calendar month

    • Total - the total rainfall since the last reset [not implemented]

  • Which range can be expected from those values?

    see Wikipedia's list of weather records - Section "Rain"

  • Which algorithms are needed? Which data has to be accumulated in nonvolatile memory? In any case, an overflow of the rain gauge value has to be taken into account. The maximum rain gauge value has to be provided and the number of overflows must be stored.

    • Daily/weekly/monthly rainfall Pseudocode:
    foreach x in {day, week, month} {
    if (tsPrev.<x> != tsNow.<x>) {
        // day/week/month has changed
        // save timestamp at begin of new interval 
        tsBegin_<x> = tsNow;
        // save rain gauge value at begin of new interval
        rainBegin_<x> = rainNow;
    }
    rain_<x> = rainNow - rainBegin_<x>;
}

    Note:

    The begin of an accumulation interval (daily/weekly/monthly) is not detected exactly if the algorithm is executed at an interval (e.g. 10 minutes). In this case, the maximum deviation of the actual (timestamp;value)-pair from the ideal (timestamp;value)-pair is the length of the execution interval and the maximum possible change of value in this interval, respectively.

    • Total rainfall in the past hour

      To determine the rainfall in the past hour, timestamps and rain gauge values are stored in a circular buffer:

           ---------------     -----------
  .-> |   |   |   |   |...|   |   |   |--. 
  |    ---------------     -----------   |
  |     ^                   ^            |
  |    tail                head          |
  `--------------------------------------'
    • new value: increment(head); rain[head] = rainNow; ts[head] = tsNow;
    • remove stale entries: if ((ts[head]-ts[tail]) > 1hour) { increment(tail); }
    • calculate hourly rate: rain_hour = rain[head] - rain[tail];

    ts: timestamp

    The required size of the circular buffer is determined by dividing one hour by the minimum sensor data reception interval, e.g. for 1 hour at an interval of 6 minutes, 10 entries are needed. For https://github.com/matthias-bs/BresserWeatherSensorTTN, only the sleep interval is fixed, the sensor reception interval and the LoRaWAN transmission interval are variable (but limited):

    t_sleep t_rx_sensor ... t_LoRaWAN ...
    SLEEP_INTERVAL < WEATHERSENSOR_TIMEOUT < SLEEP_TIMEOUT_JOINED (min.)
    or
    < SLEEP_TIMEOUT_INITIAL + SLEEP_TIMEOUT_JOINED + SLEEP_TIMEOUT_EXTRA (max.)
    • Result
    Value NV data
    Hourly ts[BUF_SIZE], rain[BUF_SIZE], head, tail
    Daily tsBegin_day, rainBegin_day
    Weekly tsBegin_week, rainBegin_week
    Monthly tsBegin_month, rainBegin_month
    ts_prev

    In the actual implementation, the following optimizations have been made to save non-volatile memory:

    • Hourly: timestamps are stored as seconds since midnight, rain gauge values are stored as 16-bit fixed point data (1 decimal)
    • Daily/Weekly/Monthly: only the required parts of a timestamp (i.e. day, week or month) are stored

  • How does this data fit into the limited LoRaWAN payload?

    Currently all four values are transmitted as float (i.e. 4 bytes each). If required, 16-bit fixed point values could be used, but care has to be taken that no overflow occurs (considering global extremes).

  • How to get the date and time of day (e.g. from the LoRaWAN network)?

  • Which re-syncing interval of the RTC to the network time is reasonable?)

    Assuming 24 hours, can be configured with CLOCK_SYNC_INTERVAL in BresserWeatherSensorTTN.cfg.

  • Do we have to consider daylight saving time?

    Yes, if we want to calculate daily (and monthly) rainfall correctly.

  • Implement reset of rain gauge statistics by LoRaWAN downlink message ✔️

Testing Rain Gauge Statistics

Due to the relative complexity and the time-scale of real input data, unit tests have been implemented with CppUTest in https://github.com/matthias-bs/BresserWeatherSensorReceiver/tree/main/test.

There are two sets of tests:

  1. TestRainGauge.cpp - artificial test data; separate tests for each function, considering corner cases
  2. TestRainGaugeReal.cpp - generated from real-world data (Pottery Fields rain gauge rainfall data - 12/06/2013 - 21/02/2016) with raindata2test.pl; all functions combined

In addition to the test results, the test coverage (line coverage) is provided.

While the actual implementation- and testing-cycle has been done locally, the unit tests are now also implemented as CppUTest.yml GitHub workflow with GitHub Actions.

The coverage report is exported to https://coveralls.io/github/matthias-bs/BresserWeatherSensorReceiver using the Coveralls GitHub Action.

Lightning sensor

In need of a lightning sensor? Stay tuned: https://github.com/merbanan/rtl_433/issues/2140

Another solution (connected via I²C and IRQ): https://www.dfrobot.com/product-1828.html

Radio Transceiver Range

Transmission range depends on many factors. In some cases the range on the original equipment (outdoor sensor and weather station/display) cannot be achieved with a home-made receiver.

In Issue #14 - Calibrate from CC1101?, it was suggested to provide a means to calibrate the CC1101 transceiver's frequency offset.

Sometimes it helps to move the receiver a few centimeters to a different position or to change the antenna's orientation. You can also try a different type of antenna - wire antenna vs. helical coil antenna. Unfortunately there does not seem to be a CC1101 board with a proper antenna connector.

The YouTube video #182 ESP32 Lora Boards: What you need to know before you buy (incl. Antenna knowledge) by Andreas Spiess also suggests that some antennas out there are not as good as you could hope for...

I also noticed that radiated emissions from other electronic devices (PC, other microcontroller boards, ...) near the receiver had significant influence on signal quality. I even noticed that the power supply had some influence (USB power from my notebook was bad, USB power from my desktop PC or wall adapter was fine).

BUT - I got almost rock solid reception when I changed to the Adafruit RFM95W LoRa Radio Transceiver Breakout with a proper antenna (Delock LoRa 868 MHz Antenna SMA plug 3 dBi omnidirectional with tilt joint black in my case). I don't know if this is due to better chip design, better PCB design or just the antenna.

ESP32 Logging

See

(With workaround for ESP8266; see https://github.com/matthias-bs/BresserWeatherSensorReceiver/blob/development/src/WeatherSensorCfg.h)

Resources

Löffler, Hans: Meteorologische Bodenmesstechnik (vormals: Instrumentenkunde). Offenbach am Main: Leitfaden für die Ausbildung im Deutschen Wetterdienst Nr. 6, Selbstverlag des Deutschen Wetterdienstes, 2012 (Link)

(Everything you ever wanted to know about ground-based weather sensors - and much more! - in German)

Clone this wiki locally