rtl_watch: Real-time monitor and display of devices seen by rtl_433 #2409
Replies: 11 comments 23 replies
-
Thanks! Be sure to update the wiki ;) Also it could be interesting to calculate a "usual" transmit interval (the regular minimum when it's not immediate repeats) and then divide the count by that to show reliability. Maybe even with a trend? |
Beta Was this translation helpful? Give feedback.
-
For anyone who downloads and runs rtl_watch, I have a request: could you let it run for a while, then do a screen capture and post what your rtl_433 has seen? I live in a fairly compact and isolated neighborhood. So I have about 6 neighbors whose thermometers I see routinely. I occasionally see tire pressure gauges from delivery trucks. I even see a couple of fuel-oil gauges from homes that must be at least 300m from my home, since our neighborhood uses gas for heating and the nearest other neighbors are at least 300m away. I had originally intended to design that display window to be scollable, anticipating that the list of devices would be fairly long. It looks like the list might not be that long under regular use, and it looks like implementing a scrollable window with tkinter might be difficult -- and I'm not sure I can test it. So I could use some feedback as to whether a scrollable window is needed. Thanks in advance if anyone can help with that. |
Beta Was this translation helpful? Give feedback.
-
@hdtodd - This all looks very potentially very useful . I own a lot of RF devices in two different places and have spent a bit of time trying to track down loss of reception or interference over the years. Two thoughts of the top of my head: First, you might want to consider combining rtl_watch and rtl_snr (at least the python version) into one repo and possibly code base. Second, (for me) the GUI app isn't that practical for running for longer periods -- laptops go to sleep. I'd need to run it on a remote virtual X desktop that I'd access with VNC. A quick hack might be just writing the output to an HTML or JSON file which also contains some JavaScript to refresh the page or refetch the JSON. What would be most interesting (to me) In the long run with rtl_433 is no GUI, but just republishing this data to MQTT so I can use what ever I want to track and graph it over time. |
Beta Was this translation helpful? Give feedback.
-
About reliability: What I do in home assistant is to send an offline notification when I haven't heard for about 3.5x the normal interval. That doesn't measure reliability so much as disappearing (battery or weather usually). I don't think standard deviation is going to be the right measure. I would try to histogram the intervals and find the shortest interval that has a good peak and then basically after 1.2x that declare/count a missing transmission. To tune this I would just record the arrival times over a few days and then look at the histogram (perhaps binned to the second) and see what it looks like. Then, try to find a repeat interval and see if t0 + k * R fits the arrival times to see what the rate variation is. But that's just me being excessive perhaps. So more concretely, take the 100 most recent times, histogram the gaps within bins at maybe 1s granularity maybe 1.25x factors, and then the hard part find the shortest significant peak and call that the expected interval. Then it's no report after expected*1.2 that is meaningful. |
Beta Was this translation helpful? Give feedback.
-
I analyzed that same JSON log file for the data for "Acurite-609TXC 51" since it's my own thermometer, about 15m from the receiver, has a high SNR, and should represent a near-optimum case for analyzing device reliability vs data reliability and learning of any other anomalies in what would seem to be a standard manufactured device. It' relatively new, I think: about 18 months old. I The device JSON log file contained 8180 packets for that sensor: 2334 were unique transmissions (in which the time of reception was exactly the same, to the second) and 5846 were duplicate packets sent to increase the probability that the data represented in the transmission would be received ungarbled. The distribution of number of packets per transmission looks like this:
I had been assuming that the transmitter always transmitted the same same number of packets per transmission, which would seem to be 6 in this case. But the actual distribution of packet count per transmission is so broad that I'm not sure that assumption is correct. It's not important for the analysis that rtl_snr or rtl_watch do, since they ignore duplicates anyway, but it's a surprising (to me) result. If it does always transmit 6 packets, then the reception in this near-optimum situation is pretty poor -- only 25% of those attempts were completely successful. Now, that 2334 number isn't precisely the number of de-duplicated packets. Since the JSON log reports time only to the integer second, some number of those non-duplicated packets are actually duplicates within rounding error of the time precision. For example, there were two records at time 00:01:42 followed by two records at 00:01:43 that contained the same temperature and humidity values. The de-duplicating algorithm in rtl_snr and rtl_watch would report those four records as one transmission, at time 00:01:42. Here is the distribution of number of transmissions vs inter-transmission time gap:
Filtering out the packets with a 1- or 2-second time gap from the previous transmission yields 1660 de-duplicated records, close to the 1682 reported by rtl_snr and with SNR mean and standard deviations that match rtl_snr. I ascribe the 22-record discrepancy to similar issues in time-rounding and de-duplicating algorithm, but close enough to make more sense of the data. It seems clear that the sensor is intended to transmit at about a 33-second frequency, and 90% occur within 1 second of that target (plus or minus). But the inter-transmission isn't as precise as I would have expected ... there's some randomization. I went back to the original logs to confirm, for example, that there really was a spurious transmission -- single packet -- 4 seconds after a a prior one. I need to think about this a bit more, but I'm not sure how to determine device reliability based on inter-transmission gap timings given the variability I see in a device that is close and has a high SNR and low SNR standard deviation. But it is certainly the case that in this situation, a gap of more than 2x the target time of 33 seconds would be a good indicator -- for this device and proximity -- that something's wrong with the device. Maybe that's good enough for others who want that kind of alert. Thoughts? |
Beta Was this translation helpful? Give feedback.
-
Be careful not to assume that probability of reception is about SNR only. There are lots of things going on in ISM spectrum, and all components are far from ideal. The idea that "device reliability" and "reception reliability" are different is a good one. |
Beta Was this translation helpful? Give feedback.
-
If one has poor reception, then 3.5x nominal may alert more than you want. Note that battery status and stopping reporting are really not the same thing. I have devices that I use NiMH in and they always report low battery. |
Beta Was this translation helpful? Give feedback.
-
FYI, Naming a device is a problem a number of rtl_433 consumers have. It looks like rtl_watch chooses to use Note: for some devices rtl_433 and the Home Assistant auto discovery script concatenate the fields For scripting and correlating across utilities, it would be good to have some consistency where possible. |
Beta Was this translation helpful? Give feedback.
-
Rob,
Thanks for sending this. v1.3.0 includes the per-device packet and
transmission counts, too.
A couple of interesting things in the (for me, anyway):
* Your SNR standard deviations are much lower than mine, and my three
primary remote sensors are about 10-15m away. Maybe my receiving
antenna is not well placed. I'll move it and see.
* You collect a LOT of data over 4 days!
* in your column header, the Greek letter "sigma" comes out as a
4-byte unicode. I had thought that Python's text code were
standardized across platforms. My apologies to all, but I don't
know how to fix that other than to replace the Greek "sigma" single
character with "Std Dev". or maybe just "SD". Did I misunderstand
what's going on here?
David
…On 3/7/23 8:08 PM, rct wrote:
I have to say that I find the ITGT statistics not to be all that
helpful. After some detailed analysis of the packets received from
a couple of
If anyone else is interested in this issue, it would help me if
you could run that SNR-ITGT.py code against some of your log files
-- something with a week or month of data -- and let me know if
anything useful pop out of the data for you.
Have to stop for tonight, but wanted to share this from one of my
rtl_433 logs, there is a bit of interesting stuff in here for me:
|Processed 161618 de-duplicated records Dated from Thu 2023-03-02
11:44:17 to Mon 2023-03-06 12:22:32 Signal-to-Noise Inter-Tranmssion
Gap Time _________________________________
______________________________________ Device #Recs Mean ± \U0001d70e
Min Max #Recs Mean ± \U0001d70e Min Max Acurite-01185M 0 19 5.8 ± 2.9
3.2 12.6 18 15317.4s ± 26243.9 279.0 104349.0 Acurite-515 1783 5401
12.3 ± 0.9 6.5 32.0 5400 64.4s ± 15.4 60.0 243.0 Acurite-515 5514 230
5.0 ± 1.3 2.7 13.4 229 795.1s ± 1715.6 61.0 15776.0 Acurite-5n1 130 1
6.9 ± 0.0 6.9 6.9 Acurite-986 4995 1 9.4 ± 0.0 9.4 9.4 Acurite-Atlas
924 3689 6.7 ± 1.8 1.8 30.5 3688 93.8s ± 969.7 10.0 49417.0
Acurite-Tower 12053 19951 12.2 ± 0.9 5.2 38.5 19950 17.4s ± 4.2 11.0
165.0 Acurite-Tower 14322 20464 12.3 ± 0.9 4.5 37.8 20463 17.0s ± 4.0
10.0 162.0 Acurite-Tower 2633 20094 12.2 ± 0.9 5.7 36.8 20093 17.3s ±
4.2 13.0 178.0 Acurite-Tower 8 1 11.9 ± 0.0 11.9 11.9 Acurite-Tower
9884 20278 12.3 ± 0.9 5.0 37.4 20277 17.2s ± 4.2 13.0 178.0
Ambientweather-F007TH 109 1 11.6 ± 0.0 11.6 11.6 Ambientweather-F007TH
115 4243 12.3 ± 0.8 7.4 32.4 4242 82.0s ± 15.9 78.0 316.0
Ambientweather-F007TH 139 4544 12.3 ± 0.9 5.1 35.6 4543 76.6s ± 16.3
0.0 219.0 Ambientweather-F007TH 155 6287 12.3 ± 0.9 5.6 36.4 6286
55.3s ± 11.6 45.0 211.0 Ambientweather-F007TH 166 4251 12.2 ± 0.9 3.8
34.4 4250 81.8s ± 15.0 78.0 237.0 Ambientweather-F007TH 192 5425 12.3
± 0.9 6.2 34.8 5424 64.1s ± 14.5 60.0 244.0 Ambientweather-F007TH 2 17
11.6 ± 0.2 11.3 11.9 16 71.2s ± 25.5 57.0 114.0 Ambientweather-F007TH
218 4664 12.2 ± 0.9 6.5 34.7 4663 74.6s ± 17.0 70.0 355.0
Ambientweather-F007TH 241 5590 11.6 ± 1.5 4.8 32.8 5589 62.2s ± 14.5
58.0 236.0 Ambientweather-F007TH 42 4713 12.3 ± 0.9 7.8 35.4 4712
73.8s ± 14.4 70.0 213.0 Ambientweather-F007TH 64 5811 12.2 ± 0.9 6.0
38.9 5810 59.9s ± 14.2 56.0 456.0 DSC-Security 2795104 58 7.6 ± 1.8
2.8 12.6 57 3247.8s ± 2893.3 0.0 18873.0 Ford-CarRemote 0 3 10.9 ± 1.7
8.9 12.0 2 80750.0s ± 35486.9 55657.0 105843.0 Markisol 256 1 12.4 ±
0.0 12.4 12.4 Schrader 1F1DE39 1 5.9 ± 0.0 5.9 5.9 Schrader 1F1E02C 1
5.8 ± 0.0 5.8 5.8 Schrader AEABAAA 1 11.6 ± 0.0 11.6 11.6 Schrader
DDAAB5A 1 12.0 ± 0.0 12.0 12.0 Secplus-v1 37188652 1 4.2 ± 0.0 4.2 4.2
Thermopro-TP11 1080 25875 12.2 ± 0.7 5.8 14.0 25874 13.4s ± 4.8 11.0
312.0 Waveman-Switch A 1 12.9 ± 0.0 12.9 12.9 |
—
Reply to this email directly, view it on GitHub
<#2409 (reply in thread)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABJWBXC2XQ7TFGCFFLFXSZTW27LZHANCNFSM6AAAAAAVPTOEPQ>.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
I had an idea for analysis that might be interesting (or completely off the wall) that seems (to me) well suited to both rtl_snr and to a lesser degree rtl_watch: Add tracking for each device's frequency, so we can see variation and whether a device is close to the limits. Frequency drift is one of the potential factors impacting reception reliability. Temperature (and maybe voltage?) change can affect the oscillators in the small inexpensive sensors. So over time devices could drift either out of the rtl_433's pass band, or into the center "DC spike". With the default sampling rate of 250k, there is a 125khz range on both sides. How much of that is usable? How much does sensitivity drop as you get close to the edges? I know there is now some compensation for removing that center spike, but how much does the ability to recover the signal is lost as the device's frequency gets near that area? SNR tracking should show some (most?). But understanding if any frequency drift occurred could yield some clues as to why that might not otherwise be obvious. Ok, let me know if I'm way out there. |
Beta Was this translation helpful? Give feedback.
-
Rob,
This is a good idea (you're just full of them, aren't you! 😉) I had
noticed frequency drift between some of the individual packets but
hadn't thought about that as a data-reliability factor -- and obviously,
it is.
I could (easily?) add mean, std dev, min, & max freq to the basic stats
in SNR and see if there's anything meaningful in the log data. If there
is, it would be pretty straightforward to add freq drift to the set of
things that might trigger alarms or warnings in rtl_watch (along with
battery, status, SNR changes, and missed transmission window.
Added to my task list. This will be an interesting analysis
Thanks for the idea.
David
…On 3/9/23 10:21 AM, rct wrote:
I had an idea for analysis that might be interesting (or completely
off the wall) that seems (to me) well suited to both rtl_snr and to a
lesser degree rtl_watch:
Add tracking for each device's frequency, so we can see variation and
whether a device is close to the limits.
Frequency drift is one of the potential factors impacting reception
reliability. Temperature (and maybe voltage?) change can affect the
oscillators in the small inexpensive sensors. So over time devices
could drift either out of the rtl_433's pass band, or into the center
"DC spike".
With the default sampling rate of 250k, there is a 125khz range on
both sides. How much of that is usable? How much does sensitivity drop
as you get close to the edges?
I know there is now some compensation for removing that center spike,
but how much does the ability to recover the signal is lost as the
device's frequency gets near that area?
SNR tracking should show some (most?). But understanding if any
frequency drift occurred could yield some clues as to why that might
not otherwise be obvious.
Ok, let me know if I'm way out there.
—
Reply to this email directly, view it on GitHub
<#2409 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABJWBXDUV7OSW3NSPHGGFALW3HYQHANCNFSM6AAAAAAVPTOEPQ>.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
Sometimes it's informative to see what ISM-band devices are broadcasting in your neighborhood. While rtl_snr can do that by analyzing the JSON logs generated by
rtl_433
, you may just want a quick snapshot of ISM activity in your neighborhood.rtl_watch monitors the devices seen by
rtl_433
to generate and update a display window that shows the devicesrtl_433
has seen and the characteristics of those devices. It is a Python3 program that functions on Mac OSX and Raspbian (Debian) Linux, at least.Beta Was this translation helpful? Give feedback.
All reactions