Skip to content

drivers/dht: correct interpreting raw values #16934

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Nov 20, 2021
Merged

Conversation

wosym
Copy link
Member

@wosym wosym commented Oct 1, 2021

Contribution description

The existing driver only had an accuracy of 1°C instead of 0.1°C. This was due to an incorrect interpretation of the datasheet. The lower bits were discarded. This patch corrects the use of how the raw data should be interpreted and converted.

The proof of the pudding is in the eating:

2021-10-02 00:56:24,832 # RAW values: temp: 23.6 hum: 52.0
2021-10-02 00:56:24,837 # DHT values - temp: 23.6°C - relative humidity: 52.0%
2021-10-02 00:56:26,857 # RAW values: temp: 22.9 hum: 54.0
2021-10-02 00:56:26,861 # DHT values - temp: 22.9°C - relative humidity: 54.0%
2021-10-02 00:56:28,881 # RAW values: temp: 22.9 hum: 54.0
2021-10-02 00:56:28,886 # DHT values - temp: 22.9°C - relative humidity: 54.0%
2021-10-02 00:56:30,905 # RAW values: temp: 22.9 hum: 54.0
2021-10-02 00:56:30,910 # DHT values - temp: 22.9°C - relative humidity: 54.0%

Testing procedure

Run tests/drver_dht

NOTE: I only have a DHT11 available, while this driver is meant to also support DHT21 and 22. I checked the datasheet, and besides the MSB being a sign bit the implementation should be identical, but... I was not able to confirm this by testing it so if anyone else has this sensor... feel free!

Issues/PRs references

Fixes #16900

@github-actions github-actions bot added Area: drivers Area: Device drivers Area: tests Area: tests and testing framework labels Oct 1, 2021
@wosym wosym added CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors) labels Oct 1, 2021
@benpicco benpicco added the Process: needs backport Integration Process: The PR is required to be backported to a release or feature branch label Oct 4, 2021
@benpicco benpicco added this to the Release 2021.10 milestone Oct 4, 2021
@benpicco benpicco requested a review from haukepetersen October 5, 2021 21:56
@maribu
Copy link
Member

maribu commented Oct 6, 2021

Please don't merge this before confirming that this still works with the original DHT11 hardware that indeed only has a resolution of 1 degree Celsius. The new DHT11 seem to have the same resolution and as the DHT22.

@wosym
Copy link
Member Author

wosym commented Oct 6, 2021

@maribu Do you have a source for that? I tried finding old datasheets, but most don't have a date written on them. The only one with a date I could find was this datasheet from 2010 (which is older than the initial commit of this driver, which was in 2015).
In this datasheet they already mention 2*8-bit. One for the integral part, one for the decimal part. That is in line with what I wrote.

The driver in its current form also retrieves 16-bit of data, so this also does not suggest that this device only supports 1°C resolution. It attempts to retrieve the decimal part by doing some bit-operations, but since they are flawed, the decimal part gets lost.

On the other hand, I did find several discussions on the Arduino forums, debating whether the DHT11 has a 1°C or 0.1°C resolution. The answers vary, but I think the bottomline is that the Arduino library drops the fractional part on purpose because the sensor is not precise enough. Important note: this remark from 2014 is no longer relevant in 2021, because when I tested it last week the library did output the fractional part.
Is it possible that this is where the confusion stems from?

Personally I think a driver should not drop measurements, even if the precision of the device is unreliable. Either the driver should fix the precision in software, or (if it's unable) it should let the user decide to what accuracy it will trust the readings.

That being said: in my personal tests the readings did not fluctuate at all. I think that, if the readings are unreliable, it has low accuracy, but rather high precision. That means that the fractional part is definitely useful for a lot of applications (including mine).

As always, YMMV of course.

@maribu
Copy link
Member

maribu commented Oct 6, 2021

The data format of DHT11 and DHT22 was always identical. If I recall correctly, the old datasheets say that the decimal part is to be ignored by in case of the DHT11. This sounded like it contains pseudo-radom noise to me. I have an old DHT11 somewhere, I can see if the decimal part is just fixed zeroes (so no harm in just reading it), or something else. In case of the latter, we would need to distinguish between the two DHT11 versions.

@wosym
Copy link
Member Author

wosym commented Oct 6, 2021

You provided the source in your comment in the Issue. I will reply here to not have the conversation in two places.

From https://www.seeedstudio.com/blog/2020/04/20/dht11-vs-dht22-am2302-which-temperature-humidity-sensor-should-you-use/ :

DHT11 old vs new

So the accuracy of 1 degree actually is correct for the original DHT11. It is a pity that they don't have renamed the new one to e.g. DTH12 or so.

Looks like the Aosong is the older one. It's for that version that you can find most of the datasheets. An example is this one.

There is a note in that datasheet:

Note: The fractional portion wherein the temperature and humidity of 0

I'm guessing this Chinglish sentence refers to the resolution of 1°C. I interpret this as saying that the lower register is always 0.
If this is the case, I conclude two things: 1) The newer driver will still work with old sensors (but I hope you can confirm this by testing on an older device), 2) The existing driver is still flawed, because it does calculations on this empty register that do no make sense.

@maribu
Copy link
Member

maribu commented Oct 6, 2021

The existing driver is still flawed, because it does calculations on this empty register that do no make sense.

If this about saving CPU cycles: IMO users will happily spend them if they don't have to configure whether they have old or new DHT11 sensors. IMO if the fractional part is zero for the older, that is the best case and we can just ignore the difference between them.

@wosym
Copy link
Member Author

wosym commented Oct 6, 2021

This is not about saving CPU cycles. Heck, I don't even know if my implementation uses more or less.
It's because the calculations were, as far as my untrained eye could see, wrong. It appears like the original author assumed that the upper and lower register were two parts of a 16-bit integer. He then combined them and divided to a meaningful number. That's not what the datasheet describes the registers represent.

@benpicco benpicco removed the Process: needs backport Integration Process: The PR is required to be backported to a release or feature branch label Oct 12, 2021
@benpicco benpicco removed this from the Release 2021.10 milestone Oct 12, 2021
@benpicco
Copy link
Contributor

It would be good to verify this still yields correct results with the old sensor - does anyone have one of them?

@maribu
Copy link
Member

maribu commented Oct 12, 2021

I do, just need some time to harvest it from an abandoned PCB

@maribu
Copy link
Member

maribu commented Oct 17, 2021

On a Seeeduino Xiao running examples/saul with a DHT11 (first new, later old revision) connected to PA4 and the following diff:

diff --git a/examples/saul/Makefile b/examples/saul/Makefile
index 7f2ca1e297..61631d30e2 100644
--- a/examples/saul/Makefile
+++ b/examples/saul/Makefile
@@ -14,6 +14,7 @@ USEMODULE += shell
 USEMODULE += shell_commands
 # additional modules for debugging:
 USEMODULE += ps
+USEMODULE += dht
 
 # Comment this out to disable code in RIOT that does safety checking
 # which is not needed in a production environment but helps in the
@@ -24,3 +25,5 @@ DEVELHELP ?= 1
 QUIET ?= 1
 
 include $(RIOTBASE)/Makefile.include
+
+CFLAGS += '-DDHT_PARAM_PIN=GPIO_PIN(0,4)'
2021-10-17 20:13:27,446 # saul
2021-10-17 20:13:27,447 # ID	Class		Name
2021-10-17 20:13:27,449 # #0	ACT_SWITCH	LED(BLUE_RX)
2021-10-17 20:13:27,450 # #1	ACT_SWITCH	LED(BLUE_TX)
2021-10-17 20:13:27,451 # #2	ACT_SWITCH	LED(YELLOW_USER)
2021-10-17 20:13:27,451 # #3	SENSE_TEMP	dht
2021-10-17 20:13:27,452 # #4	SENSE_HUM	dht

Reading New DHT11

Communication new DHT11 revision

> saul read 4
2021-10-17 20:15:49,841 # saul read 4
2021-10-17 20:15:49,866 # Reading from #4 (dht|SENSE_HUM)
2021-10-17 20:15:49,866 # Data:	           69.0 %
> saul read 3
2021-10-17 20:15:50,720 # saul read 3
2021-10-17 20:15:50,721 # Reading from #3 (dht|SENSE_TEMP)
2021-10-17 20:15:50,722 # Data:	           20.5 °C

Reading Old DHT11

Communication new DHT11 revision

2021-10-17 20:17:23,605 # saul read 4
2021-10-17 20:17:23,630 # Reading from #4 (dht|SENSE_HUM)
2021-10-17 20:17:23,630 # Data:	           43.0 %

Communication new DHT11 revision

2021-10-17 20:17:24,707 # saul read 3
2021-10-17 20:17:24,732 # Reading from #3 (dht|SENSE_TEMP)
2021-10-17 20:17:24,733 # Data:	           23.0 °C

@maribu
Copy link
Member

maribu commented Oct 17, 2021

Btw: The driver still works when dropping the distinction between DHT2x and DHT11 altogether:

diff --git a/drivers/dht/dht.c b/drivers/dht/dht.c
index 68c86e05df..359777bc6e 100644
--- a/drivers/dht/dht.c
+++ b/drivers/dht/dht.c
@@ -111,7 +111,6 @@ int dht_init(dht_t *dev, const dht_params_t *params)
 
     /* check parameters and configuration */
     assert(dev && params);
-    assert((params->type == DHT11) || (params->type == DHT22) || (params->type == DHT21));
 
     memset(dev, 0, sizeof(dht_t));
     dev->params = *params;
@@ -193,26 +192,15 @@ int dht_read(dht_t *dev, int16_t *temp, int16_t *hum)
 
         /* parse the RAW values */
         DEBUG("RAW values: temp: %2i.%i hum: %2i.%i\n", (int)raw_temp_I,
-            (int)raw_temp_D, (int)raw_hum_I, (int)raw_hum_D);
-
-        switch (dev->params.type) {
-            case DHT11:
-                dev->last_val.temperature = raw_temp_I * 10 + raw_temp_D;
-                dev->last_val.humidity = raw_hum_I * 10 + raw_hum_D;
-                break;
-            /* DHT21 == DHT22 (same value in enum), so both are handled here */
-            case DHT22:
-                dev->last_val.humidity = raw_hum_I * 10 + raw_hum_D;
-                /* if the high-bit is set, the value is negative */
-                if (raw_temp_I & 0x80) {
-                    dev->last_val.temperature = ((raw_temp_I & ~0x80) * 10 + raw_temp_D) * -1;
-                }
-                else {
-                    dev->last_val.temperature = raw_temp_I * 10 + raw_temp_D;
-                }
-                break;
-            default:
-                return DHT_NODEV;      /* this should never be reached */
+              (int)raw_temp_D, (int)raw_hum_I, (int)raw_hum_D);
+
+        dev->last_val.humidity = raw_hum_I * 10 + raw_hum_D;
+        /* if the high-bit is set, the value is negative */
+        if (raw_temp_I & 0x80) {
+            dev->last_val.temperature = ((raw_temp_I & ~0x80) * 10 + raw_temp_D) * -1;
+        }
+        else {
+            dev->last_val.temperature = raw_temp_I * 10 + raw_temp_D;
         }
 
         /* update time of last measurement */
diff --git a/drivers/dht/include/dht_params.h b/drivers/dht/include/dht_params.h
index 8cd84f6aec..ce3bdcfa34 100644
--- a/drivers/dht/include/dht_params.h
+++ b/drivers/dht/include/dht_params.h
@@ -34,15 +34,11 @@ extern "C" {
 #ifndef DHT_PARAM_PIN
 #define DHT_PARAM_PIN               (GPIO_PIN(0, 0))
 #endif
-#ifndef DHT_PARAM_TYPE
-#define DHT_PARAM_TYPE              (DHT11)
-#endif
 #ifndef DHT_PARAM_PULL
 #define DHT_PARAM_PULL              (GPIO_IN_PU)
 #endif
 #ifndef DHT_PARAMS
 #define DHT_PARAMS                  { .pin     = DHT_PARAM_PIN,  \
-                                      .type    = DHT_PARAM_TYPE, \
                                       .in_mode = DHT_PARAM_PULL }
 #endif
 #ifndef DHT_SAULINFO
diff --git a/drivers/include/dht.h b/drivers/include/dht.h
index ec54417245..eb0d1b5383 100644
--- a/drivers/include/dht.h
+++ b/drivers/include/dht.h
@@ -46,7 +46,6 @@ enum {
     DHT_OK      =  0,       /**< all good */
     DHT_NOCSUM  = -1,       /**< checksum error */
     DHT_TIMEOUT = -2,       /**< communication timed out */
-    DHT_NODEV   = -3        /**< device type not defined */
 };
 
 /**
@@ -57,24 +56,13 @@ typedef struct {
     uint16_t temperature;   /**< temperature in deci-Celsius */
 } dht_data_t;
 
-/**
- * @brief   Device type of the DHT device
- */
-typedef enum {
-    DHT11,                  /**< DHT11 device identifier */
-    DHT22,                  /**< DHT22 device identifier */
-    DHT21 = DHT22           /**< DHT21 device identifier */
-} dht_type_t;
-
 /**
  * @brief   Configuration parameters for DHT devices
  */
 typedef struct {
     gpio_t pin;             /**< GPIO pin of the device's data pin */
-    dht_type_t type;        /**< type of the DHT device */
     gpio_mode_t in_mode;    /**< input pin configuration, with or without pull
                              *   resistor */
-
 } dht_params_t;
 
 /**
@@ -110,7 +98,6 @@ int dht_init(dht_t *dev, const dht_params_t *params);
  * @retval DHT_OK       Success
  * @retval DHT_NOCSUM   Checksum error
  * @retval DHT_TIMEOUT  Reading data timed out (check wires!)
- * @retval DHT_NODEV    Unsupported device type specified
  */
 int dht_read(dht_t *dev, int16_t *temp, int16_t *hum);
 

Basically, the new DHT11 and DHT22 speak the exact same protocol with the same resolution, only the accuracy differs. (It would not surprise me much if the new DHT11 would be a DHT22 in a different casing and no calibration data added.)

@fjmolinas
Copy link
Contributor

Whats the status here @maribu @wosym? Should your patch be applied and we merge this one?

@wosym
Copy link
Member Author

wosym commented Nov 18, 2021

@fjmolinas Sorry, I've been a bit preoccupied lately. I'll try to have it finished up by tomorrow evening.

@wosym wosym requested a review from maribu as a code owner November 19, 2021 23:59
@github-actions github-actions bot added Area: cpu Area: CPU/MCU ports Platform: AVR Platform: This PR/issue effects AVR-based platforms labels Nov 19, 2021
@wosym wosym force-pushed the dht11fix branch 2 times, most recently from a9e860e to fa6f3a9 Compare November 20, 2021 00:06
@wosym
Copy link
Member Author

wosym commented Nov 20, 2021

Maribu's suggestion of removing the distinction entirely was indeed a good one. I modified the code (I think I ended up with an almost identical patch to what Maribu had), and tested it again: everything still works as expected.

At first I considered removing the if/else entirely and turn it into a one-liner. It would save a couple of clockticks, but for clarity sake I think it's best to leave the if/else in there.

If you agree, feel free to merge. If you object, let me know and I will adjust it again.

Apologies for the long delay!

(Ps. github-actions labeled this as cpu and AVR somehow? Did I touch any files that are related to AVR without realizing it?)

EDIT: whoops, I must have messed some stuff up with my git-dabbling. That explains the AVR label... will fix.
EDIT2: fixed. Sorry for making such a mess 🤡

@wosym wosym removed Platform: AVR Platform: This PR/issue effects AVR-based platforms Area: cpu Area: CPU/MCU ports labels Nov 20, 2021
Copy link
Member

@maribu maribu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. IMO it would make sense to split the extension of the code owner file into a separate commit (but within this PR). Squash at will.

@wosym
Copy link
Member Author

wosym commented Nov 20, 2021

Looks good to me. IMO it would make sense to split the extension of the code owner file into a separate commit (but within this PR). Squash at will.

I assumed it was preferred to be done in the same commit. Good to know that this assumption was wrong! I split it off into a separate commit.

Copy link
Member

@maribu maribu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACK. I did test that old DHT11 can be treated the same way as new DHT11 or DHT2*. So IMO we can just trust your testing and don't to retest again on legacy DHT11 and DHT22.

@maribu maribu merged commit 7a2ebfb into RIOT-OS:master Nov 20, 2021
@wosym
Copy link
Member Author

wosym commented Nov 20, 2021

Thanks for the help and testing @maribu ! :)

@maribu
Copy link
Member

maribu commented Nov 20, 2021

Thanks for fixing the driver :-)

@hugueslarrive
Copy link
Contributor

hugueslarrive commented May 26, 2023

Basically, the new DHT11 and DHT22 speak the exact same protocol with the same resolution, only the accuracy differs.

I'm not so sure about that as it breaks my end-of-2021 application that uses an AM2301 sensor (DHT21?).

For this sensor, there isn't one byte for the integral part and one for the decimal part. Instead, the two bytes form a 16-bit value (in tenths of a percent or tenths of a degree), so the high byte needs to be multiplied by 256 instead of 10 (left-shifted by 8), as explained in section 7.2 of the manual: https://www.haoyuelectronics.com/Attachment/AM2301/AM2301.pdf

Result of tests/drivers/dht:

2023-05-26 23:11:38,727 # DHT values - temp: 21.0°C - relative humidity: 5.9%
2023-05-26 23:11:40,753 # DHT values - temp: 21.0°C - relative humidity: 5.8%
2023-05-26 23:11:42,778 # DHT values - temp: 21.1°C - relative humidity: 6.0%
2023-05-26 23:11:44,802 # DHT values - temp: 21.0°C - relative humidity: 5.8%
2023-05-26 23:11:46,828 # DHT values - temp: 21.0°C - relative humidity: 5.9%
2023-05-26 23:11:48,853 # DHT values - temp: 21.0°C - relative humidity: 5.8%
2023-05-26 23:11:50,877 # DHT values - temp: 21.1°C - relative humidity: 6.2%
2023-05-26 23:11:52,902 # DHT values - temp: 21.0°C - relative humidity: 6.3%
2023-05-26 23:11:54,928 # DHT values - temp: 22.1°C - relative humidity: 21.4%
2023-05-26 23:11:56,952 # DHT values - temp: 23.0°C - relative humidity: 26.1%
2023-05-26 23:11:58,978 # DHT values - temp: 23.2°C - relative humidity: 26.1%
2023-05-26 23:12:01,003 # DHT values - temp: 24.8°C - relative humidity: 26.1%
2023-05-26 23:12:03,028 # DHT values - temp: 24.7°C - relative humidity: 26.1%
2023-05-26 23:12:05,053 # DHT values - temp: 25.0°C - relative humidity: 26.1%
2023-05-26 23:12:07,078 # DHT values - temp: 1.9°C - relative humidity: 26.1%
2023-05-26 23:12:09,103 # DHT values - temp: 1.6°C - relative humidity: 26.1%
2023-05-26 23:12:11,128 # DHT values - temp: 2.6°C - relative humidity: 26.1%
2023-05-26 23:12:13,153 # DHT values - temp: 2.6°C - relative humidity: 26.1%
2023-05-26 23:12:15,220 # DHT values - temp: 2.2°C - relative humidity: 26.1%
2023-05-26 23:12:17,245 # DHT values - temp: 1.9°C - relative humidity: 26.1%
2023-05-26 23:12:19,270 # DHT values - temp: 1.8°C - relative humidity: 26.1%
2023-05-26 23:12:21,295 # DHT values - temp: 1.6°C - relative humidity: 26.1%

@wosym
Copy link
Member Author

wosym commented May 26, 2023

@hugueslarrive Mhmmm... First time I hear of that sensor, but Google seems to be inconclusive about what it actually is. One source says (translated) "The AM2301 is significantly more accurate than the DHT11 and similar tot the DHT22". This would imply that it's a different sensor and would also need a different driver. (Or an extra conditional to this one)

However, other sources say that it's just a repackaged version of the DHT21, and that the other AM230x sensors are also repackaged versions of DHTxy sensors.

I think it's going to be very hard to create a generic driver if the Chinese can't come up with a consistent numbering scheme for their products...

@hugueslarrive
Copy link
Contributor

I think I order them as DHT21...

Here is the old code that differentiated between DHT11 and DHT21/22:

        /* parse the RAW values */
        DEBUG("RAW values: temp: %7i hum: %7i\n", (int)raw_temp, (int)raw_hum);
        switch (dev->params.type) {
            case DHT11:
                dev->last_val.temperature = (int16_t)((raw_temp >> 8) * 10);
                dev->last_val.humidity = (int16_t)((raw_hum >> 8) * 10);
                break;
            /* DHT21 == DHT22 (same value in enum), so both are handled here */
            case DHT22:
                dev->last_val.humidity = (int16_t)raw_hum;
                /* if the high-bit is set, the value is negative */
                if (raw_temp & 0x8000) {
                    dev->last_val.temperature = (int16_t)((raw_temp & ~0x8000) * -1);
                }
                else {
                    dev->last_val.temperature = (int16_t)raw_temp;
                }
                break;
            default:
                return DHT_NODEV;      /* this should never be reached */
        }

As you can see, in the case of DHT11, the 8 most significant bits were right-shifted by 8 (divided by 256) and then multiplied by 10. However, for DHT21/DHT22, the 16-bit values (in tenths) were copied as is.

@wosym
Copy link
Member Author

wosym commented May 26, 2023

Mhmmmm, when looking back it does indeed not make much sense that the distinction was removed. The original issue was also for DHT11, and not for 21/22. It was for 11 that the calculation was wrong.

@maribu do you remember why you suggested dropping the distinction? From reading the comments, I also think we never tested it (properly) on DHT21/22?

@maribu
Copy link
Member

maribu commented May 27, 2023

In #16934 (comment) I suggested to drop special handling for the DHT11 case and just treat it as DHT22 as well, as there was no difference in the computation. But it seems that while there was indeed no difference in the two cases in the PR at this stage, in master prior to this PR the computation was indeed different. And now everything is treated as DHT11, so DHT2x sensor readings are now broken.

Sorry for not double checking the PR. I'll provide a PR to fix this on Tuesday.

hugueslarrive added a commit to hugueslarrive/RIOT that referenced this pull request May 27, 2023
hugueslarrive added a commit to hugueslarrive/RIOT that referenced this pull request May 27, 2023
@hugueslarrive
Copy link
Contributor

I have opened PR #19674 since I had already fixed the code...

hugueslarrive added a commit to hugueslarrive/RIOT that referenced this pull request May 31, 2023
This PR reintroduces the specific handling for DHT22 that was mistakenly removed by RIOT-OS#16934 (drivers/dht: correct interpreting raw values).

The improvement in value handling for DHT11 has been preserved, except for the detection of negative values, which now occurs on the 4th byte for DHT11 instead of the 3rd byte for DHT22.

Moreover :
- improvement in reliability on ESP8266
- make it works on avr

Co-authored-by: Marian Buschsieweke <maribu@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: drivers Area: Device drivers Area: tests Area: tests and testing framework CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR Type: bug The issue reports a bug / The PR fixes a bug (including spelling errors)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

drivers/dht : driver not providing full accuracy
5 participants