One useful pattern to track device up/down status is to have a will set to .../status with value '0' and retain, and when the device comes up to publish to the .../status with value '1' with retain. The retain allows the status to be interrogated at anytime by a client. The library has retain flag for will, but not for publish. Of course, it would be useful for lots of other things too, but thought I would point out one very valuable one...

After upgrading vom ESP Version 2.3 to 2.4 MQTT_Library (0.17.0) failed to build with following errors:

Users/joerg/Documents/Arduino/libraries/Adafruit_MQTT_Library/Adafruit_MQTT_Client.cpp: In member function 'virtual bool Adafruit_MQTT_Client::sendPacket(uint8_t*, uint16_t)': /Users/joerg/Documents/Arduino/libraries/Adafruit_MQTT_Library/Adafruit_MQTT_Client.cpp:84:38: error: no matching function for call to 'min(uint16_t&, int)' uint16_t sendlen = min(len, 250); ^ /Users/joerg/Documents/Arduino/libraries/Adafruit_MQTT_Library/Adafruit_MQTT_Client.cpp:84:38: note: candidates are: In file included from /Users/joerg/Library/Arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/1.20.0-26-gb404fb9-2/xtensa-lx106-elf/include/c++/4.8.2/algorithm:62:0, from /Users/joerg/Library/Arduino15/packages/esp8266/hardware/esp8266/2.4.0/cores/esp8266/Arduino.h:240, from /Users/joerg/Documents/Arduino/libraries/Adafruit_MQTT_Library/Adafruit_MQTT.h:25, from /Users/joerg/Documents/Arduino/libraries/Adafruit_MQTT_Library/Adafruit_MQTT_Client.h:26, from /Users/joerg/Documents/Arduino/libraries/Adafruit_MQTT_Library/Adafruit_MQTT_Client.cpp:22: /Users/joerg/Library/Arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/1.20.0-26-gb404fb9-2/xtensa-lx106-elf/include/c++/4.8.2/bits/stl_algo.h:4226:5: note: template<class _Tp, class _Compare> _Tp std::min(std::initializer_list<_Tp>, _Compare) min(initializer_list<_Tp> __l, _Compare __comp) ^ /Users/joerg/Library/Arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/1.20.0-26-gb404fb9-2/xtensa-lx106-elf/include/c++/4.8.2/bits/stl_algo.h:4226:5: note: template argument deduction/substitution failed: /Users/joerg/Documents/Arduino/libraries/Adafruit_MQTT_Library/Adafruit_MQTT_Client.cpp:84:38: note: mismatched types 'std::initializer_list<_Tp>' and 'short unsigned int' uint16_t sendlen = min(len, 250); ^ In file included from /Users/joerg/Library/Arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/1.20.0-26-gb404fb9-2/xtensa-lx106-elf/include/c++/4.8.2/algorithm:62:0, from /Users/joerg/Library/Arduino15/packages/esp8266/hardware/esp8266/2.4.0/cores/esp8266/Arduino.h:240, from /Users/joerg/Documents/Arduino/libraries/Adafruit_MQTT_Library/Adafruit_MQTT.h:25, from /Users/joerg/Documents/Arduino/libraries/Adafruit_MQTT_Library/Adafruit_MQTT_Client.h:26, from /Users/joerg/Documents/Arduino/libraries/Adafruit_MQTT_Library/Adafruit_MQTT_Client.cpp:22: /Users/joerg/Library/Arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/1.20.0-26-gb404fb9-2/xtensa-lx106-elf/include/c++/4.8.2/bits/stl_algo.h:4221:5: note: template _Tp std::min(std::initializer_list<_Tp>) min(initializer_list<_Tp> __l) ^ /Users/joerg/Library/Arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/1.20.0-26-gb404fb9-2/xtensa-lx106-elf/include/c++/4.8.2/bits/stl_algo.h:4221:5: note: template argument deduction/substitution failed: /Users/joerg/Documents/Arduino/libraries/Adafruit_MQTT_Library/Adafruit_MQTT_Client.cpp:84:38: note: mismatched types 'std::initializer_list<_Tp>' and 'short unsigned int' uint16_t sendlen = min(len, 250); ^ In file included from /Users/joerg/Library/Arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/1.20.0-26-gb404fb9-2/xtensa-lx106-elf/include/c++/4.8.2/algorithm:61:0, from /Users/joerg/Library/Arduino15/packages/esp8266/hardware/esp8266/2.4.0/cores/esp8266/Arduino.h:240, from /Users/joerg/Documents/Arduino/libraries/Adafruit_MQTT_Library/Adafruit_MQTT.h:25, from /Users/joerg/Documents/Arduino/libraries/Adafruit_MQTT_Library/Adafruit_MQTT_Client.h:26, from /Users/joerg/Documents/Arduino/libraries/Adafruit_MQTT_Library/Adafruit_MQTT_Client.cpp:22: /Users/joerg/Library/Arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/1.20.0-26-gb404fb9-2/xtensa-lx106-elf/include/c++/4.8.2/bits/stl_algobase.h:239:5: note: template<class _Tp, class _Compare> const _Tp& std::min(const _Tp&, const _Tp&, _Compare) min(const _Tp& __a, const _Tp& __b, _Compare __comp) ^ /Users/joerg/Library/Arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/1.20.0-26-gb404fb9-2/xtensa-lx106-elf/include/c++/4.8.2/bits/stl_algobase.h:239:5: note: template argument deduction/substitution failed: /Users/joerg/Documents/Arduino/libraries/Adafruit_MQTT_Library/Adafruit_MQTT_Client.cpp:84:38: note: deduced conflicting types for parameter 'const _Tp' ('short unsigned int' and 'int') uint16_t sendlen = min(len, 250); ^ In file included from /Users/joerg/Library/Arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/1.20.0-26-gb404fb9-2/xtensa-lx106-elf/include/c++/4.8.2/algorithm:61:0, from /Users/joerg/Library/Arduino15/packages/esp8266/hardware/esp8266/2.4.0/cores/esp8266/Arduino.h:240, from /Users/joerg/Documents/Arduino/libraries/Adafruit_MQTT_Library/Adafruit_MQTT.h:25, from /Users/joerg/Documents/Arduino/libraries/Adafruit_MQTT_Library/Adafruit_MQTT_Client.h:26, from /Users/joerg/Documents/Arduino/libraries/Adafruit_MQTT_Library/Adafruit_MQTT_Client.cpp:22: /Users/joerg/Library/Arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/1.20.0-26-gb404fb9-2/xtensa-lx106-elf/include/c++/4.8.2/bits/stl_algobase.h:193:5: note: template const _Tp& std::min(const _Tp&, const _Tp&) min(const _Tp& __a, const _Tp& __b) ^ /Users/joerg/Library/Arduino15/packages/esp8266/tools/xtensa-lx106-elf-gcc/1.20.0-26-gb404fb9-2/xtensa-lx106-elf/include/c++/4.8.2/bits/stl_algobase.h:193:5: note: template argument deduction/substitution failed: /Users/joerg/Documents/Arduino/libraries/Adafruit_MQTT_Library/Adafruit_MQTT_Client.cpp:84:38: note: deduced conflicting types for parameter 'const _Tp' ('short unsigned int' and 'int') uint16_t sendlen = min(len, 250); ^ exit status 1 Fehler beim Kompilieren für das Board Generic ESP8266 Module.

This fixes pings discarding publish messages by always checking for a PUBLISH packet when inside processPacketsUntil. This ensures topic messages will always get handled even if we're pinging.

Also found a bug in Adafruit_MQTT_Client::readPacket which would actually read a byte rather than just returning. This would cause it to fail to read the next packet as the packet type byte had already been read.

I suspect these changes will address issues #141 , #132 , and #83 . I have not tried reproducing them though to verify.

there seem to be a couple issues with the publish packet length. currently the payload seems to be limited to 128 chars. i think the problem is related to length being stored as uint16_t, and sent as a uint8_t: https://github.com/adafruit/Adafruit_MQTT_Library/blob/master/Adafruit_MQTT.cpp#L517

also, maybe we could make publish return error codes, and generalize the error code printer? that way we could notify the user of the buffer length limit?

https://github.com/adafruit/Adafruit_MQTT_Library/blob/974f4b8713a83d01b040a0d23b798d3f2978d990/Adafruit_MQTT.cpp#L464

would result in an error if the topicname is larger than 255 bytes

The function uint16_t Adafruit_MQTT::publishPacket has buffer overflow if you send a packet that is larger than the MAXBUFFERSIZE (topic len + data len).

// as per http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html#_Toc398718040
uint16_t Adafruit_MQTT::publishPacket(uint8_t *packet, const char *topic,
                                     uint8_t *data, uint16_t bLen, uint8_t qos) {
  uint8_t *p = packet;
  uint16_t len=0;

  // calc length of non-header data
  len += 2;               // two bytes to set the topic size
  len += strlen(topic); // topic length
  if(qos > 0) { 
    len += 2; // qos packet id
  }
  len += bLen; // payload length

  // Now you can start generating the packet!
  p[0] = MQTT_CTRL_PUBLISH << 4 | qos << 1;
  p++;

  // fill in packet[1] last
  do {
    uint8_t encodedByte = len % 128;
    len /= 128;
    // if there are more data to encode, set the top bit of this byte
    if ( len > 0 ) {
      encodedByte |= 0x80;
    }
    p[0] = encodedByte;
    p++;
  } while ( len > 0 );

  // topic comes before packet identifier
  p = stringprint(p, topic);

  // add packet identifier. used for checking PUBACK in QOS > 0
  if(qos > 0) {
    p[0] = (packet_id_counter >> 8) & 0xFF;
    p[1] = packet_id_counter & 0xFF;
    p+=2;

    // increment the packet id
    packet_id_counter++;
  }

  memmove(p, data, bLen);
  p+= bLen;
  len = p - packet;
  DEBUG_PRINTLN(F("MQTT publish packet:"));
  DEBUG_PRINTBUFFER(buffer, len);
  return len;
}

As you can see from the code there isn't any check if we already went past the packet size and there fore we write over the boundaries.

Fixes https://github.com/adafruit/Adafruit_MQTT_Library/issues/109 Fixes https://github.com/adafruit/Adafruit_MQTT_Library/issues/122

After spending a long time pulling my hair to understand why I was hitting a panic when attempting to read from my registered subscriptions, I found out that the subscriptions member of the Adafruit_MQTT instance was corrupted. :(

Turns out the memory corruption was caused by my publish call, where the payload I was providing was bigger than the allocated space in the buffer for construction of the packet (see buffer[MAXBUFFERSIZE]).

To protect myself from ever making this mistake again, I am proposing a simple logic in publishPacket where instead of silently corrupting memory, the code uses as much payload as it can fit in the available space. By seeing the truncated payload, user can decide whether he/she should 1)break it up into different topics, 2) put the payload on a diet, or 3) increase MAXBUFFERSIZE.

====

• Describe the scope of your change--i.e. what the change does and what parts of the code were modified. This will help us understand any risks of integrating the code.

This pull requests improves the handling of buffer for packet creation of Adafruit_MQTT::publishPacket. It adds argument maxPacketLen to keep track of how much space is available in buffer.

• Describe any known limitations with your change. For example if the change doesn't apply to a supported platform of the library please mention it.

This change is not modifying the sizes of any buffers. Instead, it truncates the payload to avoid memory corruption of the class Adafruit_MQTT by keeping packet creation from going beyond MAXBUFFERSIZE.

• Please run any tests or examples that can exercise your modified code. We strive to not break users of the code and running tests/examples helps with this process.

See this gist with an example code where a packet has a payload that exceeds MAXBUFFERSIZE and corrupts packet_id_counter and subscriptions of the Adafruit_MQTT instance.

See comments below that gist with the crash and stack trace of code above, where memory is corrupted and the LoadProhibited exception takes place.

Lastly, see the comment that follows with the proposed modification, where the same code simply gets the payload truncated instead of allowing memory to get messed up.

• Arduino board: NodeMCU (ESP8266)
• Arduino IDE version: 1.8.5
• Library version: 0.20.3

On my Ubuntu 18.04 laptop, I installed mosquito and ran the following script:

while true; do mosquitto_pub -t robot/commands -m `date +%s%N`; sleep 0.5; done;

Then on the ESP8266 I'm pretty much running the mqtt_esp8266_callback example, modified to connect to my WiFi, my local Mosquito broker, and subscribe to my topic rather than then ones in the example.

If I run it at a one second interval, it's fine, but if I run it with anything less, I get the output below (debugging enabled). On Wireshark everything looks completely normal.

Read data:		0 [0x30], 
Packet Type:		0 [0x30], 
Read data:		# [0x23], 
Packet Length:	35
Read data:		  [0x00],   [0x0E], r [0x72], o [0x6F], b [0x62], o [0x6F], t [0x74], / [0x2F], 
	c [0x63], o [0x6F], m [0x6D], m [0x6D], a [0x61], n [0x6E], d [0x64], s [0x73], 
	1 [0x31], 5 [0x35], 3 [0x33], 7 [0x37], 0 [0x30], 2 [0x32], 0 [0x30], 3 [0x33], 
	9 [0x39], 7 [0x37], 9 [0x39], 7 [0x37], 9 [0x39], 7 [0x37], 7 [0x37], 7 [0x37], 
	8 [0x38], 0 [0x30], 9 [0x39], 
Dropped a packet

For comparison, this is what it looks like with a correctly read packet:

Read data:		0 [0x30], 
Packet Type:		0 [0x30], 
Read data:		# [0x23], 
Packet Length:	35
Read data:		  [0x00],   [0x0E], r [0x72], o [0x6F], b [0x62], o [0x6F], t [0x74], / [0x2F], 
	c [0x63], o [0x6F], m [0x6D], m [0x6D], a [0x61], n [0x6E], d [0x64], s [0x73], 
	1 [0x31], 5 [0x35], 3 [0x33], 7 [0x37], 0 [0x30], 2 [0x32], 0 [0x30], 9 [0x39], 
	4 [0x34], 3 [0x33], 9 [0x39], 2 [0x32], 4 [0x34], 1 [0x31], 4 [0x34], 7 [0x37], 
	0 [0x30], 0 [0x30], 6 [0x36], 
Packet len: 37
	0 [0x30], # [0x23],   [0x00],   [0x0E], r [0x72], o [0x6F], b [0x62], o [0x6F], 
	t [0x74], / [0x2F], c [0x63], o [0x6F], m [0x6D], m [0x6D], a [0x61], n [0x6E], 
	d [0x64], s [0x73], 1 [0x31], 5 [0x35], 3 [0x33], 7 [0x37], 0 [0x30], 2 [0x32], 
	0 [0x30], 9 [0x39], 4 [0x34], 3 [0x33], 9 [0x39], 2 [0x32], 4 [0x34], 1 [0x31], 
	4 [0x34], 7 [0x37], 0 [0x30], 0 [0x30], 6 [0x36], 
Looking for subscription len 14
Found sub #0
Data len: 19
Data: 1537020943924147006

hello, I am working on a project that includes sim800l accessing adafruit MQTT .it works fine with an arduino .my project need to use esp8266 12E with sim800l. I know esp8266 can be use standalone for MQTT but i need to use it as mcu for gsm module. Fona MQTT example shows this error"Error compiling for board NodeMCU 1.0 (ESP-12E Module)" . I am new to arduino or esp8266 , i hope this is a relevant problem.What am I doing wrong here,is there any alternative way to work with esp8266 with sim800l for accessing Adafruit MQTT?

Hi there,

I'm combining a Huzzah esp8266 with a HDC1008 and a photoresistor.

WiFi uses lots of battery so my hope was to be able to store on the esp8266 a series of data points with timestamps and then when the memory was running low or the chip's battery voltage appears to be dropping too low, publish the data points with their historical timestamps to Adafruit IO feeds.

Publish() seems to only take one value. Is there any way to do what I'm describing?

This adds retains support to published messages. It should be backwards compatible with existing clients as it defaults to false if not supplied.

Solves #20 and #161

Environment Here is the environment that I'm using':

Microconroller: . ESP8266 Core/runtime: [e.g. ESP8266 core for Arduino v3.0.2] IDE: [Arduino IDE 1.8.19] mqtt library: Adafruit_mqtt_client.h Hai iam having the issues with the port. I have saved the mqtt configurations in SPIFFS and initialized with structure. When iam trying to connect with mqtt its failed. But its getting connected when the port number defined globally. Any thoughts?

Env: Esp8266, communication via TLS

Symptom: When too many subscriber are registered, the last few one will never execute their callback.

reproducing Subscribe to multiple topics (17 in my case) with this syntax

Adafruit_MQTT_Subscribe trigg1 = Adafruit_MQTT_Subscribe(&mqtt, "triggTopic1", MQTT_QOS_1);
trigg1.setCallback(foo);
mqtt.subscribe(&trigg1);

Adafruit_MQTT_Subscribe trigg2 = Adafruit_MQTT_Subscribe(&mqtt, "triggTopic2", MQTT_QOS_1);
trigg1.setCallback(bar);
mqtt.subscribe(&trigg2);

etc... The last few subsriber will get ignored, silently (!!!!!) (they will never execute their callback, despite a successfull compilation, thus very confusing to debug ) There seems to be a silent hidden limit on the max. number of subscribed topics.

When using a local broker with an IP rather than a hostname, I experienced an inability to connect (error code -1). Using a different MQTT library with exactly the same connection settings and credentials worked perfectly.

Unfortunately, even when enabling the debug settings, I didn't get any more information to help diagnose - client->connect returned 0 in connectServer, but I didn't get any deeper into diagnosis.

• Arduino board: ESP32

• Arduino IDE version (found in Arduino -> About Arduino menu): 2.0.0

• List the steps to reproduce the problem below (if possible attach a sketch or copy the sketch code in too):

Connect to a broker on a local IP (ie. "192.168.0.12", port "1883").

as the title saying ,I dont want to receice this disconnected message(I use $SYS topic to listen) because it didn't disconnect actually,I just want the disconnect message when truely disconnecting happened.

Hello! I found that during the publish routine (bool Adafruit_MQTT::publish(...)), using QoS = 1, the packet_id_counter variable gets incremented even if the length check fails. Therefore, if a package is dropped, the counter adds one, but the broker counter doesn't, so it perpetually fails the packnum != packet_id_counter check because of an offset in these two counters.

This would make it so the microcontroller would incorrectly assume that it's failing every publish, making it so you couldn't send multiple messages during one MQTT connection using QoS > 0 if even a single publish message is dropped.

I'm using an ESP32 connected to a SIM800 module, and sending messages to a public HiveMQ topic.

Issue:

Tried to upload data to a MQTT broker (local network; IObroker) but failed to update a value which was writeable in a first try. When using a different MQTT client (Windows based standalone program), I was able to change this value without issue. I also could see my ESP8266 client connecting to the brokar and - according to the feedback in the arduino program - update/broadcast the vaalue on the broker, which was/is false.

I monitored the logfile on the MQTT server side and always found the following log details whenever my ESP client tried to connect and change a value:

2021-12-30 18:19:52.967  - info: mqtt.0 (3366) Client [] connected with secret 1640884792963_6139
2021-12-30 18:20:55.447  - info: mqtt.0 (3366) Client [] reconnected. Old secret 1640884792963_6139. New secret 1640884855445_8033
2021-12-30 18:21:57.413  - info: mqtt.0 (3366) Client [] connection closed: Error: read ECONNRESET
2021-12-30 18:21:57.878  - info: mqtt.0 (3366) Client [] connected with secret 1640884917876_1207

Solution and possible explanation: To me it looks like the Adafruit library tries to keep the connection and stores some values in memory to achive this. However, my ESP8266 calls the function ESP.deepSleep() after submitting a value, is reset and then starts from scratch - and something happens here which I can't explain. Maybe the server also tries to keep the connection which is breaking up in an undesired way when the client goes into deep sleep mode.

The solution however is simple: Add mqtt.disconnect(); before calling deep sleep to close the connection in the correct way, which will not only result in the error in the log going away but also the value finally be published on the MQTT server

mqtt.disconnect();
ESP.deepSleep(120 * 1000000); // deep sleep for 120 secs

Maybe this problem and the solution could be added to the documentation as I am sure that a lot of people use MQTT on an ESP8266 to send data points to a server while keeping the ESP in deep sleep to conserver energy (and run them on battery).