Problem in sending data to mqtt Using PubSubClient

IoT Projects
, ,
1

Hello, I am new, Using the MQTT protocol. My question is:- I have an array of 768, from the thermal sensor and able to display the array on Com Port.
Data from com Port:

Serial.print(mlx90640To[x],0); 31,30,30,30,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,31,30,30,30,30,30,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,32,31,31,31,31,31,30,31,30,30,30,30,30,30,30,31,31,31,31,31,31,31,31,31,31,32,31,32,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,30,30,30,31,31,31,31,31,31,32,31,31,31,31,31,32,31,31,31,32,31,31,31,31,31,31,31,31,31,31,31,30,30,30,31,30,31,31,31,31,31,31,32,31,32,31,32,31,31,31,32,31,31,32,31,31,32,31,31,31,31,31,31,30,31,31,30,31,31,31,31,31,31,31,31,31,31,31,32,31,32,31,32,31,32,31,31,31,32,31,31,31,31,31,31,31,30,30,31,31,31,31,31,31,31,31,31,31,32,32,32,31,31,31,32,31,32,31,31,31,32,31,31,31,31,31,31,30,31,31,31,31,31,31,31,31,31,31,31,32,31,32,31,32,31,32,31,32,31,32,32,32,31,31,31,32,32,31,31,31,31,30,31,31,31,31,31,31,32,31,32,31,32,31,32,31,32,31,32,31,32,32,32,31,32,31,32,31,31,31,31,31,31,31,31,31,31,31,31,31,32,31,31,32,32,32,32,32,32,31,32,31,32,32,32,32,31,31,31,31,31,32,31,31,31,31,31,31,31,31,31,31,32,32,32,31,32,31,32,31,32,31,32,31,32,32,32,31,32,32,32,32,32,31,31,31,31,31,31,31,31,31,31,31,32,32,31,32,31,31,32,32,32,32,32,32,32,31,32,32,32,31,32,31,32,31,31,31,31,31,31,31,31,31,31,31,32,31,32,32,32,32,32,31,32,31,32,32,32,32,32,31,32,31,32,31,32,31,31,31,31,31,31,31,31,32,31,31,31,32,32,32,32,32,31,32,32,32,32,32,32,32,32,32,32,32,31,32,31,31,31,31,31,31,31,31,31,31,31,31,31,32,32,31,32,32,32,32,32,32,31,31,32,32,32,32,32,31,32,32,32,31,31,31,31,31,31,31,31,32,31,31,31,32,31,32,32,32,32,32,32,32,32,32,32,32,32,32,32,31,31,32,32,31,31,31,31,31,31,31,31,31,31,31,32,32,32,31,32,31,32,32,32,32,32,32,32,32,32,31,32,32,31,31,32,32,31,31,31,31,31,31,31,31,31,31,31,32,32,32,32,32,32,31,32,32,31,32,32,31,31,31,32,31,31,31,31,31,31,31,31,30,30,31,31,31,31,31,31,31,31,31,32,31,32,31,32,32,32,31,32,31,31,31,32,32,32,32,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,32,32,32,31,32,32,32,32,32,32,32,31,32,32,31,31,31,32,32,31,31,31,30,31,30,31,31,31,31,31,31,31,31,31,31,32,31,32,32,32,32,32,32,31,32,31,31,32,31,31,31,31,30,31,30,31,31,31,31,31,31,31,31,31,32,31,31,32,31,32,32,32,32,32,31,32,32,32,31,32,31,31,31,31,31,31,30,30,30,31,31,31,31,31,31,31,31,31,31,31,31,32,31,32,31,31,31,31,31,31,31,31,31,31,30,30,30,30,30,30,30,nan,31,31,31,31,31,31,31,31,31,31,32,32,31,31,31,31,31,31,32,31,31,31,31,31,31,31,30,31,
After that, I converted it to JSON Format and again read JSON format on the serial port I received JSON data.
JSON data on com Port:-
Serial.println(serializeJson(doc, Serial));
{"device":"ESP32","sensorType":"T-Scanner","values":[30,30,30,30,30,30,30,31,30,30,31,31,31,31,31,31,31,31,31,31,30,31,30,31,30,30,30,30,30,30,30,30,30,30,30,30,31,30,31,30,31,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,30,30,30,30,30,30,30,30,30,30,31,30,30,31,31,31,31,31,31,31,31,31,31,31,31,31,30,31,31,30,30,31,30,30,30,30,29,29,30,30,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,30,30,30,30,30,30,30,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,30,30,30,30,30,30,30,31,31,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,30,30,30,30,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,31,30,30,30,30,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,30,30,30,31,31,31,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,30,30,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,30,30,31,30,30,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,30,30,30,30,31,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,30,30,30,30,30,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,30,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,30,30,30,31,30,31,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,30,30,30,30,30,31,31,31,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,30,30,30,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,30,30,30,30,30,31,31,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,31,31,30,30,29,30,30,31,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,30,30,30,30,30,30,31,31,31,31,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,31,30,30,30,30,30,30,30,30,31,30,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,30,30,30,30,29,30,30,30,30,30,31,31,31,31,31,31,30,31,31,31,31,31,30,31,31,31,31,31,30,30,31,31,30,30,29,30,30,30,30,255,30,30,30,30,31,30,31,30,31,31,31,31,31,31,31,31,31,31,31,30,31,31,30,30,30,30,30,30]}2359

Now before Publishing data to MQTT, I tried to read Char Buffer it only displays 128 Byte on the Com port.
Serial.println(JSONmessageBuffer);
{"device":"ESP32","sensorType":"T-Scanner","values":[30,30,29,30,30,30,30,30,31,30,30,30,31,30,31,30,31,30,30,31,30,31,30,30,30,30,30,30,30,30,30,30,30,30,30,30,30,30,30,30,30,30,31,31,30,31,31,30,31,31,31,31,30,30,30,30,30,30,30,30,30,30,30,30,30,29,30,30,30,30,30,30,31,31,31,31,31,30,31,31,31,31,31,31,31,31,31,31,30,30,30,31,30,30,30,29,30,30,30,30,30,30,30,31,31,31,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,31,31,30,30,29,30,30,30,30,31,31,30,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,31,31,30,30,30,30,30,30,30,31,30,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,30,30,30,30,30,30,30,30,31,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,30,30,30,30,30,30,31,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31

Data display on Char Buffer same data Publish on MQTT Box.
if(client.publish(“ev8051/esp32/temperature”,JSONmessageBuffer,n) == true)
CODE:-

//#include <ArduinoJson.h>
#include <WiFi.h>
//#include <PubSubClient.h>
#include <Wire.h>
#include "MLX90640_API.h"
#include "MLX90640_I2C_Driver.h"
//#ifndef MQTT_MAX_PACKET_SIZE
////#define MQTT_MAX_PACKET_SIZE 256
//#define MQTT_MAX_PACKET_SIZE 1000
//#endif
#define MQTT_MAX_PACKET_SIZE 1000
const byte MLX90640_address = 0x33; //Default 7-bit unshifted address of the MLX90640
#define TA_SHIFT 8 //Default shift for MLX90640 in open air
float mlx90640To[768];
//char mlx90640To[768];
paramsMLX90640 mlx90640;

// Replace the next variables with your SSID/Password combination
//const char* ssid = "TP-Link_63DD";
//const char* password = "123456";
//
//const char* mqtt_server = "broker.emqx.io";

const char* ssid = "TP-Link_63DD";
const char* password =  "123456";
const char* mqttServer = "broker.emqx.io"; 
//const char* mqttServer =" mqtt.dioty.co";   
const int mqttPort = 1883;
const char* mqttUser = "******";
const char* mqttPassword = "12345";
//const char* mqttServer ="broker.hivemq.com";
//const int mqttPort = 1883;
//const char* mqttUser = "***********";
//const char* mqttPassword = "12345";
const byte led_gpio = 27;
#define row 24
#define clom 32
#define pixel 768
WiFiClient espClient;
PubSubClient client(espClient);
unsigned long lastMsg = 0;
char msg[765];
int value = 0;
#define cap 800
 //int lastMsg = 0;
//int value = 0;
void setup()
{
  Wire.begin();
  Wire.setClock(400000); //Increase I2C clock speed to 400kHz
  Serial.begin(115200); //Fast serial as possible
//  setup_wifi();
//  client.setServer(mqtt_server, 1883);
//  client.setCallback(callback);
  pinMode(led_gpio, OUTPUT);
    WiFi.begin(ssid, password);
 
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.println("Connecting to WiFi..");
  }
 
  Serial.println("Connected to the WiFi network");
 
  client.setServer(mqttServer, mqttPort);
 
  while (!client.connected()) {
    Serial.println("Connecting to MQTT...");

    if (client.connect("ESP32Client", mqttUser, mqttPassword )) {

      Serial.println("connected");
 
    } else {
 
      Serial.print("failed with state ");
      Serial.print(client.state());
      delay(2000);
 
    }
  }
  while (!Serial); //Wait for user to open terminal
  //Serial.println("MLX90640 IR Array Example");

  if (isConnected() == false)
  {
    Serial.println("MLX90640 not detected at default I2C address. Please check wiring. Freezing.");
    while (1);
  }
 Serial.println("MLX90640 online!");
  //Get device parameters - We only have to do this once
  int status;
  uint16_t eeMLX90640[832];
  status = MLX90640_DumpEE(MLX90640_address, eeMLX90640);
  if (status != 0)
    Serial.println("Failed to load system parameters");

  status = MLX90640_ExtractParameters(eeMLX90640, &mlx90640);
  if (status != 0)
    Serial.println("Parameter extraction failed");

  //Once params are extracted, we can release eeMLX90640 array

  //MLX90640_SetRefreshRate(MLX90640_address, 0x02); //Set rate to 2Hz
  MLX90640_SetRefreshRate(MLX90640_address, 0x03); //Set rate to 4Hz
  //MLX90640_SetRefreshRate(MLX90640_address, 0x07); //Set rate to 64Hz
}



void setup_wifi() {
  delay(10);
  // We start by connecting to a WiFi network
  Serial.println();
  Serial.print("Connecting to ");
  Serial.println(ssid);

  WiFi.begin(ssid, password);

  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }

  Serial.println("");
  Serial.println("WiFi connected");
  
  //delay(1000);
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
}

void callback(char* topic, byte* message, unsigned int length) {
  Serial.print("Message arrived on topic: ");
  Serial.print(topic);
  Serial.print(". Message: ");
  String messageTemp;
  
  for (int i = 0; i < length; i++) {
    Serial.print((char)message[i]);
    messageTemp += (char)message[i];
  }
  Serial.println();

  // Feel free to add more if statements to control more GPIOs with MQTT

  // If a message is received on the topic esp32/output, you check if the message is either "on" or "off". 
  // Changes the output state according to the message
}

void reconnect() {
  // Loop until we're reconnected
  while (!client.connected()) {
    Serial.print("\n Attempting MQTT connection...");
    // Attempt to connect
    if (client.connect("ESP8266Client")) {
      Serial.println("connected");
      // Subscribe
      client.subscribe("esp32/output");
     // client.subscribe("esp32/temperature");
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      // Wait 5 seconds before retrying
      delay(5000);
    }
  }
}

//  StaticJsonBuffer<200> JSONbuffer;

void loop()
{
   int boardArray [pixel];
    int boardArrayTwo [row][clom];
  client.loop();
   //char mlx90640To[768];
 // long startTime = millis();
  for (byte x = 0 ; x < 2 ; x++)
  {
    uint16_t mlx90640Frame[834];
    int status = MLX90640_GetFrameData(MLX90640_address, mlx90640Frame);

    float vdd = MLX90640_GetVdd(mlx90640Frame, &mlx90640);
    float Ta = MLX90640_GetTa(mlx90640Frame, &mlx90640);
   
    float tr = Ta - TA_SHIFT; //Reflected temperature based on the sensor ambient temperature
    float emissivity = 0.95;

    MLX90640_CalculateTo(mlx90640Frame, &mlx90640, emissivity, tr, mlx90640To);
   //Serial.print("vdd= "); Serial.print(vdd);Serial.println("\n"); // used to print vdd
//  Serial.print("Ta= "); Serial.print(Ta);Serial.println("\n");   // used to print Ta
//  Serial.print("tr= "); Serial.print(tr);Serial.println("\n");  // used to print tr
//

  Serial.println(" ");
  }

  //long stopTime = millis();
//

// StaticJsonBuffer<5000> JSONbuffer;

/*******************DynamicJsonDocument*****************************/
DynamicJsonDocument doc(13000);   
  //JsonObject& JSONencoder = JSONbuffer.createObject();
 
//  JSONencoder["device"] = "ESP32";
//  JSONencoder["sensorType"] = "T-Scanner";
//  JsonArray& values = JSONencoder.createNestedArray("v");

doc["device"] = "ESP32";
doc["sensorType"] = "T-Scanner";
  char arr[786];
  for (int x = 0 ; x < 768 ; x++)   //768 384
  {
    //if(x % 8 == 0) Serial.println();
  
  Serial.print(mlx90640To[x],0);  //value from sensor print on com port
   // boardArray[x] = mlx90640To[x];
  //  values.add((uint8_t)mlx90640To[x]);
    // array.add((uint8_t)mlx90640To[x]);
    doc["values"][x] = (uint8_t)mlx90640To[x];   // AARAY value of 786 is store in values
    Serial.print(",");
    //delay(1000);
  }

 /* for (int x = 0; x < row; ++x)
{
    for (int k = 0; k < clom; ++k){
        boardArrayTwo[x][k] = boardArray[clom*x + k];
       // tempData=boardArrayTwo[x][k];
              //  values.add(boardArrayTwo[x][k]);
       // printf("%d ", boardArrayTwo[x][k]);
           
     //n=sprintf(reply_buff," number %d ",tempData);
    } }  */
digitalWrite(led_gpio, HIGH);  
//serializeJson(doc, Serial);
//  JSONencoder.printTo(JSONmessageBuffer, sizeof(JSONmessageBuffer));
  Serial.println("\nSending message to MQTT topic..");
 // Serial.println(JSONmessageBuffer);
 char JSONmessageBuffer[cap];                            // JSON MESSAGE CHAR BUFFER
   Serial.println(serializeJson(doc, Serial));
  size_t n = serializeJson(doc, JSONmessageBuffer);     // formation of serialization of data
    size_t sizeq = sizeof(JSONmessageBuffer) / sizeof(JSONmessageBuffer[0]); 
   Serial.println("  JSONmessageBuffer_size");
    Serial.println(sizeq);
//    delay(1000);
  // snprintf(JSONmessageBuffer,768,"%d",serializeJson(doc, Serial));

    //snprintf(JSONmessageBuffer,768,"%ld",serializeJson(doc, Serial));
     Serial.println("\nJSONmessageBuffer:::");
    Serial.println(JSONmessageBuffer);
//Serial.println("\nJSONmessageBuffer_size");
//Serial.println(n);
    //*************************Data send to PUBLISH TO MQTT TOPIC******************************/

//if (client.publish("ev8051/esp32/temperature",JSONmessageBuffer) == true) // sending JSONmessageBuffer data to topic "ev8051/esp32/temperature"
  // boolean beginPublish(const char* topic, unsigned int plength, boolean retained);
    // boolean publish(const char* topic, const uint8_t * payload, unsigned int plength);
    
  if(client.publish("ev8051/esp32/temperature",JSONmessageBuffer,n) == true)
{  
    Serial.println("\nSuccess sending message");
  } else {
    Serial.println("\nError sending message");
  }
  client.endPublish();
if (!client.connected()) {
   reconnect();
  }

 client.loop();
  Serial.println("-------------");
 
  //delay(1000);
  delay(1000);
}

//Returns true if the MLX90640 is detected on the I2C bus
boolean isConnected()
{
  Wire.beginTransmission((uint8_t)MLX90640_address);
  if (Wire.endTransmission() != 0)
    return (false); //Sensor did not ACK
  return (true);
}

Please Help me out

2

The library that you reaching on to send data, Pubsubclient, has a packet limit size of 128 bytes by default. If you want to send or receive messages larger than this (up to 512 bytes), you must change the value of MQTT_MAX_PACKET_SIZE in the PubSubClient.h header.

Said this, if you need to send more than 512 bytes with Pubsubclient, you will need to split your values array into at least two different packets.

Additionally, this is not a valid JSON to ingest data to Ubidots:

{"device":"ESP32","sensorType":"T-Scanner","values":[30, ... , 30]

Please refer to the valid JSON structure at our docs.

All the best

3

First of all, thank you for your instant reply,
I tried your suggestion of increasing MQTT_MAX_PACKET_SIZE to 512 but now I received 249 bytes of the array at MQTT BOX which I pasted below,

{"device":"ESP32","sensorType":"T-Scanner","values": [30,30,29,29,29,30,30,30,30,30,30,30,31,31,30,30,30,31,31,31,31,31,30,30,30,30,30,30,30,29,30,30,30,29,30,30,30,30,30,30,30,30,30,31,30,31,31,30,31,31,31,31,31,31,30,31,30,30,30,30,30,29,30,30,29,30,30,30,30,30,30,30,31,31,31,31,30,31,30,31,31,31,31,31,31,31,31,31,30,31,30,30,30,30,29,29,30,30,30,30,30,30,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,30,30,30,30,30,30,30,30,30,30,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,30,31,30,30,30,30,30,30,31,31,30,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,30,30,30,30,30,30,30,30,30,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,30,30,30,30,30,30,30,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31,31

The controller which I am using for a project is
ESP32 Wrover Module.
Is there some other alternative to sending an array of 768 bytes to the Mqtt Platform?

Please help me out.

4

Due to the limits set by PubSubclient, you will have to split and send your data in chunks of 512 bytes or to check another library to use mqtt in your device.

All the best