I am using Ubidots on a Particle Electron to send a small number of values (~5) very often - roughly at a 1 Hz rate. I am using the Ubidots Particle library.
I have configured the Ubidots API to use UDP.
I am finding that the process of sending data to Ubidots is VERY long … ~2-3 seconds … and essentially a blocking transmission.
I am also sending data to the Particle cloud … much more data (~150 octets) … at the same rate also using UDP, with acknowledgements turned off. This takes 10% of the time as Ubidots - for much more data.
Would be lovely to get this time overhead (and likely protocol overhead that is costly in $ not just time on paid links like cellular).
Great UI for displaying data … but the overhead for transmitting the data seems very high.
Hello @KenBiba well the method of particle cloud and ubidots cloud are very differente then is impossible to compare this. But we are working on webhook for Ubidots, this will be faster than library method
Thank you for you attention
Actually … they perhaps are different but they are doing fundamentally the same job - and ubidots should consider changing. Ubidots does much more with the data on its backend than Particle (at least right now) but the basic transport of the data is functionally equivalent.
As I understand Particle, they are sending a single UDP datagram with the data, optionally with a UDP end-to-end acknowledgement. For many of the types of IoT applications - dropping an occasional data packet can be fine. Over a typical 3G network … one way time for a single UDP datagram is about 150-250 msec (depending on the specific 3G network). Access times for 3G can be quite long compared to LTE, Ethernet or WiFi. The acknowledgement adds another 150=250 msec of delay. So sending a single acknowledged datagram will cost ~400-500 msec of delay to the client. Possibly missing other data due to the blocking nature of the method. Sending an unacknowledged datagram, and accepting occasional packet loss (my measurements suggest <~1% UDP packet loss ETE) cuts this time substantially. For high data rate measurement applications (say at 1 Hz or better), this is critical time.
From what I understand, ubidots is encapsulating the data in (optionally) UDP but also in an HTTP POST. HTTP requires a multi-packet acknowledged datagram exchange. Each packet exchange, particularly on a long delay network like 3G or low power wireless networks … leads to delay.
In the case we see here … to get the data reliably from the data collecting client to the data analysis server requires ~4-6x more non-productive time for ubidots than for Particle.
Without necessarily adding much value. Particle is simply sending the data in a simpler way with less overhead. ubidots could do it too … and it would improve the quality of service.
Hi @KenBiba thanks a lot for the detailed comparison!
When we first launched the Particle library early this year -thought initially for the Photon- we started with the standard HTTP protocol. After we got the Electron in March we switched to TCP, then we added UDP as an option -all of this looking to reduce data usage. I guess the next step is exploring beyond standard UDP; we’ll keep you posted on this.
@KenBiba we erased some logic that was waiting for a response from the server before sending data again; the new version of the library was pushed yesterday, it’s 2.1.0. Hopefully this will be faster than the current version, although probably not as fast as Particle.publish just yet.
What do I need to do to upgrade to 2.1.1? Something explicit?
For what it is worth … here is sample diagnostic code showing difference in speed of previous code base -
120619,-85,13,4.0,81,-999.990,-999.990,0.0,0,0,100578.25,20.6,0.31,0.25,-0.29,0.23,0.54,0.27,3.60,1.65,-0.92,63.02,-21.33,205.52,
120876: Sent Particle
123917: 0.000000
123917: Sent Ubidots
The first number is milliseconds - the first line is the data sent to Particle (~150 octets) and the we can see it takes about ~260 sec. Ubidots takes about 3100 msec.
Yup. Particle.publish. UDP. NO_ACK. When acks are enabled, time roughly doubles. But on a cell network - the rate of packet loss is very low … ~.1%. And most UDP connections have < 1% packet loss ETE. So no acks make sense. It is good to have an option to turn off acks when reporting data. Occasional lost packets for most apps should not matter much when there is such a time penalty for each transaction.
With WiFi and LTE - this time penalty is likely a bit lower (and with 2G and GPRS MUCH higher) … but remember the Internet backbone in the US domestically adds about 100 msec roundtrip delay to each packet. More packets in the exchange, means more delay.
This new version is MUCH worse. Time to send to Ubidots increased from ~3sec to ~6sec. Interestingly debug output - with Particle terminology and TCP. TCP would be VERY bad in these circumstances … many packet exchanges.
35628,-81,19,4.0,44,38.798,-77.517,72.8,1,5,100448.00,26.9,-0.00,0.24,-0.40,-0.02,-0.03,0.67,4.64,2.20,-2.99,-2.93,1.92,257.37,
35876: Sent Particle
Particle/1.1|POST|hQ17Wv3zN242zWebdOCHjakOTVHTag|400018000a51343334363138:S4_Electron_1=>GeoAlt:72.800003,Pressure:100448.000000,Location:38.797935$lat=38.797935$lng=-77.517204,Temperature:26.937500,GPS:1.000000,Storage:1.000000|end
Client connected
Particle/1.1|LV|hQ17Wv3zN242zWebdOCHjakOTVHTag|S4_Electron_1:test|end
End of TCP Response (3100ms).
42078: 0.000000
42078: Sent Ubidots
Oh my . Went back to version 2.01. and time is much too long and I have major demo tomorrow. Would rather not disable Ubidots. Which version was in place before we started this? I can tolerate 2-3sec in a demo … but not 6-7 sec.
#include "Ubidots.h"
// Instantiate Ubidots client with my token:
Ubidots client("xxxxxxxxxxxxxxx");
int init_time_UDP = 0;
int init_time_PP = 0;
int end_time_UDP = 0;
int end_time_PP = 0;
void setup() {
Serial.begin(115200);
// Tell Ubidots library to use UDP protocol
client.setMethod(TYPE_UDP);
}
void loop() {
float dummy_value = 33;
// Send dummy value through Ubidots Library
init_time_UDP = millis();
client.add("Dummy Values", dummy_value);
client.sendAll();
end_time_UDP = millis() - init_time_UDP;
// Send dummy value through Particle Publish Library
init_time_PP = millis();
Particle.publish("{\"value\":33}");
end_time_PP = millis() - init_time_PP;
// Report results to Ubidots
client.add("Ubidots UDP Response", end_time_UDP);
client.add("ParticlePublish Response", end_time_PP);
client.sendAll();
// Report results to Serial Console
Serial.print("Ubidots UDP Response Time:");
Serial.println(end_time_UDP);
Serial.print("Particle Publish Response Time:");
Serial.println(end_time_PP);
delay(2000);
}
For this test we’re using an even more simplified library. This version has less validation for UDP packets (before, we had erased a voluntary time-out that waited for a response, but now we also erased a Particle function Udp.parsePacket() that is suggested in their docs for UDP).
These are .cpp and .h files as library (again, it’s not the Ubidots public library in this moment):
Which I pasted into the .h and .cpp files of a blank Particle sketch (no library selected):
Let us know how this works for you. Our tests are showing a 500 ms response from Ubidots, which is actually a good time for cellular networks.
500msec is an ok response time in a 3G with acknowledgements … I suspect you are still doing acknowledgements … I measure ~250msec in Particle, UDP, no acknowledgements. Still 2x your 500msec. In my app with your library 2.1.4 I am seeing over 6sec - with HTTP over TCP as the protocol.
What library version are you using so I can test? Existing published Particle libraries seem to have gotten worse not better.
Which library should I should to test with my app?
Interestingly, when I change the delay to 10 seconds, the Particle seems to enter a sleep mode, possibly requiring the networking functions to be restarted after each loop. This translates into adding a 1 second delay for whichever request is first in the sketch, either Particle.publish or Ubidots:
Results with 10s delay with Particle.publish first in the loop:
Just validated … my app using Ubidots Electron library 2.1.4.
35132,-51,43,3.9,50,-999.990,-999.990,0.0,0,0,100324.75,23.4,-0.15,0.37,-0.37,-0.02,-0.01,0.66,4.64,2.38,-2.93,-1.14,1.85,278.68,
35388: Sent Particle
Particle/1.1|POST|hQ17Wv3zN242zWebdOCHjakOTVHTag|400018000a51343334363138:S4_Electron_1=>GeoAlt:0.000000,Pressure:100324.750000,Location:-999.989990$lat=-999.989990$lng=-999.989990,Temperature:23.375000,GPS:0.000000,Storage:1.000000,Voltage:3.942600,RSSSI:-51.000000|end
Client connected
Particle/1.1|LV|hQ17Wv3zN242zWebdOCHjakOTVHTag|S4_Electron_1:test|end
End of TCP Response (3100ms).
41591: 0.000000
41591: Sent Ubidots
Ubidots sending with HTTP over TCP despite my setup as UDP.
It seems you are making a GET request, which goes through TCP.
All of the troubleshooting above has been around publishing data, not getting data (as this debug line suggests you are doing Particle/1.1|LV|hQ17Wv3zN242zWebdOCHjakOTVHTag|S4_Electron_1:test|end)
Can you please post your entire code so we can have the whole picture?
We will make UDP by default for the next version.
