[SOLVED] Long Timing on Electron

IoT Devices
,
3

Actually … they perhaps are different but they are doing fundamentally the same job - and ubidots should consider changing. Ubidots does much more with the data on its backend than Particle (at least right now) but the basic transport of the data is functionally equivalent.

As I understand Particle, they are sending a single UDP datagram with the data, optionally with a UDP end-to-end acknowledgement. For many of the types of IoT applications - dropping an occasional data packet can be fine. Over a typical 3G network … one way time for a single UDP datagram is about 150-250 msec (depending on the specific 3G network). Access times for 3G can be quite long compared to LTE, Ethernet or WiFi. The acknowledgement adds another 150=250 msec of delay. So sending a single acknowledged datagram will cost ~400-500 msec of delay to the client. Possibly missing other data due to the blocking nature of the method. Sending an unacknowledged datagram, and accepting occasional packet loss (my measurements suggest <~1% UDP packet loss ETE) cuts this time substantially. For high data rate measurement applications (say at 1 Hz or better), this is critical time.

From what I understand, ubidots is encapsulating the data in (optionally) UDP but also in an HTTP POST. HTTP requires a multi-packet acknowledged datagram exchange. Each packet exchange, particularly on a long delay network like 3G or low power wireless networks … leads to delay.

In the case we see here … to get the data reliably from the data collecting client to the data analysis server requires ~4-6x more non-productive time for ubidots than for Particle.

Without necessarily adding much value. Particle is simply sending the data in a simpler way with less overhead. ubidots could do it too … and it would improve the quality of service.

4

Hi @KenBiba thanks a lot for the detailed comparison!

When we first launched the Particle library early this year -thought initially for the Photon- we started with the standard HTTP protocol. After we got the Electron in March we switched to TCP, then we added UDP as an option -all of this looking to reduce data usage. I guess the next step is exploring beyond standard UDP; we’ll keep you posted on this.

5

@KenBiba we erased some logic that was waiting for a response from the server before sending data again; the new version of the library was pushed yesterday, it’s 2.1.0. Hopefully this will be faster than the current version, although probably not as fast as Particle.publish just yet.

6

OK. Will test with some code later today.

Does the .add method invoke a network transaction or just the .sendAll?

8

What do I need to do to upgrade to 2.1.1? Something explicit?

For what it is worth … here is sample diagnostic code showing difference in speed of previous code base -

120619,-85,13,4.0,81,-999.990,-999.990,0.0,0,0,100578.25,20.6,0.31,0.25,-0.29,0.23,0.54,0.27,3.60,1.65,-0.92,63.02,-21.33,205.52,
120876: Sent Particle
123917: 0.000000
123917: Sent Ubidots

The first number is milliseconds - the first line is the data sent to Particle (~150 octets) and the we can see it takes about ~260 sec. Ubidots takes about 3100 msec.

10

Yup. Particle.publish. UDP. NO_ACK. When acks are enabled, time roughly doubles. But on a cell network - the rate of packet loss is very low … ~.1%. And most UDP connections have < 1% packet loss ETE. So no acks make sense. It is good to have an option to turn off acks when reporting data. Occasional lost packets for most apps should not matter much when there is such a time penalty for each transaction.

With WiFi and LTE - this time penalty is likely a bit lower (and with 2G and GPRS MUCH higher) … but remember the Internet backbone in the US domestically adds about 100 msec roundtrip delay to each packet. More packets in the exchange, means more delay.

11

In the Ubidots library, click on the “i” icon:

Then click on the latest version:

Let us know if it improves, looking forward.

12

Only the .sendAll method invokes a network transaction.

13

Wow!

This new version is MUCH worse. Time to send to Ubidots increased from ~3sec to ~6sec. Interestingly debug output - with Particle terminology and TCP. TCP would be VERY bad in these circumstances … many packet exchanges.

35628,-81,19,4.0,44,38.798,-77.517,72.8,1,5,100448.00,26.9,-0.00,0.24,-0.40,-0.02,-0.03,0.67,4.64,2.20,-2.99,-2.93,1.92,257.37,
35876: Sent Particle
Particle/1.1|POST|hQ17Wv3zN242zWebdOCHjakOTVHTag|400018000a51343334363138:S4_Electron_1=>GeoAlt:72.800003,Pressure:100448.000000,Location:38.797935$lat=38.797935$lng=-77.517204,Temperature:26.937500,GPS:1.000000,Storage:1.000000|end
Client connected
Particle/1.1|LV|hQ17Wv3zN242zWebdOCHjakOTVHTag|S4_Electron_1:test|end
End of TCP Response (3100ms).
42078: 0.000000
42078: Sent Ubidots

14

Oh my . Went back to version 2.01. and time is much too long and I have major demo tomorrow. Would rather not disable Ubidots. Which version was in place before we started this? I can tolerate 2-3sec in a demo … but not 6-7 sec.

15

Hi @KenBiba thanks again for reporting back to us. We setup a test to check this, you can see it here. It’s showing a latency of just 500 ms from Ubidots at the time of this writing:

This is the code that generates this test data:

#include "Ubidots.h"

// Instantiate Ubidots client with my token:
Ubidots client("xxxxxxxxxxxxxxx");

int init_time_UDP = 0;
int init_time_PP = 0;
int end_time_UDP = 0;
int end_time_PP = 0;

void setup() {
    Serial.begin(115200);
    
    // Tell Ubidots library to use UDP protocol
    client.setMethod(TYPE_UDP);
    
}

void loop() {
    
    float dummy_value = 33;
    
    // Send dummy value through Ubidots Library

    init_time_UDP = millis();
    client.add("Dummy Values", dummy_value);
    client.sendAll();
    end_time_UDP = millis() - init_time_UDP;

    // Send dummy value through Particle Publish Library

    init_time_PP = millis();
    Particle.publish("{\"value\":33}");
    end_time_PP = millis() - init_time_PP;

    // Report results to Ubidots

    client.add("Ubidots UDP Response", end_time_UDP);
    client.add("ParticlePublish Response", end_time_PP);
    client.sendAll();

    // Report results to Serial Console

    Serial.print("Ubidots UDP Response Time:");
    Serial.println(end_time_UDP);
    Serial.print("Particle Publish Response Time:");
    Serial.println(end_time_PP);
    
    delay(2000);
    
}

For this test we’re using an even more simplified library. This version has less validation for UDP packets (before, we had erased a voluntary time-out that waited for a response, but now we also erased a Particle function Udp.parsePacket() that is suggested in their docs for UDP).

These are .cpp and .h files as library (again, it’s not the Ubidots public library in this moment):


Which I pasted into the .h and .cpp files of a blank Particle sketch (no library selected):

Let us know how this works for you. Our tests are showing a 500 ms response from Ubidots, which is actually a good time for cellular networks.

16

Just added TCP to the tests, it’s quite a huge difference from UDP! up to 20s :confused: We will make UDP by default for the next version.

17

500msec is an ok response time in a 3G with acknowledgements … I suspect you are still doing acknowledgements … I measure ~250msec in Particle, UDP, no acknowledgements. Still 2x your 500msec. In my app with your library 2.1.4 I am seeing over 6sec - with HTTP over TCP as the protocol.

What library version are you using so I can test? Existing published Particle libraries seem to have gotten worse not better.

Which library should I should to test with my app?

18

Interestingly, when I change the delay to 10 seconds, the Particle seems to enter a sleep mode, possibly requiring the networking functions to be restarted after each loop. This translates into adding a 1 second delay for whichever request is first in the sketch, either Particle.publish or Ubidots:

Results with 10s delay with Particle.publish first in the loop:

Results with 3s delay (things go back to normal):

19

Just validated … my app using Ubidots Electron library 2.1.4.

35132,-51,43,3.9,50,-999.990,-999.990,0.0,0,0,100324.75,23.4,-0.15,0.37,-0.37,-0.02,-0.01,0.66,4.64,2.38,-2.93,-1.14,1.85,278.68,
35388: Sent Particle
Particle/1.1|POST|hQ17Wv3zN242zWebdOCHjakOTVHTag|400018000a51343334363138:S4_Electron_1=>GeoAlt:0.000000,Pressure:100324.750000,Location:-999.989990$lat=-999.989990$lng=-999.989990,Temperature:23.375000,GPS:0.000000,Storage:1.000000,Voltage:3.942600,RSSSI:-51.000000|end
Client connected
Particle/1.1|LV|hQ17Wv3zN242zWebdOCHjakOTVHTag|S4_Electron_1:test|end
End of TCP Response (3100ms).
41591: 0.000000
41591: Sent Ubidots

Ubidots sending with HTTP over TCP despite my setup as UDP.

bool initUbidots () {
#ifdef UBI
    ubidots.setMethod(TYPE_UDP);
    ubidots.setDatasourceName(UBINAME);
    diagnostic("Ubidots initialized");
#endif
    return (TRUE);
}

I am still seeing long delays with Ubidots library 2.1.4.

20

It seems you are making a GET request, which goes through TCP.

All of the troubleshooting above has been around publishing data, not getting data (as this debug line suggests you are doing Particle/1.1|LV|hQ17Wv3zN242zWebdOCHjakOTVHTag|S4_Electron_1:test|end)

Can you please post your entire code so we can have the whole picture?

21

These are the two files you need to use as library.

22

Thank you. I made a mistake and read a variable as you suggested … and that dramatically increases the blocked time.

When I removed that … the ubidots time reduced to about 500 msec - still about 2x the unACKed version of the Particle Publish time.

Thanks.

23

Hi @KenBiba good to hear the delayed decreased. Agree on the 2x difference with their native Particle.publish; we’re using Particle’s UDP library so there isn’t much room for optimization, at least for now. What will do is publish a documentation section on how to use a Particle WebHook so that the data travels from the Electron to Particle using their publish method, and then to Ubidots. Thanks for all your inputs!

1 Like