Ubidots Community

[SOLVED] Data Extraction limit exceeded

I get the banner that data extraction limit exceeded 51,xxx … However, when I go to My Account and check usage for today (9/11) it shows Dots Out (API) as 19,942. I have no problem if I exceeded the daily limit. However, is the usage table current or does it lag actually use.

Some of this is addressed in thread:

But the question about account usage being low vs. message being displayed was not answered. It was stated that it was possibly a bug in the account usage page. Is this true or is the count wrong.

So the 51.xxx looks suspiciously like a total of 2 days of in+out

Hi @Jmchep,

Thank you for letting us know about this situation. The Dots Out is the sum of the dots extracted through two (2) outbound channels.

  • API : Data requested by external applications or devices from our API
  • Front-end: Data requested by the web interface (Dashboard and devices). This includes all the queries required to render dashboards, CSV reports, variables, etc.

The dots out consumption are only visible in the profile usage by the end of the day, the count reset at UTC 00:00. We already checked the count of both dots out and in and the counting is working as expected.

I hope I have clarified your doubts and concerns about this, if not pleas let me know.

All the best,
-Isabel

Yep understand the concept but the reality is that without immediate feedback from viewing a dashboard, changing to a different device and displaying a variable, etc… it is virtually impossible to get a feel for how many dots are being consumed. I realize I am only using as a hobby for an ispindel but I find this frustrating to not have a way to actually have a live dot counter.

I really am impressed with what I can create in the dashboard. Also, I think the STEM account for schools and hobbyist is cool both in terns of marketing and spreading the concept of IoT. I do work for an environmental firm that does field studies where we employ many data collection devices. I can definately see using this in addition to our own data collection services. My guess is that this is not an issue in paid accounts because Dots can be counted in developed apps. If not, I would find this a severe limitation.

Hi @Jmchep,

Please receive my apologies for the delay in replying to your note. Thank you for your valuable feedback. I will add a feature request to add a live counter in the Dashboards to show how many dots are being consumed by each one. In our paid plans, there is not a daily limitation as we include in all Ubidots paid plans dots in and out in order of millions that generally cover the consumption of most users, otherwise, it will be billed the additional dots consume.

All the best,
-Isabel

Absolute beginner Ubidots STEM user with same problem. I seem to have exceeded the daily quote of data extraction but looking at the usage it was last updated for sept 25th, which I suppose is not correct.

edit: I now understand it is not the calender day, still don’t understand how i got to + 40k data extraction. Is it normal that I cannot see my data anymore ?( Error loading your data
Something went wrong on our end and we were unable to load your data.)

Hi @ArdLab

Thank you for extending us your questions.

The count of dots out and in is made at 00:00:00 UTC. We are working on a way to show this consumption base on the timezone of the accounts, to be more clear. Now, regarding your question of the Error loading your data message, yes, it is normal, the message appears once you have reached out the dots out for the day.

Best,
-Isabel

Data extraction limits are tripping me up. Mostly because I don’t understand Ubidots’ system of using and tracking Dots.
A simple explanation of “dots in” and “dots out” and “data extraction” and how they relate to the web interface would help. It took me a lot of searching to figure out that a web page widget refresh caused data extraction. Am I right about that?

If that is correct, I will try to do fewer screen refreshes.
Please provide more complete documentation of Dot usage so that users can use methods to avoid using up all their Dots.
thanks, john

Hi @jcolema1,

Thank you again for your feedback, it is valuable as it is something that has happened to other users, and we certainly don’t want to be a bump in the road for their IoT initiatives. Let me explain and clarify how the dots out and in consumption works:

Dots in : All the dots received and saved from any channel, Dashboards and API.

Dots Out : The sum of the dots extracted through two (2) outbound channels.

  • API : Data requested by external applications or devices from our API. Meaning each value extracted using any of our supported protocols (MQTT, HTTP, TCP/UDP).
  • Front-end: Data requested by the web interface (Dashboard and devices). This includes all the queries required to render dashboards, CSV reports, variables, etc. Having that in mind, yes refreshing a Dashboard, or exporting data will consume dots out.

Both consumptions are visible in the profile usage by the end of the day. The count reset at UTC 00:00. The following are measures that you can take to avoid reaching out to the daily limit.

  1. Refresh the Dashboards only when necessary.
  2. Set a short time in the Default time range of the Dashboard, so the first time you open the Dashboard, this one only requests the last data.
  3. Reduce the frequency of data extraction trough API.

Finally, I went ahead and increased your dots out capacity to 40,000 per day, to avoid you hit any limits during your IoT application’s development.

I hope my note clarifies your concerns and let me know if you have additional comments.

All the best,
-Isabel

Isabel,
thanks for your response and the dots boost.

I don’t object to the limit Ubidots sets for STEM.
What caused me problems was the difficulty in figuring out why I was exceeding the limit. Clearer error messages and more complete documentation would have saved time.
thanks,
john
p.s. I hope you mean you boosted me to 400,000 dots per day, not 40,000 because I think the standard STEM limit is 50,000/day. I fear that you made a typo because I am again today getting the message “Error loading your data”

Hi @jcolema1,

Please receive my apologies for my mistake, you are correct is 400,000 dots per day, not 40,000. You should be able to see the data now.

Intending to explain this consumption much better, we have created the following article in our help center. Please let me know if you have any additional comments.

All the best,
-Isabel

Isabel,
I read the article and feel that adding the language that you posted in this forum (see below) to that help article would improve it. The key for me was understanding that a refresh of a browser page containing a dashboard widget can use lots of dots. thanks, john

1 Like

Hi @jcolema1,

Thank you for your feedback and suggestion. Be sure we will improve the article by adding more explanations and examples.

Have a nice day!
-Isabel