A few different issues with receiving accurate Ethica data

Hi all!

Following Mohammed’s request to put the queries we have on the forum, as to accurately suss out the technical issues, the questions we have so far are: (our study in question is 752).

  1. When we are able to capture data from the various phones, a lot of the times it appears grossly inaccurate. For example- user #13066, we have had monitoring activity using a simple pedometer app on the same phone, wearing a smartwatch and the results are extremely different. Eg. on 25/6/19 it measured 209 steps on Ethica, but 14,021 on the simple phone- pedometer app, and watch showed 15,218. As we are looking at trends, we don’t mind if there is a small discrepancy (as between the watch and phone understandably), but missing this data is significant. With #12933 for example, it seems to function ok one day, capturing 5,181 steps, but 83 a day following, but following her settings and changes on Kibana and the Participant history, there doesn’t seem to be any significant settings changes as far as we can tell.

  2. What is the time-frame for uploading data? As per the settings of the study, it’s possible to select upload for wi-fi only, but how will this affect the data captured? Eg. if someone doesn’t have access to wi-fi for 6 hours or 12 hours, how much of the previous hours’ data is lost? Eg. the app will only store ‘x’ hours of activity records? However, again #13066 has access to wifi at home and at university, we have tried having the phone plugged in at these sites too, but the data does not seem to be uploading, even when pressing the manual syncing button.

  3. iPhone/iOS data to access the Participant History page does not seem to be working for us either, so we are unable to access the JSON data to troubleshoot settings issues etc. Again #13066 is an iPhone, but we only have data coming up on Kibana when the participant logged into Ethica on a Huawei phone.

  4. Operating in the 2nd plane- again we have set all of the settings as per the recommended settings (permission granted for always, not just when using the app etc), but again this has not assisted. We understand that not giving GPS permission affects the ability for the data to be collected (even if it’s just pedometer activity), but this has not changed our data accuracy. I have been through the settings in the phones according to what you have sent, and what is on the instructions on the website, but again it’s not coming up with any clear reasons as to why the data isn’t being captured and uploaded.

  5. Are there any explanations as to why we would receive accelerometer data but not pedometer data over the same time period?

  6. How would the use of smartwatches affect the data captured? My understanding from reading through the various Data sources and descriptions on the webpage, is that the Bluetooth data source only records the Bluetooth beacons that it has come into contact with, indirectly indicating the population density of the area the participant is in. Is it possible to link the pedometer data obtained from the smartwatches to a data source separately? We are potentially looking at obtaining funding for simple smartwatches for the population, but if it’s still not possible to link the data automatically from the smartwatches to our dashboard, then there isn’t any point in going down that path. We currently have #13066 wearing an apple watch, but from my understanding thus far, the number of steps won’t be recorded from this in any way?

  7. We are having difficulties with survey uploads. Phones that are uploading pedometer, GPS other data etc, are not uploading survey responses, despite performing a manual sync following the survey completion, and other phones that are not syncing any other data are uploading survey responses perfectly fine. For example #12874 is not uploading the survey responses, despite completing the survey and uploading directly after, and users #13066 and #13243 are the only phones that the responses are being received, despite inaccurate data or no data captured for the movement data sources. We cannot seem to follow the errors or issues with settings using Kibana as it doesn’t have the data available in the patient history report.

Apologies for the multitude of questions, but your assistance is greatly appreciated.

Regards,

Cristian and Georgia

Hi @cmolarte

Welcome to the forum. I would be more than happy to help with your questions.

Regarding your first point, I checked the data from User #13066 and for Jun. 25th 2019 Ethica had captured 13,679 steps. You can access this data through your Kibana account.

We did notice each record is reported twice. For example:

June 25th 2019, 13:04:02.667: 266 Steps
June 25th 2019, 13:04:02.667: 266 Steps

It’s not clear to me why this might have happened. We will investigate the data record duplication. But that can be dealt with at the moment with a simple data clean up process. If you do so, the total step count for that date will be 13,679 which is close to other numbers you reported.

On a related note, I noticed you are collecting all 8 motion sensors in your study. That almost always is unnecessary, and collects ~6000 of unnecessary data records per 5 minute interval per participant. This in turn will hugely impacts participants battery drain, causes the app to get terminated more often due to high resource usage, and will increase your Ethica licensing fee without bringing much value. I suggest check out this video either in full or just the section related to the Physical Activity sensors to learn more why this is unnecessary and how you should deal with it.

Regarding your other questions, please remove them from here and post them in a separate topic and we can discuss it on their specific topic. As we have said here, each post should focus on one topic only:

Don’t ask multiple questions in the same post. That makes it hard to keep track of the conversation. Always create a new post for each different question.

Thanks again

Ok great, I will do that, thanks and apologies.

In terms of the data- we cannot access the number of steps through Kibana on our dashboard for #13066 for example- this is an iOS phone, is there a reason why this data either doesn’t come through or comes through sporadically for us on the Patient History report?

Also, the data that is coming up on our Ethica dashboard, is only 209 for that day, why is there such a discrepancy between what is coming through Kibana (for you) and our Ethica dashboard?

Thanks,

Georgia

Here you can learn more about how to access your data as they are stored in your database directly via Kibana. This video also shows how to do so for Pedometer data.

I also checked again now and realized the double-record issue I mentioned above was in fact my mistake. The data is recorded only once. So that was a false alarm.

Also, I’m not sure what you mean by the:

… on our Ethica dashboard, is only 209 for that day,…

Can you be more specific exactly where do you see this number? Is this the number you get from the Data Quantity Report page?

One last house keeping note. It would be great if you create your own Researcher account in Ethica and use that to discuss the issues. That helps in many ways and you can always share the same data and study details with Christian and other team members.

Bests

Yes, we have set that up on Kibana, but the number of “counts” does not correlate to the number of steps in each of the JSON data lines.
Eg:
July 7th 2019, 19:28:21.779 study_id: 752 user_id: 13,066 device_id: 8B1C47C6E6D241848AFAE61B18FCBC0D record_time: July 7th 2019, 19:28:21.779 rel_record_time: January 21st 1970, 03:15:29.285 accu: -1 duration: -1 steps: 569 distance: 569 avg_active_pace: 400.519 cur_pace: 0.741 cur_cadence: 0 floor_ascended: 0 floor_descended: 0

But then the total number of steps for the whole day is listed as 119, this in on the Kibana page, but then in the Data Quantity Report page, there are 143 steps listed for the same date.

Just to be clear, you know that the Count refers to the number of records, and not the number of steps, right? Each record has a field called Steps which contains the number of steps. You have to sum over Steps for all available records.

The Data Quantity Report page only shows the number of records. It does not show how many steps are represented in each record. So you would not be able to count the number of steps using the data shown there.`

Yes, that’s understood. But even when summing it up manually through Kibana, it does not equal what is coming through to our Ethica Data report- where as I understand from the information you have provided 1 record is supposed to equal one step in regards to pedometer data?

Also for the same phone- we are missing data from all of the 6th of July, despite the application being opened on the device and active. Are there any other reasons as to why the whole day isn’t coming up on our Kibana dashboard, but there are records measured for the Ethica Data reports?

where as I understand from the information you have provided 1 record is supposed to equal one step in regards to pedometer data?

No. Sometimes a given record shows 500 or more steps. Please check the documentation of the Pedometer data source for more details.

For Jul. 6th, the app was terminated. That’s why no data collected during that period. Please refer to my first comment on why the app gets terminated in the background and what to do about it.