Participants didn't receive survey

Support
1

Hi Ethica Support,

I already posted about an issue with several participants not receiving surveys randomly (Participant randomly didn't receive surveys for a day). You then let me know that this issue is due to an underlying issue that has impacted several participants across studies.

However, this issue did not account for all participants who did not receive surveys. I was asked to report this issue separately.

The problem pertains to participants in study 1308. Specifically, we noticed that the following participants also had scheduled surveys that were not received by participants (and for whom, similar to the other issue, Issued Time, Response Time, and Device ID are missing):

#35310
#35304
#35287
#35255
#35204
#35188
#35184
#35165

In particular for participant #35287, many questionnaires were not delivered.

I hope you can help me find what is going wrong so that we can fix it!

Best,
Elisabeth

2

Hi @e.l.demoor

Maybe I misspoke in my previous post. Not every case that the Issue Time and Record Time is missing is an indication of an error. In fact, in almost all cases, it’s not really an error. The problem discussed in the other post could be identified by another issue, regardless of whether Issue Time and Record Time were empty or not.

To be sure, I checked all the participants you listed here, and below I share what I found:

#35310 -> Last completed survey was 2021-01-19 15:12 local time. Further are scheduled.
#35304 -> Last completed survey: 2021-01-19 16:13 local time. Further are scheduled.
#35287 -> Last time the participant used the app was 2021-01-14 17:48. I’m afraid this is a low compliance issue.
#35255 -> Last time the participant used the app was 2021-01-13 15:06. Similar to 35287 this appears to be low compliance issue.
#35204 -> Last completed survey was 2021-01-19 20:12 local time. No further sessions are scheduled.
#35188 -> Last completed survey was 2021-01-19 20:31 local time. No further sessions are scheduled.
#35184 -> Last completed survey was 2021-01-19 20:02 local time. No further sessions are scheduled.
#35165 -> Last completed survey was 2021-01-18 20:37 local time. No further sessions are scheduled.

Hope it helps,
Mohammad

3

Hi Mohammad,

Thank you for looking into this. We figured it must be an issue, because usually missed questionnaires are still issued and thus have an Issued Time and Device ID. It’s good to know that this doesn’t have to be the case.

Best,
Elisabeth

4

Hi @m.hashemian ,

I’m sorry to bother you again, but we are really experiencing an increase in issues with reception of questionnaires by participants. In study 1309, one participant (#35241) hasn’t received any questionnaires in 3 days even after opening the app repeatedly. In another study (1304), one participant (#35432) received questionnaires on the first day after registering but hasn’t received any for about 5 days.

Can you tell me how to fix this issue and avoid it in the future?

Best,
Elisabeth

5

Hi @m.hashemian,

Another participant in study 1304 (User #35432) has not received questionnaires for a few days and before that, received them only sporadically. Additionally, they mentioned that the app wouldn’t open correctly at times and that when they did open it, a questionnaire that was available suddenly disappeared. I hope you can help me solve this issue.

Best,
Elisabeth

6

Hi @e.l.demoor

It seems participation for 35241 is finished, so I checked the records for 354342.

Last time we received a data from her device was “2021-01-20 11:10” local time (you can see this information in Participant History Report) and that was a response to her survey. She has not opened the app since then (as you can see in the Participant History Report, there is no record since then). This also might be because she has been offline, but if she is online, when she opens the app or interacts with the app, or if there is an error in the app, we receive a report. But we have had not report since 20th 11:10.

So that’s why we have no information from her for the four surveys at the following times:

2021-01-20 15:16
2021-01-20 20:00
2021-01-21 07:00
2021-01-21 10:00
2021-01-21 11:36

I suggest asking her to open the app, go to Settings and press Sync Data. When all data is successfully synced, she can go to My Studies and press Reload Studies from Server. So the settings of the phone will be in sync with server and hopefully she will be back on track.

7

Dear Mohammad,

I’m experiencing a similar issue with one participant (study #1413, participant #35765) who did not receive surveys between 8 and 14 feb, and has received them only irregularly during the period after that. Your suggestion did resolve the issue, thank you. However, I would really like to understand the cause of this issue in order to prevent this from happening in the future.

Best,
Tim

8

Hi @t.vantimmeren

We have one problem during January which I think is the root cause. The problem occurred in one of the features we were working on to allow faster enrollment. Particularly for studies that had many criteria and surveys, the enrollment could take a few minutes and fail due to being slow. So we modified our code there to improve the performance.

The improvement we designed was rolled out to the server during the New Year. But as it turned out, it had an issue that if the participant was changing her time zone, or her participation period was modified by the researcher, the schedule of the sessions would be calculated incorrectly.

This was not detected in our tests. We run a few phases of tests before rolling our a feature:

  1. First we write automated test to make sure it works correctly.
  2. We run all previously written automated tests to make sure nothing is broken.
  3. Our engineers take turn and manually test features and look for issues.

This picks almost all errors in a feature before it’s released to the server, but in the case above, the scenario of changing time zone or participation time was not very well covered in our test scenarios, which led to the problem.

We fixed the issue as soon as we noticed it exists. Also we revisited the test flow and made some improvements to the scenarios to prevent it from happening.

Now I’m not sure if all cases discussed above are the result of this problem. I know some of those Elisabeth mentioned are different “user setting” issues. But what I explained above was the main problem we had impacting survey schedules since past few months.

Hope it explains,
Mohammad