...
Verify the Last Completed Sync date and time is AFTER the Missed Assessment. If it is prior, there will be no data recorded yet for the Scheduled Assessment.
Verify the participant’s sleep time is set before 11:30 PM. Having a sleep time after 11:30 PM interferes with scheduling and notifying assessments.
Additionally, verifying assessment settings as the participant’s schedule may not correlate with the Wake and Sleep time offsets for assessment(s).
Verify the participant’s phone is compatible with Insight.
Here is a link of the list of phones known to be compatible and incompatible with Insight.
https://mhealthotrc.atlassian.net/wiki/spaces/EDCW/pages/2266497039/Insight + Compatible + Phone +Listhttps://mhealthotrc.atlassian.net/wiki/spaces/EDCW/pages/2266497039/Insight+Compatible+Phone+ List
Note that this list is ever changing and will be updated periodically.
Verify if the participant had the app running in the background.
Enable the “Phone up time and battery” and “App Loaded” filters found under “more filters…”
Tap the refresh button to get an updated view of the participant’s activity.
The “Phone up time and battery” log will look like this:
These logs describe how long the Insight app has been running in the background and what the battery level the participant’s phone is at. These logs will typically generate every 15-20 minutes and is a good indication that the phone is running the app in the background.
Note: Some Android phones will close the Insight app while it’s running the background when the battery level gets low, typically <15%, despite the battery permission. It is important the participant’s phone is kept at a healthy charge.The “App Loaded” log will look like this:
These logs will describe when the participant opens the Insight app and what version the app is on. These logs are a good indication of whether or not the participant had the app running in the background the whole time.
The participant’s activity will reveal one of three typical Missed Assessment cases:
Case 1: The participant missed the assessment.
Observations: In the example above, we a Fixed notification, “Wake Offset Tester” was scheduled for the PT at 08:15. We see the app was running in the background (based on the Uptime logs) prior to the Fixed notification and as well as the App Loaded log matching the notification time for the Fixed notification.
Analysis: This indicates that the phone did indeed notify the participant, but perhaps the participant didn’t have their phone with them or they were busy; perhaps the notification volume and/or vibration was too low for the participant to notice; perhaps the participant had their phone on silent.
Resolution: Verify with the participant their notification volume and vibration are turned up.Case 2: The Insight app was not running in the background.
Observations: In the example above, we see the participant missed a Fixed, Random, and Follow Up and none of the scheduled assessments have a notification time. Notice the lack of Uptime logs prior and in between each of the scheduled assessments and the last App Loaded log was at 10:06, a full hour before the “Fixed Tester F” assessment.
Analysis: This indicates that the phone did not have the app running in the background. This may be due to the participant or their phone force closing the Insight app; perhaps the notification or unrestricted battery permissions weren’t set up properly; perhaps the phone was turned off, either due to low power or by the participant themselves which will look like this:
Note: The Phone shut down, Android Shutdown, and Phone power on will sometimes not be recorded in the CMS database, so it’s important to also look at the Uptime logs to see if the phone was perhaps near 0%.
Resolution: Ensure the participant has no app killers, the Insight app has the unrestricted battery permission set correctly, the participant isn’t closing the app by mistake, and that the participant keeps a healthy charge on their phone to ensure they are notified of their assessments.Case 3: The Do Not Disturb setting (and settings similar) was turned on.
Observations: In the example above, we see the participant has some missed assessments with no notification times. Additionally, we can also see Uptime logs prior and in between each of the notifications as well as a lack of App Loaded logs.
Analysis: This activity indicates that the Do Not Disturb setting, or a similar setting like Bedtime or Schedules in the Do Not Disturb sub menu, was turned on; this can also indicate that the notification permission was changed or set up incorrectly; perhaps the participant has another app running in the foreground preventing the Insight app from sending notifications; or perhaps the participant has another ParticipantInitiated assessment open interrupting the scheduled assessment from notifying; perhaps the participant had an ongoing call when the assessment was supposed to notify.
Resolution: Ensure the participant has their Do Not Disturb setting turned off as well as any setting similar to Do Not Disturb (each phone is different and may have unique settings that are similar to Do Not Disturb), ensure the participant’s notification setting on the phone is set up correctly, and ensure the participant isn’t constantly opening ParticipantInitiated assessments when scheduled assessments are supposed to notify. If the participant is using a study phone, advise the participant to close all apps other than the Insight app. If the participant is using their personal phone, there is not much we can do to prevent the participant from having other apps or phone calls interrupting the Insight app.
Special case: Follow-Up assessments
Follow-Up assessments should be scheduled at least fifteen (15) minutes after the original assessment. Scheduling Follow-Up assessment to be less than that may result in the assessment not being prompted on time.
Verify the geographical location of the participant during the assessment. Participants in rural areas may have additional issues such as a lack/weak WiFi or data network connection or access to an electrical outlet for charging.
Verify the participant’s history within the study.
Participant’s have the right to refuse to take assessments at any time during the study; in addition to, they are allowed to stop the study at any time for any reason.
If a participant has a history of turning off the phone, dismissing assessments, or starting and stopping assessments prematurely, they could be considered noncompliant. It is recommended to reach out to the participant to better understand their engagement with the app that the CMS Activity tab cannot disclose.
...