To measure the impact of your project, atingi, together with the Sectoral Department (FMB), has developed standardised indicators for e-learning. If you are using standardised indicators or have project-specific goals around course impact that you would like to measure for, you can use our customisable Post Course Impact Survey template. In this case you wouldn't use the default Learning Impact and Experience or the Application of Learning Survey.
If you need help in your M&E set up, please reach out to help@atingi.org or the M&E team of the sectoral department (FMB) and we will be happy to advise you on the best possible option.
There are two different activities that need to be in place on atingi to successfully track the indicators: atingi feedback activity and atingi re-engagement activity.
To measure the impact of your digital capacity building measures, you can add and adapt the Post-course Impact Survey in your atingi course. Please follow the steps below to add valuable survey questions that can be measured and connected to the re-engagement message reminding the learners to take part.
Results of the survey can be accessed directly on atingi or be delivered via a custom metabase report. The Post-course Impact Survey is not added to your template course by default, but you can learn below how to easily set it up.
Adding Feedback
Before you add the Feedback Activity, including the Post-course Impact Survey template, to your course, you should have all other activities and course completion set up and have the reengagement that sends a reminder to participate in the survey configured. Once this is in place, please proceed with the steps below. If you have any doubts, please don't hesitate to reach out to your atingi Customer Success Manager.
Like any other activity, feedback can be added to a course by turning the editing mode on and clicking on the “+” sign in the according section to add a new activity.
Next, you need to give the activity a name and ideally a short description like the following:
"This feedback will be made available a few weeks after you have completed the course. You will receive a notification whenever you can submit your feedback."
Please tick “Display description on course page” so that learners are informed why the feedback is not available immediately after they have completed the course.
Availability and question and submission settings can stay as default, unless you want to receive an email notification for each submitted feedback. Please note that due to data privacy regulations, surveys on atingi should always be carried out anonymously.
If you like, you can add a completion message for the learner to receive after they have submitted the feedback. Common module settings can stay at default.
Most importantly, you need to configure an access restriction for your survey to connect it to the reminder that you set up in the re-engagement activity. If you haven't set up a reengagement activity, please do so first.
There are two access restrictions needed in the feedback activity:
1) Learner must match all of the following: “Course completed – Yes”
2) AND Activity completion: “[Name of re-engagement activity] must be marked complete”
Lastly, in the activity completion settings, choose “Submit feedback"
After you have set up the feedback as well as the re-engagement activity, please make sure that they are NOT selected under course completion.
With those settings, learners will receive a message and be granted access to the feedback activity within the scheduled delay time after completion of the course.
Choosing the correct template
Now that you have configured the settings, you only need to navigate to "Templates" and select the template provided in the feedback activity, called atingi - Post-course impact survey [customisable template].
Click on the title, “Use this template” and “Delete old items” to use and edit the questions within your feedback activity (this doesn’t affect the public template).
Note:
Delete old items - Remove any questions/text types previously added to the Feedback Activity you are viewing before importing items from the selected Template.
Append new items - Add items stored within the selected template to the current Feedback activity, to any existing questions/text types.
You can now click on the three dots on the right and "Edit question" to make adjustments to the questions. Ideally, you stick closely to the provided structure and question types that have been developed by M&E experts along the lines of standardised indicators to measure impact of capacity building measures.
Obviously, you should remove placeholders (here marked in pink) and replace them by your own questions and only use the dimensions relevant for your evaluation.
You can also remove labels like “Module Indicator 1”, which refer to the standardised indicators developed by GIZ FMB. If you would like to learn more about standardised indicators for e-learning, please reach out to your atingi Customer Success Manager or help@atingi.org.
If you would like to add additional questions or question types, please have a look at the "Deep Dive: Feedback Activity" article.
To finalize the application of learning survey, click on “Save as new template” and it will be available under “Templates” and “Course” from now on.
Replicate those steps for all courses that you would like to embed the survey in. If it's needed for several courses, you can also decide to make a back-up of your survey instead and restore it in the other courses. This should ideally be done by your service provider.
Please note that the feedback activity shouldn’t be edited anymore after the course has been launched, because making any changes could lead to data loss. Therefore, please make sure that your questions fit your M&E needs and if in doubt, ask the atingi team for help.
If you have set up reengagement and feedback activity, your learner will receive a notification asking them to give feedback after they have completed the course. The feedback will be visible in the course activity and can also be integrated in your custom report on metabase – please reach out to your atingi Customer Success Manager to set this up accordingly.