Data Collection Integration
Bemba receives field data from two collection platforms: KoboToolbox and Survey Solutions. An Excel file can also be used as a source.
Overview
| Tool | Description | Uses in Bemba |
|---|---|---|
| KoboToolbox | Open source mobile collection platform | Household census, complaints, code of conduct, meetings |
| Survey Solutions | World Bank solution | Household surveys, beneficiary updates, complaints |
| Excel / CSV | File prepared from the template | Loading existing data |
Two KoboToolbox servers can be used: Kobo's online service and the project's self-hosted KoboToolbox server. The choice is made through the API URL, project by project. Installing the self-hosted server is described in Self-hosted KoboToolbox.
Where the screens are
There is no KoboToolbox menu and no Survey Solutions menu. Each use has its own screen:
| Need | Screen |
|---|---|
| Import households (Kobo, Survey Solutions or Excel) | Data Collection → Data Collection, Households tab, Import button |
| Give a project its collection account | Administration → Projects, Collection synchronisation ⟳ button on the project row |
| Global KoboToolbox account | Administration → Settings, Settings tab, API Integrations section |
| Global Survey Solutions account and questionnaires | Same section, Open the Survey Solutions configuration button |
| Complaints collected with Kobo | Complaints (GRM) → Kobo - Complaints |
| Complaints collected with Survey Solutions | Complaints (GRM) → Survey Solutions - Complaints |
| Public code of conduct form | Social Services → Code of conduct, Sync from Kobo button |
Account per project
Each project collects with its own tool and its own account. Configuration is done project by project.
Menu: Administration → Projects → Collection synchronisation ⟳ button on the project row
| Tool | Fields |
|---|---|
| KoboToolbox | API URL, API token |
| Survey Solutions | Server URL, API user, API password, workspace |
Each block has a Test the saved connection button. It queries the tool with the account as it is saved.
Fallback to the global account
A project without its own account uses the MIS global account. For Kobo, this is the token in the Integrations section of the settings. For Survey Solutions, it is the configuration opened from that same section. The synchronisation screen shows this with the global account in use label.
Secrets
The Kobo token and the Survey Solutions password are never displayed again. The screen only says that they are set. Leaving the field empty when saving keeps the existing value.
Effect on import
In the import wizard, the project is chosen in the first step, together with the collection institution. The questionnaires offered are those of that project's account.
Scheduled synchronisation
The server periodically synchronises the Survey Solutions questionnaires of each active configuration: each project's configuration and the global one. An administrator can also run a synchronisation limited to one project:
php bin/console app:survey-solutions:sync --projet=PROJECT_CODE
php bin/console app:kobo:sync <form_identifier> --projet=PROJECT_CODE
The Kobo command reads the form with the project's account. A project without its own account is refused, instead of being silently synchronised with the global account.
General workflow
- Field collection: surveyors fill in the forms on a tablet or phone.
- Upload to the server: submissions reach KoboToolbox or Survey Solutions.
- Review on the platform: supervisors approve or reject the submissions.
- Import into Bemba: imported households arrive in pre-collection.
- Validation: they are checked, validated and then integrated into the social registry. See Pre-collection.
No imported household goes straight into the registry. Validation in pre-collection is what gives it its identifier.
Form types
KoboToolbox forms
| Type | Use |
|---|---|
| PMT - Household collection | Socio-economic data for the PMT score |
| Community validation | Validation by local committees |
| Beneficiary follow-up | Follow-up visits |
| Complaint | Recording complaints in the field |
Survey Solutions questionnaires
| Type | Use |
|---|---|
| Household | Import into the social registry, through pre-collection |
| Update | Update of existing beneficiaries (main, substitute, phone, documents) |
| Complaint | Import into the grievance redress mechanism |
Import principles
Validation status
- Survey Solutions: the Import only approved interviews option of the configuration limits synchronisation to approved interviews.
- KoboToolbox: the import wizard reads every submission of the form, whatever its status. Automatic synchronisation keeps the status set in Kobo: approved, rejected or on hold.
Duplicates
A submission that was already imported updates the existing collection instead of creating a second one. The import result separates collections submitted from collections updated. Matches with registry households are checked during validation.
Good practices
- Test the project connection before the first import.
- Import a small sample first and check the mapping.
- After the import, process the households in pre-collection without delay.
- Report rejected submissions to the field team.
Technical prerequisites
KoboToolbox
- A KoboToolbox account with API access, on the online service or on the project server
- An API token
- Deployed forms
Survey Solutions
- A reachable Survey Solutions server
- A workspace
- An API user and its password