After two successful Open Calls within the robotics project ESMERA, the consortium would like to share their Open Call learnings and best practices. Further European-funded projects shall benefit from what the ESMERA consortium have experienced so far, and thus avoid common mistakes and, most of all, take over best practice approaches in the overall Open Call Management. Two Quality Management Surveys, are the base for experience sharing, learning and reflection.

After the two ESMERA Open Calls, the first one closing at the end of 2018 and the second one closing at the end of 2019, the consortium conducted an extensive survey among the participants.

The Quality Management Survey summarizes the applicants´ experiences within the Open Call process focusing on the project´s outreach, the challenges, the Open Call process, the Open Call services and on general feedback. Overall 47 responses, 8 from the first Open Call and 39 from the second Open Call, have led to quantitative and qualitative results. In the first Open Call, 12 experiments were chosen out of 32 proposals. The second Open Call got 69 proposals, and 20 experiments were selected.

Outreach

As a natural first step, it was necessary to make the ESMERA project known among relevant target groups. Whereas more than 60% of the participants got to know ESMERA from word-of-mouth during the first Open Call, this number declined to 30% for the second Open Call, but still being the major touchpoint. This is also due to the fact that the ESMERA consortium had extended their communication channels between the first and second Open Call. More than 20% of the participants had their first contact at conferences. Social Media follows on a third stage.

This feedback shows three facts: First, using your networks can be essential. Talking to people about projects, ideas, plans and sharing information on projects is key to reach target groups. Direct personal contact leads to recommendations, so mingling within your target group and actively focusing on using stakeholders and contacts as further multipliers is a success factor.

Figure 1: First Open Call – How did you learn about the ESMERA Open Call?

 

Figure 2: Second Open Call – How did you learn about the ESMERA Open Call?

The second fact supports the strength of personal and direct contact. Conferences and trade fairs are a relevant marketing tool. Meet your target group where they are, establish direct contact and use attendees for further word-of-mouth advertising.

Third, the importance of Social Media cannot be stressed enough. Nowadays, diverse channels serve as a main information and news platform. Being present on these channels (those, relevant to the respective target audience) is key to promote a project in small steps.

All three main outreach success factors have one thing in common: The direct contact, be it in person or via Social Media. Easy accessible information and a two-way dialogue seem to be key.

The Challenges

More than 80% of the applicants agreed or strongly agreed that the selected challenges were appropriate and necessary within the challenge area. Actually, before starting the Second Open Call, the ESMERA consortium conducted a survey within the robotics industry to clarify which further challenge areas apart from the previous ones are of interest. This resulted in the adaptation and set-up of four more challenge areas. The more attractive the challenges, the more probable the applications. And for the second Open Call, the consortium was able to increase the number of applications of more than 100%. Therefore, a brief survey among potential experiments can help future cascade funding projects, as well.

 

The Open Call Process

90% of the applicants agreed and strongly agreed that three month of proposal preparation is sufficient. However, the extension of the application deadline in both calls was highly appreciated, as well. Therefore, it is always recommended to stay in direct contact with the potential applicants, see, where there is help or support needed. An extended deadline can support the applicants in final preparations and guarantee a qualitatively more adequate proposal.

In the second Open Call, about 50% of the applicants were satisfied with the ESMERA proposal template, whereas in the first Open Call, only one participant did not give at least a rating of 4 or 5. The consortium sees room for improvement here. Remarks and ideas for optimization center around giving more space for the applicants´ experiment descriptions and providing clear distinction between the proposal sections and expected content, to make the template self-explanatory at all times.

 

Figure 3: Second Open Call Satisfaction with the Proposal Template

The experiment feedback shows that the following is desired in particular:

• Online-sessions with the consortium to get questions answered directly and establish a direct contact to the consortium
• Meeting the challenge providers to get first-hand information and specific challenge descriptions
• Easy-access to all Open Call documents and the Open Call platform
• Clear and self-explanatory website structure with concise and on-point FAQs
• A dummy template and sample for the proposal submission
• Precise information about requirements and expectations
• Close contact to the project consortium and fast answers to urgent questions

 

 

Open Call Services

• FAQ Section:

Since in all Open Call processes a number of questions arise for the participants, it is advisable to include a well-prepared FAQ section onto the website. Around 90% of all participants checked the ESMERA FAQ page and around 85% considered the page useful.

Figure 4: first and second Open Call Usage of FAQ Section

The closer the final submission date, the more relevant it is to “be present” as a consortium for the applicants. They need direct contact, a direct email or phone line as well as fast and easy actionable answers. A customer-centric approach is recommended here, seeing the consortium almost as a customer service center. 70% of the participants used the ESMERA general contact email, which can lead to two conclusions: direct, personal contact is preferred throughout the overall process, and it is by all means essential to offer a general email address, easy to find and use for a direct approach.

• Webinars:

Figure 5: first and second Open Call Webinar Participation

Alongside an on-site Brokerage Day in Madrid, the ESMERA consortium also offered two Pan-European webinars to inform about ESMERA and the submission process. Around 30% of the final applicants attended one of these. Concerning the wishes of the applicants to establish a good and regular contact to the consortium and even to the challenge providers, webinars are still a recommendable tool for promoting the project in the first place. It could also be an idea to provide national webinars in local languages, depending on the target audiences and to make sure that all relevant questions are answered clearly and without any language barriers.

• Pre – proposal:

Before submitting the final proposal, the ESMERA consortium offered the pre-proposal service. Applicants could hand in a non-final proposal using a template, receive feedback and suggestions for improvement and afterwards hand in the “real” document. Meant as an accommodating service, the option was used by more than half of the attendees in both Open Calls and was considered really helpful by most of them.

Figure 6: first and second Open Call Pre-Proposal Service

 

Figure 7: second Open Call Pre-Proposal Evaluation

General feedback from the participants was that the usage of a pre-proposal service does make sense in individual cases. In future projects expectation management of the pre-proposal service should probably be made even clearer mainly in respect to a) possible influence of the pre-proposal evaluation outcome on the final decision and b) extend to which feedback can be provided on the proposal content. However, fair treatment of all proposals dictates that the provided feedback does not grant selection advantages to any of the proposals. In all cases, it always makes sense to discuss an individual case by personal contact and discuss the usage and necessity of a pre-proposal.

Overall, the Open Call process and service was evaluated in a satisfying way. Two final remarks regard budget and language challenges. Calculating the budget and explaining the cost analysis turned out to be quite a challenge for the experiments especially in the case of first time participants to EU funded projects and SMEs following their own/national accounting processes. There are not always in line with the EC model and clarifications are thus required. They wished explicit help in these tasks and preferred individual consultation. Future projects could be prepared to either organize for example individual webinars focusing only on budget topics or preparing budget templates with detailed explanations. Furthermore, some experiment teams might not be as fluent in English in order to easily understand all requirements and documents. If time and effort allow, it might make sense to offer some national and local services to applicants with less elevated English skills.