Clients: Windracers and University of Bristol, United Kingdom
My roles: Software development, Project management
My roles: Software development, Project management
We demonstrated this digital twin successfully in 3-week flight trials where 3 real and 2 virtual aircraft collaborated with each other (see the video below).
My roles on the project included software development and project management.
Technical overview
SwarmControl is built for researchers as well as future swarm operators, enabling them safe experimentation with swarm algorithms before they are deployed onto real aircraft.
A user is able to:
- Define the virtual world and experiment / UAV run-time parameters. A virtual world could contain airports (from which UAVs could take off and land at), custom polygon boundaries and fires
- Run, observe and de-bug low-fidelity swarm simulations in real or faster-than-real time.
- Run low-fidelity batch simulations without the UI to gather performance data
- Run and observe realistic swarm simulations using high-end UAV and autopilot simulators enhanced by novel decentralized swarm algorithm controllers
- Directly control both simulated and real aircraft that share a common virtual world in real time
SwarmControl and the simulated agents form a system that is distributed across a local computer and multiple cloud instances.
That is, agent control code is separated from SwarmControl and is runnable by itself as standalone software.
Both simulated and real agents consist of two main parts:
- A robust Autopilot solution trademarked by Distributed Avionics. The autopilot controls the UAV and is able to autonomously perform take off, flight between waypoints and landing
- A SwarmAgent controller that gives custom commands to the Autopilot in order to enhance the UAVs with swarming capabilities
Technical contributions
My main technical contributions to the project included the following:- Consulting on the digital-twin-enabled system architecture design, that allows all parts to interact in a robust and scalable way
- Providing technical leadership to the University of Bristol team, including code design, planning and code reviewing
- Creating the basic structure of SwarmControl and SwarmAgent programs. This included applying common object-oriented design patterns and creating a queue-based finite-state-machine behaviour mechanism for SwarmAgent that the researchers could build upon
- Designing and implementing the SwarmControl user interface (UI). The UI design and implementation went through a number of iterations as the project needs evolved. The final version of the UI included a world editor with various custom-built widgets, a dynamically-generated agent behaviour parameter editor that allowed rapid code changes, a text log panel that collected and sorted data from cloud-based agents, and various information panels that showed simulated and real UAV information in real time
- Designing and building 2D world editors and visualizers, including the ability to zoom and pan around, as well as select and edit entities like UAVs, airports and fires. A simple sprite-based visualizer was used with low-fidelity agents. A map-based visualizer leveraged open-source map technology and was used for real-life experiments.
- Implementing data transfer and run-time optimization algorithms
- Creating Save and Load features so that worlds and experiment parameters could be re-used and shared. This also enabled the researchers to repeat the same experiment multiple times in the batch mode.
- Enabling users to send commands and behaviour parameters to the UAVs. During the trials, we demonstrated how a swarm operator could change the search algorithm of the swarm during an operation or order the entire swarm to land.
Project management
My role as the project manager involved overseeing the project deliverables, facilitating collaboration between the partners and reporting quarterly to the project funder, Innovate UK. I started by introducing the team to the concepts of OKRs, Kanban and agile development. This was well-received as a project organization structure.The project consisted of a number of work packages that different partners were responsible for. I helped the partner teams to identify their main OKRs and "Epics" (1-3 month) milestones and managed a Trello board where everyone could see the work that was currently planned. Every 4-8 weeks, I met with Epic owners to identify smaller short-term OKRs that were translated into concrete research and development tasks, as well as to identify and de-risk dependencies on other teams.
In addition, me and all the developers met weekly for a Scrum meeting, where current and upcoming tasks were discussed and where short-term development activities could be aligned across teams. A Trello board was used for these meetings as well, with each team member given the responsibility to manage their own tasks: Begin development as needed, ask for help and ask for reviews of the task results (these were in general code reviews or document reviews).
The combination of Scrum and Epic meetings as project management tools proved to be a great enhancement for the project. Collaborative planning and transparency were especially important during critical months before the flight trials, as researchers and industry developers needed to collaborate closely to iron-out bugs, predict and prevent issues and in general ensure safety of the trials.
Project outcomes
The flight trials were a large success as they demonstrated the possibility of using digital twinning for experimenting with new algorithms in a semi-virtual, digital twin, system. The project delivered results beyond what was originally planned and benefited the partners in multiple ways:
- By the end of the project, SwarmControl was robustly built, well-documented and on an excellent path to be transformed into a commercial product
- A cloud-based system was setup for high-fidelity simulators so that researchers could run large experiments with real autopilots
- The industry partners benefited from a much more robust cloud infrastructure, as well as a much safer Autopilot
- Multiple swarm-based search strategies were explored and work also begun on active firefighting swarm behaviours
Publications
- Georgios Tzoumas, Charles Scales, Stephen Wright, Tom Richardson and Sabine Hauert: "Developing use cases for swarms of high payload UAVs for firefighting using mutual shaping". Accepted for publication in Autonomous Robots.
- Georgios Tzoumas, Lenka Pitonakova, Lucio Salinas, Charles Scales, Tom Richardson and Sabine Hauert: "Wildfire detection in large scale environments using force based control for swarms of UAVs". Swarm Intelligence, 17:89-115.
- Lucio Salinas, Georgios Tzoumas, Lenka Pitonakova and Sabine Hauert: " Digital twin technology for wildfire monitoring using UAV swarms". Proceedings of the 2023 International Conference on Unmanned Aircraft Systems (ICUAS), IEEE. DOI: 10.1109/ICUAS57906.2023.10155819.
- Georgios Tzoumas, Lucio Salinas, Lenka Pitonakova, Alex Horlock, Jonathon Waters and Sabine Hauert: "Developing digital twins of swarms of UAVs for wildfire monitoring". Extended abstract under review in Robotics Science and Systems 2022.
- Lucio Salinas, Georgios Tzoumas, Lenka Pitonakova, Alex Horlock, Jonathon Waters and Sabine Hauert: "Real time digital twins for human monitoring and control of aerial swarms". In preparation.
