Interactive Agent-Based Modelling

Abstract

Agent-Based Modeling is a powerful type of simulation that is both intuitive and flexible, yet its reliance on emergent behaviours fundamentally obfuscates the underlying processes. A simple solution to this problem, is for researchers to validate their own models using Interactive Agent-Based Modeling systems. This is a complementary approach that provides the user with a visualization of the current simulation state and varying degrees of control over the simulation process, thereby enabling researchers to gain a better understanding of the system as a whole.

This particular project attempts to adapt an existing COVID-19 ABM system into an IABM system by: examining the system’s fundamental limitations, heavily modifying it to allow fine-grain control over agent behaviour, and wrapping the new IABM into a web application. Overall this project was successful, however, it also encountered unexpected technical limitations, rooted in the systems technological underpinnings.

Introduction

Agent-Based Modeling (ABM) is a form of simulation characterized by the actions and interactions of individual agents within a system, often replicating complex emergent behaviours. ABMs tend to be highly flexible and are generally easy to scale due to a highly parallel simulation structure. They have been successfully applied to a wide variety of domains, with much of the more recent work focusing on the context of the COVID-19 pandemic. Despite the strengths of ABMs, their reliance on emergent behaviour fundamentally makes them opaque.

One potential strategy to address this opacity is Interactive Agent-Based Modeling (IABM); this technique complements existing ABM rather than acting as a complete replacement. IABM offers the user a visualization of the simulation state and some control over the current simulation state. The visual component allows the user to appreciate the moment-to-moment interactions between agents. In contrast, the controllable elements enable the user to explore the bidirectional relationship between the environment and the agent. This heightened understanding allows researchers to more easily validate their models and thereby be more confident in the results of their ABM.

This project modified an existing COVID-19 ABM system and converted it into an IABM system with a heavy focus on the granular actions of individual agents. This approach is unique because few IABM systems act upon the agent level; many opt to allow the user to dynamically change system-wide parameters. It is hoped that this modification will enable researchers to gain even more insight into their models than they would otherwise be able to achieve through a coarser level of control. The approach taken by this project, to adapt the ABM into an IABM, was to: study the mechanics of an existing simulator, modify it to support both dynamic time control and agent schedule editing, and embed this modified simulation within an accessible GUI. The structure of the following report explores the state of the ABM/IABM space, the process of creating the IABM, the overall outcomes of this project, and finally: the broader implications of adapting the IABM along with open problems.

Background

As previously noted, ABM is characterized by the interplay of simple actions at the individual agent level. This differs from traditional simulation approaches such as Ordinary Differential Equation models (ODE) which rely primarily on applying pure statistics to model its domains. Wholly statistical approaches are often subject to proportional increases in computation and mathematical complexity as the domain’s complexity increases ( Citation: Bauer, Beauchemin & al., 2009 Bauer, A., Beauchemin, C. & Perelson, A. (2009). Agent-based modeling of host-pathogen systems: The success and challenges. Information Sciences: An International Journal. Retrieved from https://dl.acm.org/doi/10.1016/j.ins.2008.11.012 ) . This contrasts with ABMs which are widely reputed for their flexibility, reliance on relatively simple mathematical rules ( Citation: Bonabeau, 2002 Bonabeau, E. (2002). Agent-based modeling: Methods and techniques for simulating human systems. Proceedings of the National Academy of Sciences. Retrieved from https://www.pnas.org/doi/10.1073/pnas.082080899 ) , and inherent parallelism/scalability ( Citation: Bauer, Beauchemin & al., 2009 Bauer, A., Beauchemin, C. & Perelson, A. (2009). Agent-based modeling of host-pathogen systems: The success and challenges. Information Sciences: An International Journal. Retrieved from https://dl.acm.org/doi/10.1016/j.ins.2008.11.012 ) . ABM systems act at the agent level (micro), where the interactions are relatively simple, but at scale (macro), these interactions model complex desired behaviours. Additionally, ABM techniques, unlike ODE techniques, have the benefit of not presupposing that a population is homogeneous. Instead, each agent individually acts under the simulation rules that define it ( Citation: Bauer, Beauchemin & al., 2009 Bauer, A., Beauchemin, C. & Perelson, A. (2009). Agent-based modeling of host-pathogen systems: The success and challenges. Information Sciences: An International Journal. Retrieved from https://dl.acm.org/doi/10.1016/j.ins.2008.11.012 ) . The differences between ABMs and other paradigms can be so stark that care must be taken to ensure consistency across the results of disparate simulators within the same domain ( Citation: Gustafsson & Sternad, 2010 Gustafsson, L. & Sternad, M. (2010). Consistent micro, macro and state-based population modelling. Mathematical Biosciences. Retrieved from https://www.sciencedirect.com/science/article/abs/pii/S0025556410000283 ) .

Emergent behaviour is the foundation upon which ABM rests, and these emergent underpinnings have been shown to successfully model a broad spectrum of population-centric problem spaces including Economics, Business ( Citation: Bonabeau, 2002 Bonabeau, E. (2002). Agent-based modeling: Methods and techniques for simulating human systems. Proceedings of the National Academy of Sciences. Retrieved from https://www.pnas.org/doi/10.1073/pnas.082080899 ) , Biology, and Epidemiology ( Citation: Bauer, Beauchemin & al., 2009 Bauer, A., Beauchemin, C. & Perelson, A. (2009). Agent-based modeling of host-pathogen systems: The success and challenges. Information Sciences: An International Journal. Retrieved from https://dl.acm.org/doi/10.1016/j.ins.2008.11.012 ) . Indeed much of the recent work in the ABM space has focused on Epidemiology, specifically focussing on the COVID-19 virus ( Citation: Kai, Goldstein & al., 2020 Kai, D., Goldstein, G., Morgunov, A., Nangalia, V. & Rotkirch, A. (2020). Universal Masking is Urgent in the COVID-19 Pandemic: SEIR and Agent Based Models, Empirical Validation, Policy Recommendations. ArXiv. Retrieved from https://arxiv.org/abs/2004.13553 ) ( Citation: Rajabi, Mantzaris & al., 2021 Rajabi, A., Mantzaris, A., Mutlu, E. & Garibay, O. (2021). Investigating Dynamics of COVID-19 Spread and Containment with Agent-Based Modeling. Applied Sciences. Retrieved from https://www.mdpi.com/2076-3417/11/12/5367 ) ( Citation: Romero-Brufau, Chopra & al., 2021 Romero-Brufau, S., Chopra, A., Ryu, A., Gel, E., Raskar, R., Kremers, W., Anderson, K., Subramanian, J., Krishnamurthy, B., Singh, A., Pasupathy, K., Done, Y., O'Horo, J., Wilson, W., Mitchell, O. & Kingsley, T. (2021). Public health impact of delaying second dose of BNT162b2 or mRNA-1273 covid-19 vaccine: simulation agent based modeling study. The BMJ. Retrieved from https://www.bmj.com/content/373/bmj.n1087 ) ( Citation: Shamil, Farheen & al., 2021 Shamil, S., Farheen, F., Ibtehaz, N., Mahmud Khan, I. & Rahman, M. (2021). An Agent-Based Modeling of COVID-19: Validation, Analysis, and Recommendations. Cognitive Computation. Retrieved from https://link.springer.com/article/10.1007/s12559-020-09801-w ) . ABMs reliance on emergent behaviour lends itself well to the ways that populations internally transmit diseases, yet emergent behaviour also inherently introduces obfuscation. As noted by Bauer et al. (2009), being both a constructure of pure code and heavily dependent on emergent behaviours, ABM requires a degree of experimental validation to ensure that generated models correctly reflect reality ( Citation: Bauer, Beauchemin & al., 2009 Bauer, A., Beauchemin, C. & Perelson, A. (2009). Agent-based modeling of host-pathogen systems: The success and challenges. Information Sciences: An International Journal. Retrieved from https://dl.acm.org/doi/10.1016/j.ins.2008.11.012 ) . This experimentation should be on two fronts: reality must be correctly understood, and the simulation must be shown to reflect that reality.

Into this second experimental gap steps IABM. By introducing an active visualization for the current simulation state and allowing for some level of dynamic control, the user can better understand the individual agent’s relationships to each other and their environment and the effect of the controlled parameters on the environment and the broader emergent behaviour. These additions should allow researchers to about the simulation and test how it maps to their perceptions of reality. While IABM assists with dispersing some of the obfuscation of ABM, IABM must sacrifice scalability; multiple simulations can no longer be parallelized or run at once. Accordingly, IABM serves a supplemental or complementary role in relation to ABM. This can be seen in the work of Kai et al. 2020, where the IABM was designed to assist policymakers with more concretely understanding the effects of various masking policies ( Citation: Kai, Goldstein & al., 2020 Kai, D., Goldstein, G., Morgunov, A., Nangalia, V. & Rotkirch, A. (2020). Universal Masking is Urgent in the COVID-19 Pandemic: SEIR and Agent Based Models, Empirical Validation, Policy Recommendations. ArXiv. Retrieved from https://arxiv.org/abs/2004.13553 ) .

In a similar vein, this project sought to adapt an existing COVID-19 transmission ABM system into an IABM system. This was done to better understand transmission and more easily validate models. In the context of this project, MILA’s COVI-AgentSim was selected as the base for an IABM system ( Citation: Gupta, Maharaj & al., 2020 Gupta, P., Maharaj, T., Weiss, M., Rahaman, N., Alsdurf, H., Sharma, A., Minoyan, N., Harnois Leblanc, S., Schmidt, V., St, Charles, P., Deleu, T., Williams, A., Patel, A., Qu, M., Bilaniuk, O., Caron, G., Carrier, P., Ortiz-Gagne, S., Rousseau, M., Buckeridge, D., Ghosn, J., Zhang, Y., Scholkopf, B., Tang, J., Rish, I., Pal, C., Merckx, J., Muller, E. & Bengio, Y. (2020). COVI-AgentSim: an Agent-based Model for Evaluating Methods of Digital Contact Tracing. ArXiv. Retrieved from https://www.researchgate.net/publication/345152377_COVI-AgentSim_an_Agent-based_Model_for_Evaluating_Methods_of_Digital_Contact_Tracing ) , referred to here as COVI-InterSim, or InterSim. The choice to select COVI-AgentSim was based on its flexibility and modifiability. A technical breakdown of the approach to this modification will be expanded upon in the Approach section. The remainder of this section will instead focus on the technical details of COVI-AgentSim’s internal mechanics as they directly map to InterSim’s mechanics.

Sample output from COVI-AgentSim simulation

The COVI-AgentSim simulates agents going about their daily schedules at discrete locations across the simulation space. The schedule of a given agent is comprised of any combination of the following activities:

• idle - The default state of an agent. Under this state, the agent will occupy its assigned household and wait for the next scheduled event.
• sleep - A transitional state used to mark the start and end of every simulation day. An agent in the sleep state will perform identically to an agent in the idle state.
• work - In this state, the agent will occupy its assigned workplace. If a work from home order is in effect, the agent will instead occupy its household during this period. Agents designated as children will instead spend this activity at their assigned school. Agents designated as senior citizens will spend this time occupying a communal dayroom sublocation of their assigned seniors' residence.
• exercise - In this state, the agents will occupy one of the available parks.
• grocery - In this state, the agents will occupy an available retail location.
• social- In this state, some subset of agents will coordinate to occupy a pre-determined location.

Each of the above locations is set to occur at one of the following types of discrete locations:

• household - A location assigned to a subset of agents. These agents will default to this location whenever they encounter idle time, sleep time, or their other events are cancelled. This location is meant to approximate homes or apartments.
• workplace - A location assigned to a subset of agents. These agents will default to this location during their work activities unless their work is set to “cancelled.” This location type represents office, industrial, or other non-retail and non-healthcare environments.
• senior residence - Acts as a large household for senior agents and a workplace for some subset of adult agents. During scheduled work time, senior agents will occupy a shared sublocation here. This location represents long-term care homes, where attending healthcare workers act as the primary vector for contamination.
• park - Acts as a location where agents will go during their exercise activity. Represents communal outdoor spaces.
• store - Acts as a workplace for some agents and a destination for agents during their grocery activity. Represents retail settings.
• school - Acts as the default workspace for all child agents and a workspace for some adult agents. Represents education settings.
• hospital - Acts as a workplace for some agents and as the default location for all sick agent activities; critically ill agents occupy an ICU sublocation.
• misc - Acts as a catch-all workplace for any other location that may warrant simulation.

Under COVI-AgentSim, disease transmission occurs between agents according to a statistical distribution and be based on the length of shared occupation of discrete environments. The schedules that dictate which locations an agent will occupy are mostly precalculated, with slight alterations occurring when cancelled activities or social events are coordinated at the transition between days. The only exception to this precalculation policy is with agents designated as children, whose schedule is dynamically set to follow that of some guardian agent. The simulation is preconfigured to run for a fixed number of simulated days and hours and ceases at the preselected endpoint.

Approach

The stated goal of this project was to implement an active visualization layer for the internal state of COVI-AgentSim and modify its backend to support both dynamic time control and granular control over individual agents. To achieve this goal, a three-phase approach was taken. The first stage was exploratory, the second was to affect the necessary changes within the simulator, and the third was to embed the simulator and deliver the desired functionality in a usable and approachable manner. Before examining any of the three stages, it is critical to grasp the technologies that enable both COVI-AgentSim and InterSim’s existence. This is because any modification is fundamentally constrained and shaped by the existing implementation.

In a modification project, two types of limitations come into play. The first are hard limitations; these are restrictions on what is possible given the framework or tools used to design and implement the baseline work. The second type is soft limitations; these restrictions only exist due to how the software was designed. Hard limits are technical non-starters and may require complete rewrites of the baseline system to overcome. Meanwhile, soft limits may require much work to overcome but are rooted within the system itself, not the tools used to build it; soft limitations can be addressed through modification. There were only a couple of hard limitations encountered when modifying COVI-AgentSim to become an IABM system, there were however numerous soft limitations. An exploration of the hard limitations is presented, followed by a brief analysis of the soft limitations.

COVI-AgentSim was written using Python and the SimPy framework. Configuration is handled through the Hydra framework. SimPy is a flexible framework that models discrete-time events as processes. Here, agents register their current activity with the SimPy environment, then the agent’s process sleeps until the scheduled event end-time is reached. The agent’s next activity is registered, and the aformentioned cycle repeats. The benefit of handling agent behaviour in this way is that the simulation’s computational load is minimized until the agent event transitions. Unfortunately, this behaviour imposes a hard limit on precisely which activities can be modified in an agent’s schedule. Once an agent has registered their activity with the SimPy environment, that activity is effectively immutable. This means that it cannot be interrupted even if an activity has started but not yet been completed. Accordingly, even with modification, only future events can be edited. Similarly, SimPy models events discretely; therefore, the event’s location is also inherently modelled discretely. That is to say that there are hard limitations on how free an agent is to move around its environment while still being simulated. While the agent is not spatially tied to its simulated location, per sae, its activity logically binds it to that location until the activity is over, meaning that even if one moved an agent in the simulation space, functionally, this would have no effect.

SimPy’s discrete approach lends itself well to queues, where the next activity is popped off the queue, registered, and then disposed of. Indeed, this is the exact approach adopted by COVI-AgentSim: at the agent level, each day is stored as a queue of activities in a queue of days. While this is desirable for an ABM system where the agent’s history does not matter, and modification of future events is limited, this design poses fundamental challenges to IABM systems. To work with a queue of queues is to work with an evershifting target; the exact insertion point or modification point for a given event changes based on both the simulation’s state and the contents of the individual queue; this poses a complex problem for editing or modifying a respective agents behaviour. Additionally, there are many cases where an IABM system may find it useful, let alone desirable, to query the agent’s past activities. Queues in COVI-AgentSim pose a serious soft limitation on potential modifications. Yet because this limitation is soft, it is not unfeasible to implement event editing.

Another soft limitation imposed by COVI-AgentSim’s existing implementation is that the simulation proceeds from beginning to end with no breaks. This is undesirable for an IABM system, where the user needs to be able to visualize the ongoing process. Thankfully, SimPy supports manually stepping through a simulation natively, meaning that this is a very soft limitation.

While COVI-AgentSim and SimPy are generally well suited to the required modifications of this project, the Hydra framework is not. Hydra is used to populate the configuration parameters and adopts the problematic policy of occupying the highest place on the call stack. This is due to the developer’s assertion that Hydra produced a high number of side effects, and to limit these it must only be run once. Hydra builds its configuration files/data structures before invoking the main function of its paired program. This poses several compatibility issues when trying to use other top-level frameworks such as Flask or Streamlit. To get around this problem, the script that initializes the IABM must first invoke Hydra to generate the configuration structures, then cache the results, and then invoke the host and load the cached configuration into memory. This is a sub-optimal workaround, but reengineering the entire configuration system could affect compatibility between InterSim and COVI-AgentSim.

With the above problems informing the design choices of the larger project, the first phase of development was exploratory. In this step, COVI-AgentSim’s supporting documentation was consulted, its source code was studied, and its memory space was examined using strategically placed breakpoints under the PyCharm IDE’s Debugger tool. These steps were taken to attain a more thorough understanding of the studied system. This process occupied roughly a month of development time. Such a period is not unreasonable given that ABMs can be just as complex as the underlying phenomenon they model ( Citation: Bauer, Beauchemin & al., 2009 Bauer, A., Beauchemin, C. & Perelson, A. (2009). Agent-based modeling of host-pathogen systems: The success and challenges. Information Sciences: An International Journal. Retrieved from https://dl.acm.org/doi/10.1016/j.ins.2008.11.012 ) . By engaging in the exploratory process, the next phase of development was able to proceed with a clearer understanding of precisely what modifications would be required to meet the target feature-set and deadlines.

The second phase of development focused heavily on modifying the backend. The fundamental changes are briefly mentioned here and will be explored more thoroughly below.

1. Implement a secondary execution path that allows for a controlled flow of simulation time.
2. Rework the agent scheduling subsystem to support:
   1. The querying of past events.
   2. Intuitive insertion of new events.
   3. Intuitive deletion of current events.
   4. Intuitive modification of existing events.
3. Expose the above modifications through some form of API.

The secondary execution path was simple to implement and took the least development time. The baseline COVI-AgentSim follows a rigid Initialize->Simulate->Finish flow, where each step directly invokes the next. To overcome this, IABM wraps the environment and initialization functions into a new ControlledSimulation object and decouples the invocation of the simulation step. Given that SimPy natively supports stepping forward by N steps, the ControlledSimulation object exposes this stepping operation through a function. This function takes a number N and will step the simulation forward by N steps; if the sum of N and the current step would be greater than the predetermined end time, the stepping function bounds the step to the end of the simulation. By mirroring and wrapping existing functionality, as little of the baseline behaviour was touched as possible.

Modifying the agent scheduling subsystem was significantly more involved. As previously noted, the existing queue setup was ideal for ABM systems but suboptimal for IABM. To address this problem, a new data structure called a RangeDict was devised and implemented such that queues were still used for the base logic of the simulator, but the contents of those queues were based on and backed by RangeDicts. Updates to the schedule are first made to the RangeDict and are then propagated out to the correct queue, which is regenerated to reflect the RangeDict’s contents. In this way, all edits could be centralized to a single static reference frame.

The RangeDict comprises a HashMap and corresponding pair of sorted lookup tables. The keys to the RangeDict are the start and end times of a given activity; the keys are also tracked in the sorted lookup table. If the user knows the start and end time, the system has O(1) access time. In cases where the user does not know both the start and end time, the user need only supply a time that falls between the start and end times: this triggers a binary search of the lookup tables, which returns the correct key and therefore the correct value. Accordingly, the case where a key is unknown still offers an O(log(n)) access time. The RangeDict also supports retrieving a range of activities; the user only needs to supply the start and end time of the range, and all events inclusively in that range are returned. The only major constraint is that events must have lengths in the natural numbers' range and cannot overlap. These two constraints are enforced through insertion logic validation. As an aside, the implementation of the RangeDict uncovered a bug in COVI-AgentSim where events of length 0 were slipping through constraint detection; this bug was raised with MILA, and fixed locally.

The use of the RangeDict vastly simplified event editing. While logic to operate on a moving target was implemented, the logic only had to determine the correct subset of queues; affected queues were regenerated rather than edited. Activities in the RangeDict could be overwritten or deleted, and new activities could be inserted into unused space. Activity editing is handled through supplementary logic, which modifies an activity’s timing or location and inserts any necessary idle time into gaps.

The final modification exposed all of the above functionality using an API. This API is primarily centred in the scheduling_interface class but has some token functionality in the ControlledSimulation Object. This token functionality was simply an auto_step() function, allowing developers to step the simulation forward by N steps. The bulk of the API focuses on modifying agent schedules and enforcing constraints on exactly how the simulation can be modified. A brief summary of these constraints follows:

• No editing of the past - events that have already happened cannot be edited. This is because to do so would, at best, have no impact on the current simulation state and, at worst, directly upset the flow of SimPy’s process handling.
• No editing the present - events that are currently in progress have already been registered with SimPy, and thus to modify them would be to interfere with the underlying framework. This is a hard limitation.
• No editing sleep activities - while technically just a special case of idle activities, sleep holds symbolic value and is used to demark the transition between simulation days. Accordingly, some logic in the backend uses the occurrence of sleep to partition day events queues. This is a soft limit introduced by this project’s approach.
• No editing child agent schedules - child agents have a special scheduling algorithm where they follow the schedule of their guardian agent. Accordingly, they have no pre-set future schedule, and if something were inserted into their future, it would be overwritten; supporting editing would be a waste of time.

Every modification to a schedule must satisfy the above constraints; attempting to proceed with a modification that violates these constraints triggers an exception. The following functions facilitate the desired level of control.

• update_location() - changes the currently set location of an activity to a new location of the user’s choice
• cancel_activity() - uses the built-in activity cancellation function, to relocate the activity to the agent’s household.
• adjust_start_time() - move the start time of an agent’s activity to some other time.
• adjust_end_time() - move the end time of an agent’s activity to some other time.
• adjust_time() - move both the start and end times, useful if the user wishes to shift the entirety of the activity to some completely new timeslot.
• delete_activity() - cancel an activity, then backfill it in the schedule with idle time.
• insert_activity() - Inserts an activity into existing idle time.

The final phase of this project’s approach was to encapsulate the simulator into a web application and produce a viable front-end for the user to interact with. Initially, the goal here was to use Flask for the backend and either React, Angular, or Vue.js for the frontend, with JSON being used as an intermediary to communicate between the two halves. Due to time constraints, this approach was dropped. Instead, the Streamlit API for Python was adopted due to the speed at which one can develop and deploy web interfaces. The resulting interface included a visualization of the current simulation state, a terminal readout for viewing the output of the COVI-AgentSim application, and a set of editing/control widgets to interact with the current simulation state.

Rendering of the simulation state is handled using the Plotly API, where agents and locations are depicted as coloured points on a grid. Plotly was selected due to its ease of use and friendly UX. A minor issue encountered here is that COVI-AgentSim treats locations and agents as occupying precisely one point in space; this made visualizing multiple agents in a location problematic but was quickly addressed by applying a slight random offset [-1, 1] to each agent X and Y position before plotting them. The terminal readout was a custom component that piped the stdout of the simulator into the web application and rendered it for the user to see. The editing/control widgets include a readout of the current simulation time, a widget to select the next simulation time step, a widget to select a valid adult agent, a widget to select a non-idle and non-sleep activity for the selected agent, a widget to change the selected activity’s location to any other possible location, a widget to delete the selected activity, a widget to adjust the start and end times of selected activity, and a widget to select idle time and insert a new activity.

Widgets relating to time control are handled via sliders and apply buttons. The timestep control widget has a linear slider covering the values of 1 second, 10 seconds, 1 minute, 10 minutes, 1 hour, 4 hours, 12 hours, and 24 hours. When the user wants to advance time, they need to press the step forward button, and the simulation will step forward by the given number of time steps. Similarly, the activity start-and-end-time tool uses a double-ended slider. Each extreme on the slider represents the furthest the respective starts and end times can be pushed without interfering with a non-idle activity. When the user has adjusted the bounds of the selected activity to their liking, they hit the apply button, and the changes will be applied to the event in the backend and propagated back out to the user. Widgets relating to selection are handled via combo-boxes. The User selection widget contains an up-to-date list of all living adult agents in the simulation; the event selection widget includes an up-to-date list of the current agent’s future non-idle activities. In contrast, the insert activities window selection pane contains an up-to-date list of all agents' available idle time. Meanwhile, widgets such as the delete function are simple buttons that the user presses.

Given that this project was about meeting a set of requirements and delivering a finished product rather than conducting research or analyzing, evaluation will, accordingly, address how well the target feature set was met.

Evaluation

The initial proposal for this project was straightforward: the simulation should be visualized in some way(text-based, graphics library, game engine, or other visualization medium), and the simulation should support some meaningful form of interactivity. This latter point bears contention over what interactivity explicitly entails in this context; the original conception of what interactivity would look like was not actually possible, the best possible approach was taken instead.

A cluster of agents congregating at a workplace

This implementation was successfully able to achieve the desired level of visualization. The user is able to pan around the environment and zoom in on the locations of individual agents. This visualization system remains in sync with the internal simulation and should provide insight into the interactions at play at any given moment. While there were hopes that a more complex representation of locations could be achieved (shaded polygons tracing the outline of each building), such functionality was not present in the baseline COVI-AgentSim, nor was it within the scope of what this project sought to achieve. A more complex visualization remains the subject of future work. Additionally, there were also hopes that a localization system would be implemented such that simulated environments could reflect real-life environments: locations being correctly rendered at the correct locations on top of real city maps complete with roads layouts. While other teams had worked on this feature, such functionality was not supplied within the necessary timeframe to implement it here, and thus remains the subject of future work. In spite of the stretch goal features, the visualization component is technically feature complete and therefore meets the project requirements.

Regarding the Interactive section of this project, as much control of the individual agents as possible was delivered. Originally the idea was that the user would be able to physically control the agent with something like the arrow keys; the IABM would resemble something like a game. Unfortunately, this wound up being technically impossible given the hard limitations of the simulator. Instead, the user is restricted to modifying discrete future activities and locations. Other control limitations such as not being able to control children come from the direct implementation of the original baseline simulator.

It is therefore entirely fair to say that the IABM project was successfully able to achieve its stated goals given the practical realities of the underlying technologies. While many of the original aspirations were not possible, the current implementation reaches the very limits of what is possible. Much development time and effort went into ensuring that the functional requirements were met and that as little of the existing source code was modified as possible.

The user interface for InterSim

Another look at InterSim, this time focusing on the lower half of the editing bar

Summary

Despite being technically complete, there are still some problems with the IABM. These problems are primarily in the area of User Experience. The underlying functionality is solid and works well, but the Interface has some serious issues. The choice to use Streamlit was in part informed by the recommendation of a knowledgeable peer, and in part by the low development time Streamlit offers. While Stream can work fairly well for less dynamic applications, it did not serve the IABM well.

One of the major issues with Streamlit is that a designer is fundamentally forced down one of two paths. Either the designer embeds their interactive components directly into the page, or they embed the components into a form. Under the first paradigm, any adjustment of control components causes the entire webpage to refresh; this is suboptimal for dynamically rendered scenes like the Plotly visualization and suboptimal from a broader UX standpoint; lists and information can be dynamically updated at the expense of dynamic renders and usability. Under the latter paradigm, updates are only triggered when a form is submitted. This has the tradeoff of a stable UX but without the possibility for dynamically populated content. The form approach is also problematic in that there is an intermittent bug where submitting a form does not trigger a refresh: this leads to content that should be updating not updating, and thus outdated information being displayed. For an application like the IABM system, there is no winning, even if a mixture of form and non-form control components are employed, one only winds up with the worst of both paradigms. Additionally, while many Streamlit demos appear to show a great deal of flexibility in the framework, these demos are highly tailored and lack real meaningful depth from the perspective of dynamic applications; their functionality cannot be properly applied here.

An open problem for this project is the clear need for a ground-up rebuild of the front-end using a better set of tools. The choice in development to abandon Flask was necessary to get the project over the finish line, however, that choice also hurt the potential quality of the application and produced technical debt which must be worked of in future. Another related problem is the need for a more intuitive Agent Control System. The current design focuses on function over form. The ideal system would resemble Google Calendar and use draggable and resizable blocks to easily edit a given agent’s schedule. Unfortunately, the given time constraints on this project were not conducive for such taxing front-end development to proceed.

Overall, time actually ended up being one of the biggest limiting factors in terms of this project. While the original concept for modifying an existing simulator seemed simple on paper, the time required to come to terms with COVI-AgentSim, let alone redesign core components took up significantly more of this project’s budgeted time than was originally expected.

All that being said, IABM is still a relatively new field, and in the context of virology/epidemiology, there are few if any simulators with the same level of Agent level control presented by this project. This can be seen through Kai et al. 2020’s IABM; the user was only able to change meta-parameters about the simulation and not effect change at the individual agent level. It is hoped that by allowing researchers to explore their models at a much lower level, and thereby more easily validate their own work. This offers serious benefits in terms of improving the accuracy and viability of Agent-Based Models.

References