Actionable Data Science

Emphasizing usability in modeling software

Emphasizing usability in modeling software

Emphasizing usability in modeling software

The historical tendency has been to develop modeling software with limited consideration for the end users, who are frequently not modelers but subject-matter experts. Even if the software provides access to a calibrated model, which runs fast and makes accurate predictions, it may not be used by its intended audience when usability is overlooked. In this project, we focus on establishing the key usability principles for modeling software in specific disciplines (policy making and socio-environmental management). Based on these principles, we evaluate and improve the software. Through this process, we seek to move away from the proliferation of models that die on a shelf and instead ensure that models can be effectively used to generate evidence-based decisions.

Key references:

  • Giabbanelli, P.J., Flarsheim, R.A., Vesuvala, C.X., Drasic, L. (2016) Developing technology to support policymakers in taking a systems science approach to obesity and well-being. Obesity Reviews, 17:194-195.

Key collaborator:

  • Dr Steven Gray, Michigan State University, USA

Model Simplification

Reducing uncertainty in discrete models

Reducing uncertainty in discrete models

Reducing uncertainty in discrete models

Fuzzy Cognitive Mapping (FCM) represents the ‘mental model’ of individuals as a causal network equipped with an inference engine. As individuals may disagree or evidence is insufficient, causal links may be assigned a range rather than one value. When all links have range, the massive search space is a challenge to running simulations. We are designing, implementing, and evaluating new approaches to identify which ranges are important and simplify models accordingly. This involves creating a new Python library for Fuzzy Cognitive Maps and using Design of Experiments (DoE) techniques to efficiently understanding the behavior of a model. Our solutions are currently able to simplify models with a few dozen of links using widely accessible hardware such as personal laptops. Larger models still incur a significant computational load and we are thus using high-performance computing to accelerate computations, using GPUs and CPUs.

Learning at Scale

Data mining and network analysis in MOOCs

Data mining and network analysis in MOOCs

Data mining and network analysis in MOOCs

Massive Open Online Courses (MOOCs) provide a wealth of data, for example by recording interactions between learners. This is a quickly growing area of research, with the promise of better understanding learning experiences. Two approaches have been primarily employed so far: Social Network Analysis (SNA) and Machine Learning (ML). For example, they can respectively be used to assess learner-learner interactions (Tawfik et al., 2017), identify the type of forum posts that learners create, or examine the exact effects of innovations in pedagogy. This interdisciplinary project combines our expertise in computational techniques applied to human behaviors with the research on MOOCs conducted by Dr. Andrew Tawfik. Most recently, we (i) applied ML to understand how the level of learning progresses through a series of learner-learner interactions, and (ii) examined the differences between groups of learners taught using success or failure based cases.

Key reference:

  • Tawfik, A.A., Reeves, T.D., Stich, A.E., Gill, A., Hong, C., McDade, J., Pillutla, V.S., Zhou, X., Giabbanelli, P.J. (2017) The nature and level of learner–learner interaction in a chemistry massive open online course (MOOC). Journal of Computing in Higher Education.

Key collaborator:

  • Dr. Andrew Tawfik, Northern Illinois University, USA

Crowd-sourcing models

Leveraging the crowd to design models

Leveraging the crowd to design models

Leveraging the crowd to design models

There exists a variety of techniques to build a computational model in a participatory manner, including companion modeling (using agent-based models) and group model building (using system dynamics). Among them, Fuzzy Cognitive Maps (FCM) has been emphasized for problems with high uncertainty and vagueness and applied to critical settings (e.g. medical therapies) or settings where stakeholders’ perspectives can differ significantly (e.g. ecosystem management). While the typical process of participatory modeling involves inviting stakeholders and facilitating sessions, this is not always feasible as complex problems may require around the world (i.e. asynchronous model building) and skilled facilitators increase the costs of the model building. Consequently, software solutions have been developed such that FCMs can be constructed online (e.g. mentalmodeler.com). The next frontier to effectively use FCM in participatory modeling is to cope with a massive number of participants: that is, crowdsourcing a model. Latest technical advances are able to take in the causal structural of a model, automatically distribute it to crowds in order to compute relationship strengths, and assemble the results in a model. However, the power of the crowd is still under-utilized as participants are asked either all questions or a random subset, thus over-sampling when sufficient data has been gathered or wider confidence margins are acceptable, and under-sampling what may be critical parts of the model. We are thus developing efficient solutions to bring participatory modeling to the scale of a crowd, both through adaptive algorithms to acquire data, and through innovative solutions to aggregate it.

Key collaborators:

  • Dr Andrew Tawfik, Northern Illinois University, USA
  • Dr Steven Gray, Michigan State University, USA

Bridging Views

Comparing mental models

Comparing mental models

Comparing mental models

Difficult real-world problems are often open-ended. There is no one answer but rather a set of perspectives, reflecting the different beliefs and values of stakeholders. The fundamental question addressed in this project is: how can we measure the gap between perspectives? This assessment problem is useful in many contexts. For example, in an educational context, teachers are often reluctant to provide open-ended problems as they are difficult to grade. Automatically finding differences between a student’s answer and an expert’s perspective could thus offer the key support that educators need. Similarly, in participative modeling, groups of stakeholders are often assumed to be homogeneous based on a few variables (e.g., job or role in the community). Measuring their actual differences could provide a more accurate tool to design working groups with either shared or complementary perspectives, depending on the task. Our approach to measuring the gap between perspectives is rooted in network theory. We’re currently implementing our approach, with a pilot study planned for Fall 2017.

Key collaborator:

  • Dr Andrew Tawfik, Northern Illinois University, USA