EEG brain control for ALS and stroke patients


Emotiv Epoc EEG headset

In the SoftICE lab, we have had several bachelor projects over the years that have examined how to use inexpensive commercial off-the-shelf (COTS) electroencephalography (EEG) equipment to enable brain control in virtual environments. Specifically, we have been using the scientific version of the Emotiv Epoc EEG headset, which has 14 sensors that measure raw EEG signals on top of the human scalp. These signals can be filtered (converted) in real-time to suitable control signals via the Emotiv software and passed on to virtual environments in the 3D game engine Unity, thus enabling real-time control of objects and characters in a virtual world only by the use of brain waves.


Screenshot of the Unity demo game Lerpz Escapes

In the first bachelor project we ran as early as 2011, our students were able to demonstrate a proof of concept by developing an interface between readings from the EEG headset and a demo game in Unity called Lerpz Escapes. After some training sessions for finetuning of personal Emotiv control profiles (the Emotiv control software needs to ‘learn’ the EEG signals of each individual user), the students were able to control a 3D third-person character in the computer game only by using their mind.

A YouTube video demonstrating the results is shown below:

This year, we have had two bachelor projects going one step further from this initial work.

The first project was made by a group consisting of students Fredrik Hoel Helgesen, Rolf-Magnus Hjørungdal, and Daniel Nedregård, and was supervised by AAUC staff Robin T. Bye and Anders Sætersmoen, with additional insights provided by staff members Filippo Sanfilippo and Hans Georg Schaathun. The students used Unity to develop a virtual reality environment that can serve as a training platform for controlling a motorised wheelchair only by means of brain waves (EEG). Their work was inspired by patients who suffer from amyotrophic lateral sclerosis (ALS), which is also known as Lou Gehrig’s disease, and therefore gradually become completely paralysed and unable to control conventional electric wheelchairs using their hands or chin. Following a set of training sessions, users develop their brain control skills and are able to control a motorised wheelchair in realistic virtual environments with streets, buildings, pedestrians, trees, and so on.

The group also did some preliminary work using artificial neural networks to map the neural EEG signals to appropriate motor commands as well as examine using the Oculus Rift for virtual reality.

The source code is freely available on GitHub. The usual standards for citing, using and modifying scientific intellectual property apply.

A YouTube video demonstrating the results is show below:

The second project was made by international exchange student Tom Verplaetse (originally at University College Ghent, Belgium) and supervised by AAUC staff Robin T. Bye and Filippo Sanfilippo. Tom examined how one can use EEG control as a new rehabilitation technique for stroke victims who have lost the ability to move a single hand or both of their hands, a condition called partial paraplegia. Partial paraplegia can be healed by months or sometimes years of physical therapy and other therapies, and developing new rehabilitation techniques is an active field of research worldwide. In the work of Tom, the idea was to create a 3D environment in which the rehabilitating patient can move a visual representation of the paraplegic hand, thus achieving the same effect as that of mirror therapy. Mirror therapy relies on the ability to trick the brain into thinking it can move a hand that is not really there but is merely a visual representation.

The software developed in this project provides a 3D representation of that hand and lets the brain control it by using its own brain waves. Clever use of visual stimulation at specific frequencies by means of a flickering light led to steady state visually evoked potentials (SSVEP) that clearly enhanced both alpha and beta EEG activity.

Hopefully, this process of brain pattern recognition and brain activation of the specific regions needed for motor function could lead to a faster and more efficient rehabilitation process, without much need of expensive equipment or human helpers such as physioterapeuts or nurses.

Source code can be obtained upon request.

A YouTube video demonstrating the results is shown below:

For more information, please contact SoftICE member Robin T. Bye.


Please like and share!

A Riemannian field theory of human movements

Building on his PhD work in neuroengineering and modelling and simulation of human movement control systems, SoftICE member Robin T. Bye has together with researchers Peter D. Neilson and Megan D. Neilson from the Neuroengineering Laboratory at the University of New South Wales, Sydney, co-authored a paper about a Riemannian theory of the entire human body moving in a 3D environment. The paper details a highly mathematical intelligent control engineering approach using Riemannian geometry to develop the intuitive groundwork for a Riemannian field theory of human movement encompassing the entire body moving in gravity and in mechanical interaction with the environment. In particular we present a geodesic synergy hypothesis concerning planning of multi-joint coordinated movements to achieve goals with minimal muscular effort.

The full paper is published in the journal Human Movement Science and can be freely accssed from ScienceDirect.


Peter D. Neilson, Megan D. Neilson, and Robin T. Bye. A Riemannian Geometry Model of Human Movement: The Geodesic Synergy Hypothesis. Human Movement Science 44: 42–72, 2015.

Selected illustrations


Drawing of the two-DOF arm moving in the horizontal (x–y)-plane.


The MATLAB/Simulink geodesic trajectory generator (GTG) simulator for the two-DOF arm moving in the horizontal plane.


  • We develop a Riemannian theory of the entire human body moving in a 3D environment.
  • The theory addresses nonlinear inertial interactions within the body and externally.
  • Geometric concepts are explained intuitively to aid access for non-mathematicians.
  • We show how to plan geodesic synergies to achieve task goals with minimum effort.
  • We integrate the theory with previous descriptions of response planning and control.


Mass-inertia loads on muscles change with posture and with changing mechanical interactions between the body and the environment. The nervous system must anticipate changing mass-inertia loads, especially during fast multi-joint coordinated movements. Riemannian geometry provides a mathematical framework for movement planning that takes these inertial interactions into account. To demonstrate this we introduce the controlled (vs. biomechanical) degrees of freedom of the body as the coordinate system for a configuration space with movements represented as trajectories. This space is not Euclidean. It is endowed at each point with a metric equal to the mass-inertia matrix of the body in that configuration. This warps the space to become Riemannian with curvature at each point determined by the differentials of the mass-inertia at that point. This curvature takes nonlinear mass-inertia interactions into account with lengths, velocities, accelerations and directions of movement trajectories all differing from those in Euclidean space. For newcomers to Riemannian geometry we develop the intuitive groundwork for a Riemannian field theory of human movement encompassing the entire body moving in gravity and in mechanical interaction with the environment. In particular we present a geodesic synergy hypothesis concerning planning of multi-joint coordinated movements to achieve goals with minimal muscular effort.

Please like and share!

Parallel Randomness


My latest paper Evaluation of splittable pseudo-random generators has appeared online last week in the Journal of Functional Programming.

What is the big deal of randomness? Randomness is the key to several common applications of computers, including games and secure communication to name but two of the most obvious ones. Games, such as lotteries and poker, are obviously supposed to be random. Imperfect randomness amounts to loading the dice. Secure communications may be less obvious to the uninitated, but the cryptography which is used to keep secrets depends on random keys to be secure. Loaded dice gives the attacker or intruder an edge. And obviously we want our banking and credit card transactions to be secure.

Programming randomness in computers was recognised as an importent challenge even in the early days of computing (around 1950), and it has been studied ever since. One would think the topic be exhausted by now. In fact, I believed enough had been written on computer randomness when I started this work in 2013.

In fact, randomness is well understood for application in sequential computer programs, i.e. a program which only utilises one of the CPU cores in the computer. A typical consumer-end computer these days has four. If you want to use all of them in one application, the software must be written for parallel execution.

True randomness is hard to achieve in a computer. It is possible to a certain degree, but a large number of random values take time to generate. Instead, the common solution is a so-called pseudo-random number generator (PRNG). Many exist. Most of them are flawed, but there are enough PRNG constructions which are considered trustworthy. However, the well-known solutions are all sequential. The random numbers are generated one by one in sequence. Distributing randomness across parallel threads (or multiple cores in the CPU to take a concrete view) is non-trivial.

Parallel randomness was recognised as an important problem already in the first half of the 1980-s. Yet, the literature is sparse. Some constructions have appeared, but few inspires any confidence. In fact, my paper demonstrates serious flaws in almost all of the known constructions. The first solution which inspires any confidence is that of Claessen and Pałka in 2013.

A preprint is available on my web page.

Please like and share!

Seminar: The New Visualisation Lab – The What and the How

SoftICE member Arne Styve will tomorrow Wednesday 17 June give a brief presentation of the background for the new state-of-the-art visualisation lab at Aalesund University College, the technologies used, and the plans and the vision. The lab will be at the core for research and education purposes across all faculties at AAUC, and central the our newly started master programme in Simulation and Visualisation.

We invite for a broad discussion on possible applications after the talk.

We expect the giant canvas to arrive within a week or two, ready to be installed over the Summer. Thus there is little to see in the actual lab at the moment, and the tour and demo of the lab is postponed till the Autumn.

The seminar is open for all and will take place in room Borgundfjorden at 12.30 on Wednesday 17 June 2015, AAUC main building.


Please like and share!

Seminar: Environmental Disasters from Grounding Accidents: A Case Study of Tugboat Positioning along the Norwegian Coast

SoftICE member Robin T. Bye together with PhD student Brice Assimizele will today present an introductory talk by Bye on the DRAMA research project as well as recent research on tug fleet optimisation (TFO). The main talk is by Assimizele and is called Environmental Disasters from Grounding Accidents: A Case Study of Tugboat Positioning along the Norwegian Coast. The talk is based on a recent research results documented in a journal paper manuscript co-authored by Brice Assimizele, Johannes Royset, Robin T. Bye, and Johan Oppen and soon to be submitted to a world-renowned journal.

The seminar is open for all and will take place in room Borgundfjorden at 12.30 today 13 May 2015, AAUC main building.

This work is part of the PhD research of Brice Assimizele and builds on previous DRAMA research.

To read more about the paper, please see the abstract is included below.  Continue reading

Please like and share!

Seminar: A Computer-automated Design Tool for Intelligent Virtual Prototyping of Offshore Cranes

SoftICE member Robin T. Bye will today present recent research on virtual prototyping of offshore cranes. The talk is called A Computer-automated Design Tool for Intelligent Virtual Prototyping of Offshore Cranes and is a based on a recent paper that will be presented at the 29th European Conference on Modelling and Simulation (ECMS 2015) in Varna, Bulgary, late May.

The seminar is open for all and will take place in room Borgundfjorden at 12.30 today 6 May 2015, AAUC main building.

This work is part of the research project Artificial Intelligence for Crane Design (Kunstig intelligens for krandesign (KIK)) funded by RFF/Research Council of Norway.

To read more about the paper, see this blog post. The abstract is included below.  Continue reading

Please like and share!

ONSITE – Design driven field studies for safer demanding marine operations

SoftICE takes part in the ONSITE project, which is headed by the Ocean Industries Concept Lab at the Oslo School of Architecture and Design. We had the kick-off meeting 24 March 2015 with project partners from DNV GL, Ulstein Group, and Pon Power AS.

ONSITE seeks to strengthen the Norwegian maritime industry by securing an efficient feedback loop between field studies carried out in maritime operations and design processes for new ships and equipment onshore. SoftICE’s role is to develop data models and the software architecture to support gathering, management, and retrieval of field data.

Field studies is a diverse data source. We have to handle video and audio streams, field notes, narratives, still images, drawings and sketches, as well as numeric data from sensors and ship instruments. Post-field research may add annotations and cross-references to the data set, and these have to be managed as well. The user will have relate data from multiple field studies to study particular phenomena.

Our work will apply modern technologies including semantic web, ontologies, multimedia metadata, et cetera to support field data retrieval. We will consider both the linking of data associated with the same scene in a given field studies, and the linking of field data with contextual models, such as models of the operation or the ship, so that the designer can easily find information relating to particular tasks, roles, or work stations.

We still have an an opening for a PhD candidate to work on this project. The application deadline is 14 April 2015. See the advert for further details.

Please like and share!

Paper on intelligent virtual prototyping of offshore cranes accepted for ECMS 2015

ECMS_15smallSoftICE members Robin T. Bye and Ottar L. Osen have together with eminent automation student Birger Skogeng Pedersen written a scientific research paper called “A Computer-Automated Design Tool for Intelligent Virtual Prototyping of Offshore Cranes.” The paper has been peer-reviewed and accepted for publication in the Proceedings of the 29th European Conference on Modelling and Simulation (ECMS’15) and will be presented in the Simulators for Virtual Prototyping and Training (SVT) track (chaired by Robin T. Bye and AAUC colleague Vilmar Æsøy) at the conference in Varna, Bulgaria 26-29 May 2015.


Example of an offshore knuckleboom crane.

AAUC researchers have been regular attenders and contributors at ECMS conferences over the years, which have resulted in AAUC chairing several own tracks and even hosting the conference in Ålesund in 2013 (chairs were AAUC staff Webjørn Rekdalsbakken, Robin T. Bye, and Houxiang Zhang), a conference that was honoured to have recent Norwegian Nobel Prize laureate May-Britt Moser as a keynote speaker.

We will make the full paper available when the conference proceedings have been released. In the meantime, we provide the paper abstract below.

Continue reading

Please like and share!

SoftICE presenting pedagogical research at STEM-conference

SoftICE members Hans Georg Schaathun and Robin T. Bye together with first author Welie A. Schaathun will be presenting pedagogical research on “Active Learning Using Microcontrollers (Aktiv læring i Mikrokontrollarar)” at the Norwegian STEM conference in Bergen 18-19 March (MNT-konferansen 2015).

AAUC is also represented with other work, including “Development and Testing of Method to Avoid Quitting in Engineering Education (Utvikling og utprøving av metode for å hindre frafall i ingeniørutdanningene)” and “Student-Active Research in the Course Real-Time Computer Engineering (Studentaktiv forskning i emnet Sanntids datateknikk)“.

Abstract of our paper (in Norwegian only): Continue reading

Please like and share!

DRAMA project completed

The research project Dynamic Resource Allocation with Maritime Application (DRAMA) was funded by Regionalt Forskningsfond (RFF) Midt-Norge and the Research Council of Norway,  grant no. ES504913. A complete final report can be downloaded here.

The project was officially ended during summer 2014, although work has continued since then through a PhD candidate, Brice Assimizele, and the professor scholarship of the project manager, Robin T. Bye.

Please visit the DRAMA website to read more!

The main goal of the project was to develop new and stringent algorithms for fleet optimisation based on methods from areas such as artificial intelligence, cybernetics, stochastic optimisation, and others.


Figure 1: Ship traffic along pink corridor along northern Norwegian coast. NOR VTS is the vessel traffic service centre in Vardø.

In particular, the project focussed on the the tug vessel preparedness in the north of Norway (see Figure 1). Annually more than 1500 high risk ships transit along the Norwegian coast, out of which about 300 carry oil or petroleum-related cargo. A fleet of three tugs as depicted in Figure 3 (two tugs since January 2014) need to be dynamically positioned along the coast in order to reduce the risk of oil tankers or other ships causing oil spill from drift grounding accidents.


Figure 2: The tug fleet of the Norwegian Coastal Administration.

Continue reading

Please like and share!