raahnsimulation

Real-Time Autoencoder Augmented Hebbian Network(RAAHN) Simulation (Version 1.0)

Description

RAAHN Simulation is a simulator for testing and developing the Real-Time Autoencoder Augmented Hebbian Network algorithm. The simulation consists of a car driving in a simulated environment. It learns how to navigate the environment with modulated hebbian learning (modulation signals influencing hebbian learning, similar to reinforcement learning), and an autoencoder. RAAHN Simulation uses libraahn for a RAAHN implementation. libraahn is developed in parallel with RAAHN Simulation.

Command Line Interface

--headless Run the simulation in headless mode. An experiment file must be specified.

--experiment [experimentfile.xml] Uses experimentfile.xml for headless mode runs.

--help Displays this help message.

Hot Keys

Globally

Esc - Close the application. Scroll - Zoom in and out.

Simulation state

Left - Rotate the car left. Right - Rotate the car right.

Map state

Space - Place the starting point for the car represented by a flag. Right Click - Add wall or point entities, depending on the selected mode.

Configuration Files

RAAHN Simulation uses XML to define experiment configurations. The configuration files are stored in raahnsimulation/Data. Four folders in this directory are for XML configuration files:

• Experiments - Contains RAAHN experiment files. Two experiment setups are already included for quickly testing the simulator. The XMap prefix indicates that the experiment uses a map shaped like the letter "X." Experiments suffixed with Hebbian use only a single layer Hebbian neural network. Experiments suffixed with Autoencoder use the complete RAAHN system, that is, a single layer autoencoder connected to a single layer Hebbian neural network. The autoencoder is feed the sensory inputs, and computes an encoding which is given to the single layer Hebbian neural network.
• Maps - Contains map layout files. Pretty intuitive to edit, but easier to use the map builder for actually creating maps.
• Networks - Contains RAAHN network architecture files.
• Sensors - Contains agent sensor configuration files.

Experiments

Each experiment file contains the basic format:

<?xml version="1.0" encoding="utf-8"?>
<Experiment>
	<TicksPerRun>10000</TicksPerRun>
	<RunCount>200</RunCount>
	
	<MapFile>XMap.xml</MapFile>
	<SensorFile>Default.xml</SensorFile>
	<NetworkFile>XMapAutoencoder.xml</NetworkFile>
	
	<PerformanceMeasurement>LapsPerformance</PerformanceMeasurement>
</Experiment>

• TicksPerRun - describes the duration of headless-mode trials
• RunCount - describes the number of runs to perform in headless-mode
• PerformanceMeasurement - describes the class used to evaluate agent performance. Currently only "LapsPerformance" is implemented, which counts the number of laps completed around a central point. A performance measurement can be added by extending the PerformanceMeasurement class. To register the new performance measurement in RAAHN Simulation (so it is available via XML), update the Method enum and METHOD_STRINGS array in PerformanceMeasurment to contain an ID and XML string respectively. THE IDs MUST BE CONSECUTIVE. The value of the ID is the index of the XML string. Also the CreateFromMethod function must be updated to allow returning a new instance of the newly defined performance measurement.

Note: Most of the properties explain themseleves. TicksPerRun and RunCount are only useful for headless mode, since the GUI mode lets you run the simulation as long as you want.

Maps

Maps are meant to be created through the map builder, however editing their properties is trivial. A map contains a postion for the robot (angle has no effect at the moment):

<Robot>
	<X>1561.9331742243444</X>
	<Y>505.79080279525812</Y>
	<Angle>0</Angle>
</Robot>

and a list of entity objects:

<Entity Type="Wall">
	<X>1301.3842482100238</X>
	<Y>675.13128038897889</Y>
	<RelX>769.83293556080639</RelX>
	<RelY>1.4438228390645236E-11</RelY>
	<Angle>0</Angle>
</Entity>

An entity can have multiple types. A Wall entity is a, well, wall. It is a line from (x,y) to (x+RelX, y+RelY). The angle property is not needed, but still generated by the XML serializer.

A point entity specifies a point in space, like that used for calculating performance. See the example files for use of a point entity.

Networks

A network file specifies the neural network architecture for the agent. An example of a Hebbian only file is XMapHebbian.xml:

<?xml version="1.0" encoding="utf-8"?>
<NeuralNetwork>
	<ControlScheme>RangeFinderControl</ControlScheme>
	<WeightCap>10</WeightCap>
	<HistoryBufferSize>1</HistoryBufferSize>
	<OutputNoiseMagnitude>0.1</OutputNoiseMagnitude>
	<WeightNoiseMagnitude>0.1</WeightNoiseMagnitude>

	<NeuronGroup Id="0">
		<Count>11</Count>
		<Type>Input</Type>
	</NeuronGroup>

	<NeuronGroup Id="1">
		<Count>1</Count>
		<Type>Output</Type>
	</NeuronGroup>

	<ConnectionGroup UseBias="false">
		<InputGroup>0</InputGroup>
		<OutputGroup>1</OutputGroup>
		<LearningRate>1.0</LearningRate>
		<TrainingMethod>Hebbian</TrainingMethod>
		<ModulationScheme>WallAvoidance</ModulationScheme>
	</ConnectionGroup>

	<Parameter>400</Parameter>
</NeuralNetwork>

The control scheme specifies how the agent will use sensory data with its neural network to control its movement. There are two control schemes at the moment: RangeFinderControl and SensorControl. RangeFinderControl only uses rangefinder input to control the agent, while SensorControl uses both rangefinder and pie slice sensor input (see Sensors).

A WeightCap of n specifies a maximum possible weight magnitude of n. HistoryBufferSize specifies the size of the history buffer. An OutputNoiseMagnitude of n adds noise to the final output of the network in the range [-n, n]. A WeightNoiseMagnitude of n adds noise to each weight update in the range [-n, n].

Next follows neuron groups. A NeuronGroup can have a id number associated with it, used for connecting it to other groups. Count specifies the number of neurons. Type specifies whether it is an Input, Hidden, or Output group.

Next follows connection groups which fully connect neuron groups. InputGroup and OutputGroup are the ids of the groups to use. LearningRate is the learning rate for the connection group. TrainingMethod is specifies how to train the connections (as an autoencoder or a Hebbain neural network). ModualtionScheme specifies how to modulate the Hebbian neural network. The "WallAvoidance" scheme is the only ModulationScheme at the moment. It penalizes the agent for moving into walls and rewards it for moving out of walls.

Next follows generic parameters to be interpreted by a ModulationScheme and ControlScheme. In this case, the parameter refers to how far to detect walls for use for neuromodulation.

Adding new control and modulation schemes is similar to adding a new performance measurement. No class is extended, instead a method is defined within ControlScheme/ModulationScheme for the new scheme. As with PerformanceMeasurement, an ID and XML string must be specified. Edit the Scheme enum to add a new ID. THE IDs MUST BE CONSECUTIVE. Edit SchemeStrings to add the XML string. The ID indexes correspond to the XML string. Also a function pointer to the new scheme must be added to the SchemeFunctions array (also chosen by ID index).

Sensors

Finally sensor files describe agent sensors. Below is the default sensors file:

<?xml version="1.0" encoding="utf-8"?>
<SensorConfig>
	<RangeFinderGroup>
		<Count>11</Count>
		<Length>350</Length>
		<AngleOffset>-90</AngleOffset>
		<AngleBetween>18</AngleBetween>
		<DetectEntity>Wall</DetectEntity>
	</RangeFinderGroup>
</SensorConfig>

Sensors are specified in groups. The two possible groups are RangeFinderGroup and PieSliceSensorGroup. Count specifies the number of sensors for that group. Length specifies the length of a rangefinder. AngleOffset is where to begin the group relative to the agent's orientation. AngleBetween specifies the angle between each rangefinder. DetectEntity specifies what kind of entities is should detect (can be used more than once).

A PieSliceSensorGroup is defined similarly. Below is an example:

<PieSliceSensorGroup>
	<Count>9</Count>
	<MaxDetection>3</MaxDetection>
	<AngleOffset>-90</AngleOffset>
	<AngleBetween>20</AngleBetween>
	<OuterRadius>400</OuterRadius>
	<InnerRadius>0</InnerRadius>
</PieSliceSensorGroup>

Count specifies the number of sensors. MaxDetection specifies the maximum number of entities a sensor will detect. AngleOffset and AngleBetween are the same as for RangeFinderGroup. OuterRadius specifies how far out the outer edges of the sensors should go from the agent. InnerRadius specifies how far out the inner edges of the sensors should go from the agent.

Logs

Log file output is generated by headless mode runs. Each run generates a new file called Scores.txt stored in raahnsimulation/Data/Logs. The file is the performance of the agent for every run separated by newlines.

Building and Running

RAAHN Simulation depends on gtk-sharp 2.12 and OpenTK 1.0. OpenTK is used for OpenGL API access. GL 1.5 core profile is required, make sure your appropriate graphics driver is installed.

If using a GNU/Linux distribution, your package repositories probably have gtk-sharp and OpenTK. In Debian and its derivatives the required packages are gtk-sharp2 and libopentk-cil-dev. The package names are probably similar for most other distributions. A mono development environment is also needed. For Debian the package for mono development tools is called mono-devel.

The easiest way to install the dependencies on Windows is to install the NuGet package available at https://gitlab.com/jbowren/raahnsimulation/wikis/home. If your IDE does not have NuGet installed by default, check its extension manager or get it from https://www.nuget.org/. Newer versions of MonoDevelop should include NuGet by default.

To install the NuGet package, add a repository for the location on you hard drive where the NuGet package is located. Then go to the installation manager, view packages from the newly repository and install deps 1.0.0.

Upstream binaries for all platforms can be obtained from: gtk-sharp 2.12 can be obtainned here: http://www.mono-project.com/download/#download-lin OpenTK 1.0 can be obtainned here: http://sourceforge.net/projects/opentk/files/opentk/

For all platforms an IDE compatible with solution and csproj files can be used to build the project.

Command Line Building

Building with Mono

From the root of the source tree type:

xbuild

Building with .Net

Open the solution file with an IDE, or from the root of the source tree type:

msbuild