20141013

Harvard Librarians Advise Open Access Publishing

Excellent. The Harvard university librarians have written a letter the Harvard faculty and staff encouraging they start publishing in journals that make content free to the public, known as open access journals, as opposed to hidden behind a pay wall. I have been watching this debate for some time as a number of the UofU CS professors have been arguing for exactly this change.

I quite like the policy at the Machine Learning Lab here in Montreal which requires us to publish our articles on Arxiv.org, a database for freely publishing and accessing of scholarly works. It’s not without it’s challenges. For instance you never know the quality of a given paper that you find on Arxiv until you have invested time in reading it. Many arguing for the open access model have been actively trying to devise strategies for such problems. Regardless I believe it’s preferable to not having access to a paper that should probably be cited.

From a grad student's perspective it is nice because I don’t have to spend time submitting special requests for access to articles and then waiting to receive them. It could end up meaning that I have to pay to have my articles published but I personally prefer this because I want my work available to others to hopefully build upon.

20140413

Installing RL-Glue and ALE without Root Access

In 2012 Bellemare, Naddaf, Veness, and Bowling [1] introduced the Arcade Learning Environment (ALE) for developing and evaluating general AI methods. It interfaces with an Atari 2600 simulator called Stella. They motivate their use of the the Atari 2600 because it permits access to hundreds of "game environments, each one different, interesting, and designed to be a challenge for human players." ALE also interfaces with RL-Glue, a collection of tools for developing and evaluating Reinforcement Learning agents.

RL-Glue is a two part system; a language agnostic part, referred to as the core, and a language specific part, referred to as the codec. There are multiple codecs supporting development in C/C++, Java, Lisp, Matlab, and Python.

The following discusses installing RL-Glue and ALE. First I cover installing RL-Glue and then ALE.

Installing RL-Glue

The instructions for installing the RL-Glue core and Python codec are drawn from the technical manuals for versions 3.04 and 2.0 and assume versions 3.04 and 2.02 respectively. However, looking over older versions of the manual it appears these instructions have not changed much, if at all, which means they may also work for future versions with the correct changes in the commands. I leave it as an exercise to the reader to determine what changes are needed. (Don't you love it when an author does that?)

Installing RL-Glue in the user space as opposed to system wide requires compiling the core code from source. To do so, execute the following commands.

RL-Glue Core

1. Download RL-Glue Core
$ cd ~ && wget http://rl-glue-ext.googlecode.com/files/rlglue-3.04.tar.gz
2. Unpack RL-Glue Core
$ tar -xvzf rlglue-3.04.tar.gz
3. Make a directory into which RL-Glue will be installed:
$ mkdir ~/rlglue
4. Configure RL-Glue Core
At this point it is necessary to identify the location wherein you wish to install the core as you must tell this to the configure script. I made a directory called rlglue in my home directory resulting in the following command:
$ cd rlglue-3.04 && ./configure --prefix=<rlglue>
where <rlglue> is the absolute path to the directory into which you want RL-glue installed.
5. Build and install RL-Glue Core
$ make && make install
6. Update your PATH
Because the core has been installed in a non-standard location it is necessary to inform the system of it's location. This merely entails updating your PATH environment variable to include rlglue/bin, i.e.:
$ export PATH=~/rlglue/bin:$PATH
Note that you can make this change in your .bashrc to avoid doing it every time you open a new terminal.

At this point executing rl_glue should result in:
RL-Glue Version 3.04, Build 909
RL-Glue is listening for connections on port=4096
This indicates that RL-Glue is waiting for programs managing the agent, environment, and experiment to connect. For now you can type ctrl-c to exit the program.

Python Codec

1. Download the Python Codec
$ cd ~ && wget http://rl-glue-ext.googlecode.com/files/python-codec-2.02.tar.gz
2. Unpack the codec
$ tar -xvzf python-codec-2.02.tar.gz
3. Update your PYTHONPATH
This is another step that is only necessary because we're installing into a non-standard location.
$ export PYTHONPATH=~/python-codec/src
Note that this is another command that can be placed into your .bashrc to avoid executing it every time you open a new terminal.


Installing ALE

Installing ALE takes a little more effort. This is due to the fact that ALE does not supply a configure script to build a makefile specific for your system. These instruction are for installing ALE 0.4.3 and may not extend to newer versions well so your mileage may vary. As before, execute the following commands.

1. Download ALE
$ wget http://www.arcadelearningenvironment.org/wp-content/uploads/2014/01/ale_0.4.3.zip
2. Unpack ALE
$ unzip ale_0.4.3.zip
3. Select Makefile
ALE supports Linux, OSX, and Windows and as such a makefile is supplied for each platform. Installing ALE requires making a makefile from one of these. I advise copying the one you want as opposed to renaming it:
$ cd ale_0.4.3/ale_0_4/ && cp makefile.unix makefile
Update 2014-04-15: Frédéric Bastien notes that this step can be avoided by supplying the name of the preferred makefile to make on the command line as follows:
$ make -f makefile.unix
Still be sure to change your working directory to ale_0.4.3/ale_0_4 as the following commands assume that context.

4. Enable RL-Glue support
RL-Glue support in ALE is disabled by default. To enable it edit the makefile and change the line:
USE_RLGLUE := 0
to
USE_RLGLUE := 1
It is also necessary to inform ALE where the RL-Glue headers are located. This can be done by changing the line:
INCLUDES := -Isrc/controllers -Isrc/os_dependent -I/usr/include -Isrc/environment
to
INCLUDES :=-Isrc/controllers -Isrc/os_dependent -I/usr/include -Isrc/environment -I<rlgluedir>/include
where <rlgluedir> indicates the directory into which rlglue was installed earlier.

Similarly it is necessary to inform ALE where the RL-Glue libraries are installed. This is done by changing the line:
LIBS_RLGLUE := -lrlutils -lrlgluenetdev
to
LIBS_RLGLUE := -L<rlgluedir>/lib -lrlutils -lrlgluenetdev
Update 2014-04-15: Frédéric Bastien notes that one can override these variables on the command line. For example:
$ make USE_RLGLUE=1
sets USE_RLGLUE to 1. However it is unclear how to append to variables via the command line so this may only work for the USE_RLGLUE variable without restating the existing variable value.

5. Build ALE
$ make
6. Update LD_LIBRARY_PATH
Because the RL-Glue libraries have been installed in a non-standard location it is necessary to tell ALE where to find them. This is done using the LD_LIBRARY_PATH environment variable as follows:
$ export LD_LIBRARY_PATH=<rlgluedir>/lib:$LD_LIBRARY_PATH
Note that this is another command you can add to your .bashrc to avoid needing to execute the command every time you open a new terminal.

7. Update your PATH
As before it is necessary to update your path to inform the system of the location of the ALE binaries since they are installed in a non-standard location.
$ export PATH=~/ale_0.4.3/ale_0_4:$PATH
Note that this is yet another command you will have to execute every time you open a terminal unless you add this command to your .bashrc.

At this point you can execute ale which should result in:
A.L.E: Arcade Learning Environment (version 0.4)
[Powered by Stella]
Use -help for help screen.
Warning: couldn't load settings file: ./stellarc
No ROM File specified or the ROM file was not found.
Disregard the warnings, they are simply saying that ALE was unable to find your stella configuration and that a game ROM was not specified. This is to be expected since you did not specify the locations for them.

Conclusion

This covers installing RL-Glue and ALE. I have not discussed anything beyond basic testing as such material can be found in the documentation for the tools.

In a future post, or possibly more than one, I will outline the processes for making and executing agents, environments, and experiments.

References

[1] Bellemare, Marc G., et al. "The arcade learning environment: An evaluation platform for general agents." arXiv preprint arXiv:1207.4708 (2012).

20140404

What's Wrong with the Turing Test?

If I could answer one question through the course of my research it would be "what is intelligence?" This question like no other drives my studies. I wrote this post a while ago but did not post it. I did not post it because I intended to refine it. But the reality is I will always be refining my thoughts on this topic. Tonight I went out to the pub with several of my colleagues at Université de Montréal and this topic came up reminding me that I need to just put this out there. As such I am posting it now, with some small changes. I look forward to your responses.

The question "what is intelligence?" is non-trivial. We have been seeking an answer for millennia. While many definitions have been offered [1] no single definition has really ever dominated. Even in the sixty or so years that we have been seriously studying how to create an artificial intelligence we have never actually formally defined intelligence. Ask 100 people to define it and you will likely receive 100 different definitions [1], [2]. The de facto standard is the Turing test developed by Alan Turing [3]. History tells us that he was worried about getting mired in a long drawn out philosophical debate which would likely prevent any progress on actually creating an artificially intelligent being.

The Turing Test as it has come to be known is a variation on a game known as the imitation game [2] wherein a participant, the interrogator, attempts to identify which of the other two participants, one male and one female, was in fact male. Of course the female's objective was to fool the interrogator. The crux of the game was that the decision had to be made solely from communication patterns. In other words, the interrogator did not meet nor could they see the other participants. Additionally, to avoid cues from writing styles, messages would be passed between them in an anonymous way, such as through a computer terminal.

In the Turing Test the objective is to identify the artificial being [4], the computer, as opposed to the male. The hypothesis being that if the interrogator cannot differentiate the artificial being and the human, the artificial being must necessarily be intelligent if we accept that humans are intelligent. This test is clever because it does not require a definition of intelligence nor a measure of intelligence other than agreement that humans are intelligent. However the Turing Test is a behavioral test. Not everyone accepts the test. One of the more well known opponents is John Searle, a professor of philosophy at the University of California, Berkeley. Dr. Searle offers the Chinese Room argument in counter to the Turing Test.

According to the Chinese Room argument we can construct a process that appears intelligent but in fact is not. We do so by placing a person in a room, particularly one that does not speak Chinese. The resident will receive messages from outside the room written in Chinese. The resident must then consult a set of instructions that, when followed, dictate the appropriate response. One, that to any outside observer, would have to have been written by someone that knows Chinese. Since the resident does not know Chinese it supposedly follows that intelligence had only been imitated by the process.

There are a number of counter arguments to the Chinese Room argument. Some argue that the analogy breaks down as a result of the fact that a human performing the processing would simply take too long. Others argue that the room itself is intelligent. But I digress.

While I don't personally accept the Chinese Room argument I do agree there is a flaw in the Turing Test. Specifically, by the nature of its construction, it will permit us to classify a being as intelligent if it behaves like a human. From this we have to conclude that everything else is either not intelligent or at least not classifiable as intelligent without some added criterion.

This not only applies to the animals but to all other beings. Consider the scenario wherein we are visited by aliens that can talk to us, that can do incredibly complicated mathematics, even teach us a few things, and possess technologies way beyond our understanding such as a clearly engineered means of interstellar travel which they used to come to Earth. Would we consider these beings intelligent? We can all think of scenarios wherein the answer is "not necessarily" but in all likelihood we would agree that they are in fact intelligent. But how likely is it that the Turing test will apply?

Of course this problem applies to artificial beings as well, i.e. our computer programs. Have we already created an artificial intelligence? Some might argue we have with Cleverbot garnering 59.3% positive votes from 1,334 participants at the Techniche 2011 festival. Others would likely respond that the real turing test involves physical interaction, i.e. shaking hands with the being, and still not being able to discern a difference. This again highlights the problem.

A precise definition of intelligence would address this problem. However it would not only allow us to differentiate between intelligent and not and answer the question of whether we have already created an artificial intelligence. But it could allow for the development of metrics for comparing intelligences and even help us understand why our existing creations are not intelligent, if that is truly the case.

[1] Legg & Hutter, 2006, A Collection of Definitions of Intelligence, http://www.vetta.org/documents/A-Collection-of-Definitions-of-Intelligence.pdf
[2] Pfeifer, 1999, Understanding Intelligence
[3] Turing, 1950, Computing Machinery and Intelligence http://www.csee.umbc.edu/courses/471/papers/turing.pdf
[4] Russel and Norvig, 2009, Artificial Intelligence: A Modern Approach Third Edition

20140308

Day One in Canada

I crossed the US-Canada border today.

The process was fast and simple enough. However many of the "facts" on my visa were wrong and the gentleman at the border had to fix them. Evidently my last name was "Dustin James Webb" and I had no first or middle names, I was listed as a mechanical engineer not a computer scientist, and I was destined for Calgary, Alberta, not Montreal, Quebec. Oh, and I came to Canada in 2002, not today. I guess all the paper work we submitted to the NY consulate was for show as was the processing time for our visas. And no, my identity was not stolen. I won't go into the details on how I know this though.

Shortly after leaving the border I stopped for food. Ironically the first place I saw was a Tim Hortons. Naturally I had soup and a coffee to combat the cold. Even given all the warnings from friends and family about just how cold it is here I was still not prepared. It is insanely cold. And yes, I consider 30-40 degrees + large windchill factor to be insanely cold. I don't look forward to sub-zero temperatures.

Unfortunately I lost internet access on my phone at the border. Apparently T-mobile does not offer it to their US customers through their local partner. Worse yet I have had horrible cell reception. Google maps cached the info to get from the border to Toronto. But that took me downtown and I was unable to get direction anywhere else at that point. As such I set out to find a gas station and a hotel which are both very difficult tasks without reliable means of communication. This would not have been a problem if I had actually let my wife book my hotel when she had intended. As it is I am paying nearly twice what I should be for a hotel tonight. C'est la vie!

I also had fun trying to fill my tank. For reasons I still don't understand the gas pumps at the station I found would not accept our credit card. So I went into the store to pay in advance. The attendant asked how much I wanted so asked the price. He said it was 1.50 CAD!  I was astounded. I have been paying about $3.50 on average throughout this trip. But I shrugged it off and asked for $12 because our Prius only has an 8 gallon tank. It turns out they measure gas in litres here. Duh! Place this one in the category "should have seen that coming." Oh well, my tank is currently half full which is better than the gallon or so I had when I found the station.

Tomorrow, Montreal!

20140116

Making Learning Fun with Electronics and Robotics

In my last two posts I discussed my thoughts on making learning fun. In the first post the vehicle was games while in second post the vehicle was scientific experimentation. Electronics and robotics seem to me another possible vehicle. While I have a fair bit of experience with these topics I have not done much with them as a educational tool.

I'm most excited about using the Lego Mindstroms. This is Lego's robotics kits. While it is recommended for kids ages 10 and up it actually allows for construction of real autonomous robots. It comes with several sensors for measuring features of the environment such as light intensity, color, distance to the nearest object, audio, and the rotations of it's own motors. It introduces children to programming using a language called NXT-G which is a visual programming language. Think programming via legos. And of course it's compatible with all things Lego.

I look forward to use the Mindstorms to teach everything from mechanics and programming to how to use sensors and basic motion planning techniques.

My motivation for using the Mindstorms as an educational tool comes from my experience volunteering for First Lego League (FLL). FLL is a program for children between 9 and 15 years of age that promotes science and technology. Each year the kids are given a topic about which they must learn. To aid in the learning process they are given a large game board with lots of challenges built out of Lego pieces. The kids must then build one or more robots to solve the challenges. FLL is part of a larger program called FIRST, but is my favorite because the robots made by the kids are actually truly autonomous. For several years now I have volunteered as a robot design judge and through that effort I have seen just how excited the kids can get about learning and solving real world problems.

My children are not yet old enough to participate in FLL. However my son and I have started a Jr. First Lego League (Jr.FLL) team. Jr.FLL is like FLL but shoots to teach the basics of design, mechanics, research, and team building. In fact he and his team will be showing off they have learned about natural disasters on January 25th (2014) at the University of Utah Student Union building. Please feel free to come talk to them. I must warn you though, the FLL finals will be going on at the same time so the place will be an absolute mad house.

An alternative to the Mindstorms are Bo & Yana by iPlay. They are meant to teach the basics of programming, sensing, and actuation. We have not received ours yet but they look promising and are even compatible with other systems like the Mindstorms which should allow for a simple transition when the time is right.

Still another alternative are the solutions from Modular Robotics called Cubelets and their latest product called MOSS. These are just cubes with basic sensing and actuation capabilities that snap together using magnets. But in connecting them one is making simple robots. Unfortunately the MOSS Kickstarter came and went before I could get involved. If anyone has these, I would be interested in hearing about your experience with them. Specifically MOSS.

Of course to build a robot it helps to know something about electronics. One need not be an expert by any means but it helps to be able to build simple circuits. A few years ago I encountered a method for teaching children about circuits called Squishy Circuits. The foundation of this idea is to use "playdough", i.e. modeling compound, to make circuits. It turns out that if you make a modeling compound using a salt base it conducts electricity. Conversely, if you make a modeling compound with a sugar base it does not conduct electricity. As such you can make small sculptures from the two different types. Circuits can then be formed by connecting conductive portions of the sculpture with discrete components like a battery pack and LEDs.

There are a lot of other solutions out there for teaching children about electrics and circuits but I have no experience with any of them. Some of the ones I have found are:

Again, if you have any experience with these I would like to hear your thoughts.

20140108

Making Learning Fun with Experiments

In my last post I talked about how to take advantage of games to make learning fun. By no means is this an original idea. In fact it's rather obvious. Another approach which is probably again pretty obvious is to do experiments.

Their are numerous sites describing fun experiments. One that everyone seems to know is the classic baking-soda-vinegar volcano. For anyone that may be unfamiliar you take something like dirt or clay and sculpt a volcanic cone with an extra deep caldera. From there you pour in some baking-powder into the caldera. Finally you pour in some vinegar. The baking soda and vinegar react and discharge carbon dioxide in the form of bubbles that are heavier than air so, when done correctly, the bubbles overflow the volcano and spill down the sides much like a lava flow from a real volcano would.

This experiment is a great experiment because it opens the door to talking about all kinds of things from the structure of the earth, to how volcanoes form, to how volcanos can lead to other natural disasters like earth quakes and tsunamis, how pyroclastic flow can in a way preserve whatever it hits. For us the volcano devolved to just mixing baking soda and vinegar. After all, who doesn't like watching a vigorous chemical reaction? Even this is great though as it opens the door for talking about things like pressure, surface tension, and chemistry. In a similar vein, the Coke and Mentos experiment is always fun to do. Last I read the reaction wasn't well understood but it's clear that the process is releasing a lot of gas in short order.

Another experiment we have had fun with is the basic electromagnet. Again for those that are unfamiliar you wrap a length of wire around something like an iron bolt and connect the two ends of the wire to the opposite ends of a battery. It is important that the object around which you wrap the wire is ferrous. We usually call this object the core. The movement of the electrons through the wire then produces an electromagnetic field which is amplified by whatever you use for your core.  From there you can use whatever other magnetic objects you have lying around to show the formation and destruction of the magnetic field as you connect and disconnect the battery.

It is not necessary for the experiment to seem obviously fun though. Take for instance testing soil types. In this experiment you gather a couple of soil samples from different places. Potting soil and dirt from outside are great. You put the potting soil in a jar, the dirt in another, and then mix of both into yet a third. Then fill the jars with water, cap them, and shake. What do you get? Mud! Of course the educational part comes from the discussion that ensues when everything settles and talking about how long it takes to settle. My son and I played with this experiment for a couple of days.

As I eluded to earlier, the list of possible experiments is endless. They often lead to the same discussions but that doesn't make them any less fun. To close out this post I will leave you with some links to other particularly fun experiments:
  • Rock Candy: This one takes a bit longer but you get a treat in the end.
  • Fluorescent Jello: Not so tasty, but it glows in the dark!
  • Squishy Circuits: This is great for introducing children to both chemistry and electronics.

20140101

Making Learning Fun with Games

In the last few days I have had a couple of conversations with friends about teaching children. These conversations have inspired me to write a bit about my thoughts on the topic. My experience basically comes from teaching my own son. I strongly believe in the need for parents to supplement their childs education. To the point that I try to work with my kids a little bit every day.

As anyone with children will likely tell you it can be difficult at times to maintain their interest. Even if you are incredibly passionate about a topic it can be challenging to imbue them with that passion. One thing I have noticed while working with my son is that he gets excited when the solutions come easily. Conversely, failure to immediately understand quickly leads to disinterest. This would seem to imply a need for instant gratification. For that reason I often seek ways of removing that need or replacing that need in some way.

One obvious but still great method to address this problem while still providing a lesson is through games. Any gamer will tell you as much. Not just because they are attempting to justify their pastime but because any games provide myriad lessons in the guise of entertainment. Two of my favorite games are DragonBox and LightBot.

DragonBox teaches the principles of algebra without focusing on the mathematical foundations. It simply challenges the child with a puzzle that involves isolating an object from a set of others using the rules of algebra. Early in the game it doesn't even use numbers, just pictures, so that the child may focus on the rules. Unfortunately there isn't enough content. My son has beat this game numerous times and basically lost interest.

LightBot teaches the basics of programming. The objective is to get a robot to turn on lights placed throughout an environment. The crux is that the series of commands needed to execute the task must be provided before the robot ever does anything. As with all games it starts off simply and increases in complexity. In this case the complexity comes from restricting the number of commands the player can use, requiring the use of subprocedures, requiring the application of recursion, and the like.

Another game we have been playing is from MindSnacks. Specifically we've been studying French to help our son prepare for entry into the Montreal educational system where at least a third of the class is taught strictly in French. In total it has nine subgames but only permits the child access to two at the beginning. The child must gain levels to unlock the others. It also focuses the child on certain aspects of the topic of study. In the case of French, and probably the other languages it supports, the topics include numbers, colors, days of the week, and greetings for a total of 50 different topics.

Of course games don't have to be deemed educational to in fact be educational. One of my son's favorite games is Minecraft. He prefers creative mode and will play for hours constructing little houses and zoos for all the animals he hatches. I don't particularly care for the game myself but it has a lot of great educational aspects to it. For instance it is great for talking about Geometry, from the different types of shapes we study to the difference between 1D, 2D, and 3D. Because it is in part focused on crafting it also offers a segue into talking about how real things are made.

Another "non-educational" game that I like is StarMade. This one is inspired by Minecraft but takes place in space. The objective is to make a spacecraft and fly it around collecting materials and fighting space pirates. I particularly like this one because it is more challenging than Minecraft but my son finds it interesting enough to work through the challenges. For instance, he would prefer to play and have to practice his reading to accomplish his goal than not. This is significant because he has not yet found that reading for the sake of reading is fun. It also offers additional lessons to those found in Minecraft. For instance, building a spacecraft requires an understanding of the different parts including the different computers required (e.g. control computer, weapons computers) to engines and shielding.

Games are not the only method for making learning fun. I also like to use experimentation to bring lessons to life but I'll talk more about this in my next entry. I have also been looking for a way to introduce my son to real world electronics and robotics. As part of this we have created a Jr. FLL but this is limited to simple machines and designing solutions. There are a lot of products being made to go beyond this which I will also discuss later.