Category Archives: racket

Choose a random function from a list in Racket

185924A few days ago I asked students to use the Racket library for ASIP to control the movement of a MIRTO robot to explore an unknown area, as follows:

  • The robot should start by moving forward.
  • When the robot hits an obstacle, it should stop immediately and move backward for 0.5 seconds
  • At this point, the robot should perform a left or a right rotation (randomly), and then restart and move forward until the next obstacle.
  • As an additional feature, the amount of the rotation should be random as well, say between 0.3 and 1.5 seconds (not needed for this discussion).

A group of students wrote two functions, one to rotate left and one to rotate right, something similar to the following:

#lang racket
 
(define moveLeft
  (λ ()
    ;; code here to move left, using the 
    ;; racket-asip library
    (printf "The robot moves left\n")
    )
  )
 
(define moveRight
  (λ ()
    ;; code here to move right, using the 
    ;; racket-asip library
    (printf "The robot moves right\n")
    )
  )
 
(list-ref (list (moveLeft) (moveRight)) (random 2))

The key part here is the very last line: the students defined a list of two functions with (list (moveLeft) (moveRight)) and then used list-ref to get one element from this list at a position which is randomly 0 or 1, depending on the result of (random 2). I really liked their idea and I was very happy to see that they were thinking “in a functional style“.

There is, however, a problem with the code above: if you try to run it, the robot will move both left and right. If you copy-paste the code above in DrRacket, you will see that, when you run it, both statements “The robot moves left” and “The robot moves right” will be printed. The students called me and I suggested a solution that – I now realise – was affected by my many years of imperative-style programming… Instead of going through the solution I suggested in the lab, let’s see how we can fix the code above in a more appropriate way. Let’s start from the original problem: try the following code in DrRacket

(list (+ 1 2) (+ 2 3))

You should obtain a list with values 3 and 5. That is to say: the list contains the results of the operation. The same thing happens in the code above: writing (list (moveLeft) (moveRight)) produces a list that contains the result of invoking moveLeft and moveRight. Effectively, this means printing something on screen, because the functions have a side effect. But what is the content of the list? It is just the value returned by the two functions: in this case, it is void, because printf returns void, and as a result choosing a random value from a list of voids returns nothing!

What should been done is to build a list of references to the functions, and this is very easy: just remove the brackets around them, and the references will not be executed! Try the following in DrRacket:

(list moveLeft moveRight)

Now you should see ''(#<procedure:moveLeft> #<procedure:moveRight>): that’s correct! This is a list of two procedures! What happens if you try now to get a random value? Try the following code a few times:

(list-ref (list moveLeft moveRight) (random 2))

Sometimes you will get a reference to moveLeft, and sometimes you will get a reference to moveRight. Hey, it works! But wait, we are not done yet: we have a reference to a function, but how do we call it now? This is the easy part… If you have a function, how do you ask Racket to “execute” it? You  put it in brackets, so overall you just need to surround the list-ref command with another pair of brackets, and the reference to the function selected randomly from a list will be executed:

( (list-ref (list moveLeft moveRight) (random 2)) )

If you had an actual robot, you would see that the robot now moves sometimes left, and sometimes right. If you don’t have robot, try the code above a few times and you will see that sometimes it prints “The robot moves left”, and sometimes it prints “The robot moves right”. You can obviously extend the pattern to a list of functions of arbitrary length and obtain a much shorter code compared to using a switch statement.

How to make a time lapse using a Raspberry Pi and Racket

racketlapseI have been using the Raspberry Pi camera for a few days and I have been really impressed by the quality of the image and video. As you know, I have also been playing a lot with Racket recently. Yesterday I thought that one could take a time lapse using Racket. In fact, I thought that one could build a “portable time lapse maker” as follows:

Put everything in your backpack, go on location, switch on the Raspberry Pi, attach your phone to the WiFi created by the Raspberry Pi, launch the job and you have an extremely portable, light and (hopefully) not-too-power-hungry time lapse maker. For instance, you could leave it on top of a mountain or next to the sea at night and go to pick it up the following day.

There was nothing on TV yesterday night, so I tried to see if it was possible to build a prototype. In the end, it turned out to be easier than I expected :-). I just had to play a bit with the parameters of raspistill but overall it took me:

  • 45 minutes to write takeTimeLapse.rkt from beginning to end.
  • 20 minutes to debug it and fix a few problems with parameters for raspistill.
  • 35 minutes to build the Illy waterproof box as described in the link above.
  • 10 minutes to build the cardboard version to be attached overhead (see picture above).
  • 25 minutes to write WebRacketLapse.rkt (with very little debugging, so it is probably not going to work very well… I have already noticed a bug: the thread variable is not set to null when the thread finishes, this needs to be fixed).

I have uploaded the code here: https://github.com/fraimondi/racketlapse. If you want to try it, just download it to your Raspberry Pi and launch racket WebRacketLapse.rkt.

This is the result of 1h30 mins of coding, so use with extreme caution! Contributors are welcome…

These are the results of some experiments:

  • Dawn in Temple Fortune: http://www.youtube.com/watch?v=BHUBwiwjZcs. I left the Illy box outside for the night using a 9000 mAH rechargable battery and I managed to have enough power to go for 8 hours and a half (the video is trimmed at the begin and at the end); the temperature was around 7 C. Considering that the WiFi USB dongle is probably using a lot of power, one can expect at least 10 hours of battery life if the dongle is disconnected. 8 hours of 1920×1080 images taken every 15 seconds took approx 2 Gb of disk space. I have and 8 Gb SD card, so plenty of space left.

I am happy with the results so far and. If anyone  has time to make this more user-friendly with a bespoke image for the Raspberry Pi and a better user interface in HTML, then I think it could have a number of applications. Drop me a line if you are interested.

 

The MIddlesex Robotic plaTfOrm: MIRTO

MIRTOMIRTO (the MIddlesex  Robotic plaTfOrm, also known as Myrtle) is an Arduino+Raspberry Pi platform currently used for teaching in the first year of the Computer Science degree at Middlesex University. It is developed as an open-source platform and the design and source code are available online (see links at the bottom of this post).

An overview of this first year has recently been given by Tony Clark at http://clarktony.blogspot.co.uk/2013/12/computer-science-approach-to-teaching.html, including  the motivations for our choice of Racket as the core programming language.

Myrtle is the main character of “block 3″ for first year Computer Science students (see previous link for an introduction to the idea of “blocks” and for an overview of our teaching and assessment strategy). The structure of Myrtle is the following:

1. A base layer incorporates Infra-Red sensors, bump sensors and a pair of HUB-ee wheels (see figure below):

sensors

 2. In the center, an Arduino Uno collects the data from the sensors and is connected to the wheels to drive them (see figure below): arduino-layer

3. The top layer for Computer Science students is a Raspberry Pi that is connected to the Arduino Uno board by means on a serial connection on the GPIO pins (a WiFi USB dongle is also visible in the figure below).

IMG_20140204_152433

This is a very flexible platform: Engineering students will employ mainly the Arduino layer (possibly two Arduino layers, one on top of the other) together with the Arduino IDE. Computer Science students will work mainly at the Raspberry Pi level, possibly extending the core platform with additional features (USB cameras, a fourth layer on top, etc.). More importantly, the platform offers a number of opportunities for teaching core Computer Science notions: from product of finite state machines to concurrency and synchronisation when reading the encoders, from networking at all levels (IP, TCP, applications) to data structures, functional programming, etc.

The software architecture of the platform for Computer Science students involves:

  1. A modified version of Firmata running on the Arduino Uno. This is an extension of Standard Firmata developed by Michael Margolis, the author of various books including the “Arduino Cookbook” and currently at Middlesex University.
  2. A Racket Firmata module extended with SysEx messages and other features used in Myrtle. This version works on Windows, Linux and Mac and detects the operating system automatically, together with the port to which the Arduino is attached.
  3. A Racket module (MIRTOlib.rkt) to interact with Myrtle using the Firmata library described above.

The following is an overview of block 3 for first year Computer Science students:

The students are initially exposed to the Arduino layer only to familiarize with MIRTOlib.rkt. In the figure below, Myrtle is provided without the Raspberry Pi layer and is connected directly to a PC or laptop using a USB cable. We employ this configuration to reason about interaction between components (wheels and sensors) using finite state machines, use cases, statecharts and sequence diagrams. We also have a local installation of MediaWiki for students so that they can document their progress using wiki pages they create. In the picture below, DrRacket is running on the PC/laptop and the Arduino layer of Myrtle is connected using a USB cable:

arduino-layer

Teaching then moves to networking, introducing IP addressing, TCP and network applications. The main result is an HTTP server built in Racket that can control an Arduino board using Firmata to turn on and off LEDs. The following is an example handout for the students addressing the HTTP server in Racket, starting from unquote + splicing, then moving to xexpr and finally to the actual HTTP server (click on the link to download the PDF):

A simple HTTP server in Racket (PDF)

In parallel, we start introducing some basic functions provided by MIRTOlib.rkt so that the students can move the wheels and read bump and IR sensors.

We then move to introducing a new architecture and a new operating system: Raspberry Pi  (ARM) and Linux. At this point we add the third layer to Myrtle and we “detach” the robot from the PC/laptop. We now access the robot over a wireless connection on the Raspberry Pi, where students upload Racket code and run it from the command line.

myrtlelab-top

The material now asks the students to reason about control . For instance, students are asked to print the IR sensors every 2 seconds and to move the wheels for 2 rotations, stopping immediately if one of the bump sensors is activated. In parallel, student learn how to read and post tweets using the Twitter API from Racket and they study encryption algorithms, authentication mechanisms and, in particular, OAuth 1.0 for Twitter.

Finally, students have a go at “control theory” with the problem of line following. This gives us a chance to talk a little bit about calculus (derivation and integration) to build a PID controller for the robot (click on the image for a video about this, available at http://www.youtube.com/watch?v=laafaTZ7mDU).

linefollowing

In the last two weeks the students will split into groups and will work on a project of their choice. Ideas include a web interface to drive the Myrtle wireless, control via majority voting using Twitter, advanced line following in various conditions, dancing robots, etc. I will provide additional details at the end of this block, together with the Racket code for line following and other examples: I don’t want to publish it now because students need to find it on their own!

So far, the results are very encouraging: we are nearly at the end of the course, with only 6 weeks left, and attendance is regularly above 90% (yes, this is 90% for all lab sessions in a cohort of approximately 120 students). Students are engaging with the material and it is common to see students remaining in the labs well after the end of the sessions.

Setting up this block has required a lot of preparation work by a number of people. Apart form all the academics and teaching assistants from the Computer Science department, we had enormous help from Michael Margolis, Puja Varsani and Nick Weldin.

In terms of practical issues:

  • So far, the robots seem to cope well with students using them, no major hardware failure reported in nearly 5 weeks of continuous usage. We have approximately 20 robots and we typically use 7 or 8 of them per session (each session is attended by approximately 20 students).
  • We have approximately 20 battery banks (9000 mAh), which are enough to go through a day of sessions. The batteries are recharged overnight.
  • Every week we provide a set of SD cards for the Raspberry Pi, setting up an environment with Racket and all the additional software required (voice recognition and image capture, for instance)

The source code for this platform is available at:

  • https://github.com/fraimondi/myrtle: Arduino code for the Arduino Uno layer (modified Firmata), MIRTOlib.rkt and some simple testing functions. Design files will be added very soon.
  • https://bitbucket.org/fraimondi/racket-firmata: platform-independent (in the sense that it seems to work well in Linux, Mac and Win) Firmata client for Racket. We have tested it with “standard” boards and it seems OK, even without Myrtle.
  • We also have an SD card image ready that includes Racket 5.3.6 pre-compiled. Send me an email if you want one of these.

Feel free to contact me if you need additional details!

An interesting problem with my socks

4160OFhBZiL._SX280_SY418_SH10_QL100_I have recently bought “7 Pairs of Cotton Rich Freshfeet Days of the Week Socks” from Marks & Spencer. Essentially, these are standard socks with the name of the day printed on them, from Monday to Sunday (i.e.: I have two socks marked “Monday”, two socks marked “Tuesday”, etc). They also come in different colours so that you don’t risk mixing socks from different days…

For the first week I tried to wear the correct socks for the correct day, but I immediately gave up and now I just pick a random pair from my drawer. After a few weeks I have noticed that the days in which I’m wearing the “correct” socks are a lot more than I expected (for instance, today is Monday and I have the correct pair, and last week I had the correct pair on two days of the week!). This looks like an interesting instance of the “Availability Heuristic” at work… What’s the probability of having the correct pair of socks?

Let’s simplify the problem: suppose that the week starts on Monday and all the 7 pairs of socks are available on that day. Suppose that every day you pick a random pair and you remove it from the available socks for the rest of the week (you obviously need to wash them…). What is the probability that at least once that during the week you have the right pair of socks for the correct day?

I’m going to show you 2 ways to compute this number (which is probably higher than you’d expect, and consistent with my surprise when I pick the correct pair of socks during the week):

1) The “I don’t have time to think of a solution / I hate Maths” way. Just write a simple Monte Carlo simulation and let the computer do the job. The idea is the following: start from the list (1 2 3 4 5 6 7), randomly shuffle the numbers around to obtain something like (3 4 7 2 1 6 5) (this is called a permutation and it represents the order in which you pick the socks: in this case, on Monday I pick the socks for Wednesday, on Tuesday those for Thursday, etc.), check if one of the numbers is in the right position (in this case yes: number 6 is in position 6, meaning that on Saturday I get the socks for Saturday), repeat the process N times and keep a counter of how many times the shuffled list has at least one number in the right position. At the end, just compute the ratio of this counter with respect to N. Let’s do it in Racket (obviously):

#lang racket
 
;; The number of days in the week
(define DOW 7)
 
;; The total number of iterations
(define N 1000000)
 
;; A simple function that takes a list and returns
;; True if one of the elements is in the right place
(define (oneIsOK myList)
  ;; The ormap builds first of all a new list by 
  ;; comparing the values of the list (1 2 ... DOW)
  ;; with the values of the input parameter myList, one
  ;; by one, thus producing a list of Boolean values.
  ;; It then compute the disjunction of this new list of 
  ;; Boolean values
  (ormap (lambda (value1 value2)
         (= value1 value2))
       (range 1 (+ 1 DOW))
       myList))
 
;; This is a simple for loop that repeats N random
;; shuffles of the list (1 2 ... DOW) and increments 
;; a counter when at least one element is in the right place.
(define (simulation)
  (define counter 0)
  (for ([i (range 0 N)])
    (cond ( (oneIsOK (shuffle (range 1 (+ 1 DOW))))
            (set! counter (+ 1 counter)))
          )
    )
  ;; at the end we just print the percentage
  (printf "The percentage is: ~a" (* 100.0 (/ counter N)))
  )

The code isn’t too difficult, check the comments above. The code runs 1 million shuffles and computes the results, which is 63% (more or less). Sixty-three percent! Wow, this means that it is more likely that in a week I have at least once the correct pair of socks! You can change the number of socks and see how this ratio changes. For instance, with 3 pairs of socks the probability is 66.6%: this is what you would expect, just check the 6 possible permutations of 3 elements.

2) The second method is the “I get a pen and a piece of paper“: let’s look again at the notion of permutation. What is needed here is to compute the number of permutations of (1 2 3 4 5 6 7) in which at least one of the elements “does not move” (more precisely, you should say “all the permutations with at least one fix-point”). One way to compute this number is to count the number of permutations in which all elements move (i.e., the number of permutations without fix-points): a permutation without fix-points is called a derangement. The notation !n is used to denote the number of derangements of n elements. Overall, we want to compute (n! – !n)/n!: this is the total number of permutations, minus the number of derangements, divided by the total number of permutations. To compute the number of derangements, observe that on Monday I can choose between 6 pairs of socks (i.e., all the ones not marked with Monday). Suppose that on Monday I get the Wednesday socks. There are now two options:

  1. I have a derangement in which on Wednesday I get the Monday socks. This leaves me with 5 pairs of socks to be arranged without fix-points, and there are !(n-2) ways for doing this.
  2. I have a derangement in which on Wednesday I do not get Monday’s socks, and there are !(n-1) ways for doing this.

Thus, there are (!(n-2)+!(n-1)) derangements in which on Monday I have the Wednesday socks. But Wednesday was just one of the possible (n-1) choices, so overall the number of derangements for n elements is:

!n = (n-1)(!(n-2) + !(n-1))

This is a nice recursive definition (just set !0=1 and !1=0), which we can compute with the following Racket code:

(define (der n)
  (cond ( (= n 0) 1)
        ( (= n 1) 0 )
        (else (* (- n 1) (+ (der (- n 1)) (der (- n 2)))))
        )
  )

With this function you can now compute the exact probability for 7 pairs of socks, which is 63.21%.

EXERCISE: compute the average number of days per week in which I wear the correct pair of socks. HINT: the solution is 1, and this is equivalent to the claim that “the average number of fix-points in a random permutation is 1“, irrespective of the size of the list…

The classic Simon memory game in Racket + Arduino

OriginalSimon

(Source: Wikipedia)

The Simon memory game is one of the first electronic games I remember from my childhood, together with late 70′s Atari consoles and early 80′s  Game & Watch Nintendo handheld (Donkey Kong was my favourite).

The game is very simple: there are four coloured buttons that correspond to four different tones. The game presents a sequence of colours and the player has to repeat the sequence. If the player is successful, a further colour is added, otherwise a horrible 42 Hz saw-shaped sound is emitted; check this commercial http://www.youtube.com/watch?v=yF0ZUXclW8Y

It is not difficult to implement the game in Racket using an Arduino UNO board with four LEDs and four digital buttons, connected using Firmata.

IMG_20131207_074841I have connected the LEDs to PINs 8, 9, 10 and 11; I have connected the buttons to PINs 2, 3, 4 and 5. The figure on the left shows the spaghetti-like circuit. If you want to see what happens when you run the code, have a look at this link of my daughter playing with it: http://youtu.be/tk69MUwQs5A (in Italian with some subtitles…)

This game is very interesting for teaching purposes: it is easy enough to be covered in a session, but it also contains some issues that are not so simple, such as distinguishing between a state in which the code displays the sequence from a state in which the player is entering the sequence. The sequence is not fixed and it needs to be reset when the player makes a mistake. One exercise I will do in class is to ask the students to write down all the possible states and the possible transitions.

Let’s now  have a look at a possible Racket code for this. I assume you have Firmata installed on your Arduino UNO and you also know how to connect it to Racket using Firmata to read a button. If this is not the case, it is probably better if you have a look at this other link before proceeding: http://www.youtube.com/watch?v=7Zzh7qrQph4. I start with

The following is the description of a possible Racket implementation of the game; it is available with sound files and firmata at this link: http://www.rmnd.net/racket/simon. The implementation could be improved in a number of ways, see at the bottom of this post for a list of tasks for students. I start by including the appropriate Firmata module and by declaring the PINs to which the LEDs and the push buttons are connected to. I then define two lists that I use in other places in the code to initialise the PINs and to do boot sequences.

;; change as appropriate for your OS
(require "firmata-mac.rkt")
 
(define yellow 8)
(define green 9)
(define red 10)
(define blue 11)
 
(define in-yellow 2)
(define in-green 3)
(define in-red 4)
(define in-blue 5)
 
(define colours (list yellow green red blue))
(define in-colours (list in-yellow in-green in-red in-blue))

To keep track of the various buttons being pressed, I extend the code at http://www.youtube.com/watch?v=7Zzh7qrQph4 defining four sets of states for the buttons (all initially UP):

(define curStateYellow "UP")
(define previousStateYellow "UP")
 
(define curStateGreen "UP")
(define previousStateGreen "UP")
 
(define curStateRed "UP")
(define previousStateRed "UP")
 
(define curStateBlue "UP")
(define previousStateBlue "UP")

I then declare a list that I use to store the sequence of states to be guessed with
(define seq '()). The key idea of the game is to use “states” to keep track of what the code should do. We start from a state “gameOver”, declaring it with (define state "gameOver").
I then declare a few support functions, see comments in the code below:

;; Show a sequence of colours: switch on a PIN, play the
;; corresponding sound with play-index (see below), clear
;; the PIN, wait 0.2 seconds and then call itself recursively until
;; there is something to show.
(define (showSeq s)
  (cond ( (&gt; (length s) 0)
          (set-arduino-pin! (first s))
          (play-index (first s))
          (clear-arduino-pin! (first s))
          (sleep 0.2)
          (showSeq (rest s))
          )
        )
  ) ;; end of showSeq
 
;; play a note using its position in the list (yellow - green - red - blue)
;; (see below the definition of play)
(define (play-index i)
  (cond ( (= i yellow) (play "yellow"))
        ( (= i green) (play "green"))
        ( (= i red) (play "red"))
        ( (= i blue) (play "blue"))
        )
)
 
;; play a sound with afplay (works on a mac)
;; in Linux, you can use aplay.
;; It just calls a command-line using system
(define (play note)
  (system (string-append "afplay " note ".wav &amp;"))
  )
 
;; This is used to set-up the mode of the various PINs. I do
;; so using the high-order function map, together with a lambda
;; function
(define (setup)
  (map (λ (i) (set-pin-mode! i OUTPUT_MODE)) colours)
  (report-digital-port! 0 1)
  (map (λ (i) (set-pin-mode! i INPUT_MODE)) in-colours)
)

The core of the code is the function mainLoop that loops through the following sequence of states (code below, after this list):

  • gameOver: this is either the initial state or the state following a wrong choice by the player. In this state we clear the sequence of colours and we generate a new random initial colour. After gameOver the code shows (and play with sounds) the sequence to the player.
  • After showing the sequence, the code moves to state playing. In this state we go through a new loop, see playingLoop below.
  • The code exits playingLoop either when the player guesses all the sequence correctly or when the player makes a mistake. In the former case the code enters a state called “correctSequence”; in the latter it enters a state called “wrongChoice”.
  • In state “correctSequence” the code adds a new colour to seq and then shows the sequence. It then goes back to state “playing”.
  • In state “wrongChoice” the code moves to gameOver and repeats the loop.

This is the full code for the function:

(define (mainLoop)
 
  (cond ( (equal? state "gameOver")
          (printf "DEBUG: Current state is gameOver\n")
          ;; If we are in game over, we clear the sequence, we add
          ;; a random value and we set state to "playing"
          (set! seq '())          
          (set! seq (append seq (list (list-ref colours (random (length colours))))))
          (sleep 0.5)
          (set! state "showSequence")
          )
 
        ( (equal? state "showSequence")
          (printf "DEBUG: Current state is showSequence\n")
          ;; We show the sequence and then we enter playing
          (sleep 0.3)
          (set! state "playing")
          (showSeq seq)
          )
 
        ( (equal? state "playing")
          (printf "DEBUG: Current state is playing\n")
            ;; We enter the main playing loop, see below (player does things)
           (playingLoop seq)
        )
 
        ( (equal? state "wrongChoice")
          ;; The player has made a wrong choice:
          ;; we enter game over state
          (sleep 0.5)
          (set! state "gameOver")
          )
 
        ( (equal? state "correctSequence")
          (set! seq (append seq (list (list-ref colours (random (length colours))))))
          (sleep 0.5)
          (showSeq seq)
          (set! state "playing"))
  )
  (mainLoop) ;; call recursively
) ;; end of mainLoop

The player interacts with the push buttons in the playingLoop function. The function takes in input a sequence (of colours) and expects that the player presses the first colour in the list. If this is the case, it calls itself recursively on the tail of the sequence of colours. Otherwise, it sets state to “wrongChoice” and plays the horrible 42 Hz sound. The code is a simple extension of the code described in FIXME to 4 buttons. The full code is the following:

;; The playing loop: the player has to guess the sequence s, we call ourselves
;; recursively on s.
(define (playingLoop s)
 
  ;; to keep track of a correct choice, used to loop at the end
  (define correct "")
 
  (cond ( (= (length s) 0)
          ;; if the list is empty, we guessed all the colours correctly!
          (set! state "correctSequence")
          (printf "DEBUG: well done, the sequence is correct!\n")
          )
        (else 
         ;; There are more colours to be guessed, we keep looping here
         ;; recording what is being pressed
         (cond ( (is-arduino-pin-set? in-yellow) (set! curStateYellow "DOWN"))
               (else (set! curStateYellow "UP")))
         (cond ( (is-arduino-pin-set? in-green) (set! curStateGreen "DOWN"))
               (else (set! curStateGreen "UP")))
         (cond ( (is-arduino-pin-set? in-red) (set! curStateRed "DOWN"))
               (else (set! curStateRed "UP")))
         (cond ( (is-arduino-pin-set? in-blue) (set! curStateBlue "DOWN"))
               (else (set! curStateBlue "UP")))
 
         ;; switch on the lights and check if it is correct when we release
 
         ;; Blue
         (cond ( (and (equal? curStateBlue "DOWN") (equal? previousStateBlue "UP")) 
                 (printf "DEBUG: blue pressed\n")
                 (set-arduino-pin! blue)
                 (play "blue")
                 ))
         (cond ( (and (equal? curStateBlue "UP") (equal? previousStateBlue "DOWN")) 
                 (printf "DEBUG: blue released\n")
                 (clear-arduino-pin! blue)
                 (cond ( (= (first s) blue) 
                         ;; the player chose the right colour
                         (set! correct "T")
                         )
                       (else (set! correct "F"))
                       )                       
                 )
         ) ;; end of blue button released
        ;; You need to repeat the code above for the three remaining colours,
        ;; see source code available on-line at the link below.
         ;; Setting the previous states
         (set! previousStateBlue curStateBlue)
         (set! previousStateRed curStateRed)
         (set! previousStateGreen curStateGreen)
         (set! previousStateYellow curStateYellow)
         (cond ( (equal? correct "T") 
                 (playingLoop (rest s)))
               ( (equal? correct "F") 
                 (set! state "wrongChoice")
                 (printf "DEBUG: ops, wrong choice!\n")
                 (play "wrong"))
               (else (playingLoop s))
         )
         ) ;; end else (sequence not empty)
        ) ;; end cond
)  ;; end playingLoop

This completes more or less the code. There are other minor things that could be added, see the code available online at http://www.rmnd.net/racket/simon. You can launch the code by setting the appropriate port for Firmata. I have run this code both on a Mac laptop and on a Raspberry Pi and it works fine. I have not tried Windows… let me know if it works for you.

Possible tasks for students:

  • A lot of code is repeated, for instance in playingLoop. Improve it by reducing the amount of duplicated code.
  • Calling OS instructions using (system ) causes the code to stop while the command is being executed because it is a synchronous call. Implement system calls using (process ). You need to manage timing for this is a different way…
  • In the original Simon game, the timing between different colours / sounds in the presentation state as you progress with levels. Modify the code to take this into account.
  • Introduce a winning state, e.g., if the player can guess a sequence of 15 colours.
  • Save best scores, or publish them on-line (Twitter, Facebook, Google+)

Speech recognition on Raspberry Pi with Sphinx, Racket and Arduino

IMG_20131112_112628 copyIn this post I put together a number of things to control two LED from a Raspberry Pi with voice recognition (via Sphinx), Firmata and Arduino. Before you start, you may want to have a look at this other post on how to connect a Raspberry Pi and an Arduino board using Firmata and Racket: http://jura.mdx.ac.uk/mdxracket/.

First of all, we need to install PocketSphinx on Raspberry Pi to do speech recognition. I am using a standard USB camera with microphone (supported by Raspberry Pi) and I’m following the instructions available here: https://sites.google.com/site/observing/Home/speech-recognition-with-the-raspberry-pi. In essence, this is what I’ve done (as root on the Raspberry Pi), please see the link above for additional details:

apt-get install rpi-update
apt-get install git-core
rpi-update

-> Connect your USB microphone (or camera+mic) and
-> reboot the RPi at this point

vi /etc/modprobe.d/alsa-base.conf 

# change as follows:
# Comment this line
# options snd-usb-audio index=-2
# and add the following:
options snd-usb-audio index=0

-> close the file and reload alsa:

alsa force-reload

wget http://sourceforge.net/projects/cmusphinx/files/sphinxbase/\
0.8/sphinxbase-0.8.tar.gz/download
mv download sphinxbase-0.8.tar.gz
wget http://sourceforge.net/projects/cmusphinx/files/\
pocketsphinx/0.8/pocketsphinx-0.8.tar.gz/download
mv download pocketsphinx-0.8.tar.gz
tar -xzvf sphinxbase-0.8.tar.gz
tar -xzvf pocketsphinx-0.8.tar.gz

apt-get install bison
apt-get install libasound2-dev

cd sphinxbase-0.8
./configure --enable-fixed
make
make install

cd ../pocketsphinx-0.8/
./configure
make
sudo make install

Et voila’, you are now ready to test your PocketSphinx installation. Go to pocketsphinx-0.8/src/programs and run:

./pocketsphinx_continuous

If you are lucky, you should get some text back… I don’t have space here (and time) to go into the details of (pocket)sphinx. Try some simple words and see if they are recognised. I ended up building a very very simple language model using the on-line tool at this link: http://www.speech.cs.cmu.edu/tools/lmtool-new.html. I use just “green”, “red” and “off” and they seem to work fine in spite of my Italian accent.

In the next step we need to connect the output of PocketSphinx with Racket. I do this in a very primitive way: I modify the source code of pocketsphinx_continous to output just the word that is recognised. This is very simple: just modify continous.c under pocketsphinx-0.8/src/programs, comment all the printf statement and output just the recognised word (drop me an email if you don’t know how to do this). I append a “RACKET: ” string at the beginning of the printed string to make sure that this is something I have generated. You can then run pocketsphinx and redirect the output to a file with something like:

./pocketsphinx_continuous -lm /home/pi/sphinx/simple/4867.lm \
   -dict /home/pi/sphinx/simple/4867.dic > /tmp/capture.txt

(notice that I’m using the language model + dictionary generated on-line)

Time now to go back to Racket. I assume you know how to connect a Raspberry Pi with an Arduino board and talk to it in Racket using Firmata (if this is not the case, please have a look at the instructions available at http://jura.mdx.ac.uk/mdxracket/index.php/Raspberry_Pi,_Arduino_and_Firmata). In the following Racket file I simply read the file /tmp/capture.txt and send instructions to the board according to the instructions received. If a command is not recognised, I print a message on screen. The code for this is the following:

#lang racket
(require "firmata.rkt")
 
(define green 12)
(define red 13)
 
(define in (open-input-file "/tmp/capture.txt"))
 
(define (process-input str)
  (printf "processing input ~a\n" str)
  (set! str (substring str 8))
  (cond ( (string=? (string-upcase str) "RED")
          (printf "I'm setting red\n")
          (set-arduino-pin! red)
          )
        ( (string=? (string-upcase str) "GREEN")
          (printf "I'm setting green\n")
          (set-arduino-pin! green)
          )
        ( (string=? (string-upcase str) "OFF")
          (printf "I'm clearing the PINs\n")
          (clear-arduino-pin! red)
          (clear-arduino-pin! green)
          )
        (else
         (printf "Sorry I cannot understand: ~a\n" str)
         (flush-output)
         )
  )
  )
 
(define (read-loop)
  (define str (read-line in))
  (unless (eof-object? str)
               (process-input str)
    )
  (read-loop))
 
(define (start-everything)
  (open-firmata "/dev/ttyACM0")
  (set-pin-mode! green OUTPUT_MODE)
  (set-pin-mode! red OUTPUT_MODE)
  (read-loop)
  )
 
(start-everything)

Job done. Now check that:

  • The modified version of pocketsphinx_continuous is running and redirecting the output to /tmp/capture.txt
  • Launch the file above with something like racket sphinx-arduino.rkt
  • Check that your Arduino board is connected and wired up appropriately

and you should get something like this:

http://youtu.be/XGYNRHWY4Ag

Future work:

  • I don’t think there is a need to write to a file… maybe pocketsphinx can redirect to a port and Racket can listen to it?
  • Improve the language model for a domain of your choice
  • Add a couple of speakers to the Raspberry Pi so that Racket can tell you what it is doing, if something has not been recognised, etc.

Racket and TinyOS via USB

IMG_20131029_151628Communication between a PC and a TinyOS mote is normally achieved using Java. But what if you want to use Racket?

Here I give a very simple overview of how packets from a TinyOS mote connected via USB to a Linux or Mac machine can be read using Racket (I have not tried Win). An advantage of this approach is that it does not require the installation of TinyOS, which could be quite cumbersome… However, it may be a good idea to have a working installation of TinyOS on your machine so that you can modify the code on the motes. It would be beneficial to have at least a bit of familiarity with nesC (no need to be able to write code, just to understand a little bit what is going on).

As an example, I start from a simple application that broadcasts temperature and humidity. The application can be installed on various motes (just 2 in my configuration) and then a base station collects all the readings. In my case the base station is connected to the PC using a USB port. In my configuration I have a Ubuntu virtual machine running on a Mac where I have configured the TinyOS environment and Racket is running in this virtual machine. The same code runs directly on Mac. We are only interested in the structure of the message being broadcasted. This is a small variation of the standard examples that can normally be found in a .h header file; in my example the header file contains the following:

1
2
3
4
5
typedef nx_struct BlinkToRadioMsg {
  nx_uint16_t nodeid;
  nx_uint16_t humidity;
  nx_uint16_t temperature;
} BlinkToRadioMsg;

This says that each message contains 2 bytes (16 bits) for nodeid, 2 bytes for humidity and 2 bytes for the temperature, for a total of 6 bytes. Messages are transmitted in TinyOS packets over the wire, and these packets have a special format. See Section 3.6 of the document available at http://www.tinyos.net/tinyos-2.x/doc/html/tep113.html if you want more information. In summary, a packet looks like this:

7e 45 00 ff ff 00 01 06 22 06 00 01 04 5c 18 69 37 6b 7e

where the sequence 7e is a reserved sequence to denote the start and the end of a packet. The actual payload (i.e., the 6 bytes we want to receive) in our case is the sequence “00 01 04 5c 18 69″, where “00 01″ is nodeid, “04 5c” are the two bytes for humidity and “18 69″ are the two bytes for temperature. The rest of the packet includes a CRC check, protocol ID, etc. See link above if you are interested in the details.

To read these packets I modify a little bit the code that is used in firmata.rkt to communicate with an Arduino board using firmata. The idea is simple: I create an input file attached to the USB port and I keep reading bytes. When I recognise a packet, I parse it an print the values of ID, humidity and temperature on screen. Let’s start by creating the input file:

 

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
;; This is the input file to be defined below
(define in null)
 
;; This is used to open the connection and to launch the reading loop
;; You typically launch it with (open-port "/dev/ttyUSB0)"
;; (check the port name with the command-line instruction motelist)
(define (open-port port-name)
  (printf "Racket-TinyOS starting... please wait\n")
 
  ;; This is used to associate the port to the file
  (set! in (open-input-file port-name #:mode 'binary))
  (if (system (string-append "stty -F " port-name " 115200 cs8 cread clocal"))
    (begin
     (read-loop) ;; see below for the definition of this
    )
    (error "Could not connect to " port-name)
  )
)

The (read-loop) simply reads one byte at a time and calls itself:

1
2
3
4
(define (read-loop)
  (process-input (read-byte in))
  (read-loop)
)

The function (process-input) is where the packet is built:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
;; This is where we store the packet
(define packet null)
 
;; These are used to define the end-of-packet and start-of-packet sequences
(define sop #x7e) ;; start of packet
(define eop #x7e) ;; end of packet
 
;; These are used to record the fact that an end-of-packet 
;; (or start-of-packet) have been received
(define eop-rec 0) ;; end-of-packet received
(define sop-rec 0) ;; start-of-packet received
 
(define (process-input data)
  ;; First of all we add the byte to the packet
  (set! packet (cons data packet))
  (cond ( (and (= eop-rec 0) (= data eop))
          ;; If it is an end-of-packet sequence, we record it
          (set! eop-rec 1)
          )
        ( (and (= eop-rec 1) (= data sop))
          ;; If we find a start of packet after an end-of-packet we parse
          ;; the packet (we have to reverse it, because it was built in
          ;; the opposite order)
          (parse-packet (reverse packet))
          ;; and then we start again and we empty the packet.
          (set! eop-rec 0)
          (set! packet null)
          )
        )
)

Up to this point the code is re-usable for any packet content. In the function (parse-packet), instead, we have to process the bytes according to the message format defined in the header file (see above). This function is defined as follows:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
(define (parse-packet packet) 
 
  ;; Very easy: ID is at position 10 and 11
  ;;            Humidity position 12 and 13
  ;;            Temperature position 14 and 15
 
  ;; These functions get the two bytes into the actual ID,
  ;; humidity and temperature. 
  ;; for the actual formula
  (define id (+ (* (list-ref packet 10) 256) (list-ref packet 11)))
  (define rh (+ (* (list-ref packet 12) 256) (list-ref packet 13)))
  (define temp (+ (* (list-ref packet 14) 256) (list-ref packet 15)))  
 
  ;; We then convert RH and temp according to datasheet for sensirion sht11
  (set! rh (+ (* rh 0.0367) -2.0468 (- (* 0.00000159 rh rh))))
  (set! temp (+ (* temp 0.01) -39.7))
 
  ;; Omissis: print the actual date. See link below for full code.
 
  (printf "~a ~a ~a\n" id rh temp)
  (flush-output)
  )

The final result of this code is the following output

2013/10/29 17:04:55 1 61.36022401 10.379999999999995
2013/10/29 17:05:02 2 35.30190249 23.799999999999997

Where the first number after the time is the node id, followed by the relative humidity, followed by the temperature. I have redirected this output to a file called office-all.txt for one hour, placing one sensor inside my office (with sensor id 2) and one sensor outside the office (sensor id 1). At the end of the hour I split this file with the command-line instruction

grep " 1 " office-all.txt > office-out.txt
grep " 2 " office-all.txt > office-out.txt

Then, I plot the temperature with the following gnuplot script:

set timefmt "%Y/%m/%d %H:%M:%S"
set xdata time
set terminal png font "/Library/Fonts/Arial.ttf" 28 size 1600,1200
set output 'office-temp.png'
plot "office-out.txt" using 1:5 title "Outdoor", "office-in.txt" using 1:5 title "Indoor"

This is the final result:

office-tempNotice how the indoor temperature remains constant (in green), while the outdoor temperature drops in 10 minutes when the sensor adjusts to the outdoor temperature, and it keeps decreasing as the sun sets. A very similar script for relative humidity produces this graph:

office-rh

(notice how the relative humidity outside increases as the temperature decreases).

You can download the full Racket code by clicking here.

*** NOTICE!!! *** The parse-packet function is broken: it should take into account the escape sequences before parsing! I leave this to you to fix :-).

A very simple introduction to Fluxus and Fluxa

Screen Shot 2013-10-23 at 13.02.45This is a very simple write-up of my experiments with Fluxus and Fluxa to generate sounds (I don’t call it music…). It is intended for students who know a bit of Racket but have no experience at all with Fluxus and Fluxa. The results are just an example of what can be done, I hope someone will start from these to produce something better.

Back to Fluxus and Fluxa: if you have managed to install it (see my other post on How to Install Fluxus on Mac), make sure that you have Jack running and that you launch fluxa from the command line. Then, launch Fluxus and you can start with the experiments.
First, you need the following line:

(require fluxus-018/fluxa)

Then, you can start playing sounds immediately. For instance, try this:

(play-now (mul (sine 440) (adsr 1 1 1 1)))

The instruction above does the following: it generates a sinusoidal wave of frequency 440 Hz with the instruction (sine 440), and then it multiplies this signal with an adsr envelope with attack, drop, sustain and release all set to 1. If you are like me and don’t have an idea of what an adsr envelope is, please have a look at this simple introduction to synthesizers: http://beausievers.com/synth/synthbasics/

There are different wave forms available in Fluxa: in addition to sine, there is squ (square), tri (triangle) and saw. There are also white and pink noise generators.
It doesn’t stop here: you can use a lot of filters, including various moog filters. Have a look at this page if you are interested: http://www.pawfal.org/fluxus/docs/0.17/en/fluxa.html.

All these ingredients are typically put together in a loop to play music. In fluxa, you use the following construct:

(seq
  (lambda (time clock)
    (when (zmod clock 2)
      (play time (mul (sine (mul 100 (pow (adsr 0 0.1 0.8 1) 6)))
       (adsr 0 0.5 0.4 5))))
  0.5)
)

This is a loop that is repeated every 0.5 second. The lambda function has two parameters: the absolute time (in seconds, since the start of Fluxus) and an integer counter called clock.
The function (zmod clock 2) is true when the value of the clock is a multiple of two, i.e., every other iteration. As each iteration lasts 0.5 seconds, the play instruction is repeated every second; notice that here we use play and not play-now. Play takes and additional time parameter to specify when it should be executed. The magic combination of mutlipliers and adsr envelopes above generates a heart beat (I have copied this from one of the example available in the source code).

You can use fluxa to learn everything you need to know (and a bit more) about modulo functions, synchronization, and reasoning about time. These are some of the examples I have created:

  • Close Encounters of the Third Kind (CETK): just five notes repeated with increasing BPM. You can download cetk.scm here.
  • The SOB beat: instead of using the seq mechanism of fluxa, I use every-frame and plot a graph together with a beat for each SOB, more or less (a SOB is a “Student Observable Behaviour”, a new mechanism of assessment that we use for our undergraduate students of Computer Science; each student is supposed to demonstrate a certain number of SOBs during the year, and we record these using a dedicated tool from which I have exported the observations so far). The graph is equivalent to the one visible from our SOB admin dashboard, with students on the x axis and number of observed SOBs on the y axis. For each student I play a number of beats equal to the number of SOBs. This result in an increasing heartbeat. You can download SOB-beat.scm here and you can also watch a screen capture on Youtube at http://www.youtube.com/watch?v=WWMEfdbk4Kw (you may need to increase your volume)
  • The SOB melody: this is very similar to the previous one, but instead of playing a beat I play a note from a scale. For each student I change the scale in a random way (only 2 scales for the moment). You can download SOB-melody.scm here and you can watch the corresponding screen capture (with some live comments) here: http://youtu.be/1dO1CSgfpBc
  • Finally, back to the seq mechanism and a few operations with modulo: I take the list of SOB id in the order in which they are observed and I play them back in a random scale (selected at the beginning). You can download The delicate sound of sobs here.

Feel free to play around with these files and let me know if you obtain something better!

Installing Fluxus on Mac OS X 10.6-10.8

kleineFluxus is a “a live coding environment for 3D graphics, music and game” available at http://www.pawfal.org/fluxus/. If you know Racket (or LISP, or Scheme) it will look immediately familiar. I wanted to install it for our new Computer Science degree, where we teach Racket, and I wanted to do some experiments with its music capabilties. However, the installation is not exactly user-friendly… I have tried the precompiled package available from the website but sound did not work for me (flexa was missing). In the end, I installed the source files and compiled them, and I managed to have the sound.

This is what I have done on a Mac OS X 10.6 (a 6 year-old Mac Pro) and Mac OS X 10.8 (a 3-week old Mac Book Pro). It is essentially what the README file in the source tree tells you to do; the problem is that you find a lot of websites giving you different suggestions, including the README file itself. For instance, the README file tells you that you can use Homebrew to install fluxus, but the homebrew installation is broken for the moment… So, here it goes:

  • Install the 32 bit version of Racket, dowloadable from http://racket-lang.org/download/. I have installed 5.3.6. Notice that you really need the 32 version: fluxus will not link against the 64-bit version.
  • Install Macports from https://www.macports.org/install.php. This is needed to install all the fluxus dependencies below.
  • Clone the fluxus source code: git clone git://git.savannah.nongnu.org/fluxus.git
  • Add the following two lines to your .bash_profile:
    PATH=$PATH:/opt/local/bin:/Applications/Racket\ v5.3.6/bin/
    export DYLD_FRAMEWORK_PATH=/Applications/Racket\ v5.3.6/lib/

    Make sure that the Racket version is correct, and change it appropriately if needed.The file .bash_profile shoud be in your home directory, typically something like /Users/franco/. If it does not exist, just create it and make sure to open a new terminal so that these variables are loaded. You can check this with the command echo $PATH from the command line.

  • Install all the dependencies using macport. From the command line, run:
    sudo port install fftw-3 +universal glew +universal freetype \
      +universal jpeg +universal liblo +universal libpng +universal \
      libsndfile +universal ode +universal tiff +universal zlib \ 
      +universal scons

    (all this should be on a single line. If not, remember to add the backslash at the end). This will take a bit of time, don’t worry and have a coffee or tea in the meanwhile.

  • Install JackOSX from http://www.jackosx.com/. I have installed version 0.90 beta 15. This requires to reboot your computer, do this now and have another coffee or tea in the meanwhile.
  • Go back to the directory where you have downloaded fluxus. In this directory, type:
    scons ADDONS=0

    to compile fluxus. Again, you will need to wait a bit (but not too much, not enough for another coffee or tea). Scons is yet another building system that was installed using macports a few lines above.

  • Finally, you can install fluxus with:
    sudo scons ADDONS=0 install

You are now ready to test the sytem. You need to start Jack and fluxa, and then you’ll be able to play some music. More in detail:

  • In Applications, look for the folder called Jack, open it and double click on JackPilot. Check that the configuration is correct, save it, and then start JackPilot. Wait a few seconds and it should be ready, giving you CPU load.
  • In a terminal, launch the command fluxa (it should be in your path). It should just say “fluxa server ready”
  • Finally, let’s try to play something. In a terminal, go to the source tree of Fluxus, enter the “examples” directory, and at the command line type fluxus sound.scm. Press ctrl+e (or F5) and enjoy…