EV3 Direct Commands: Lesson 7 - Multithreading, doing things parallel

EV3 Direct commands - Lesson 07

Introduction

Last lesson, we have heard of multitasking and multithreading, but we have not seen anything of it. This will be changed now. We create a second subclass of EV3 and name it Jukebox. This class plays tones and music, later we will add some light effects. In this lesson we use it as a playground for multithreading.

Then we will look at different aspects of multithreading. We will write little programs and we will get familiar with it.

Class Jukebox

We code a second subclass of EV3 with a design, that realizes the qualities, we formulated at the end of the last lesson:


#!/usr/bin/env python3

import ev3, time

TRIAS = {
    "tempo": 80,
    "tones": [
        ["c'", 1],
        ["e'", 1],
        ["g'", 1],
        ["c''",3],
    ]
}

HAPPY_BIRTHDAY = {
    "tempo": 100,
    "tones": [
        ["d'", 0.75],
        ["d'", 0.25],
        ["e'", 1],
        ["d'", 1],
        ["g'", 1],
        ["f#'", 2],
        ["d'", 0.75],
        ["d'", 0.25],
        ["e'", 1],
        ["d'", 1],
        ["a'", 1],
        ["g'", 2],
        ["d'", 0.75],
        ["d'", 0.25],
        ["d''", 1],
        ["b'", 1],
        ["g'", 1],
        ["f#'", 1],
        ["e'", 1],
        ["c''", 0.75],
        ["c''", 0.25],
        ["b'", 1],
        ["g'", 1],
        ["a'", 1],
        ["g'", 2]
    ]
}

class Jukebox(ev3.EV3):
    def __init__(self, protocol: str=None, host: str=None, ev3_obj=None):
        super().__init__(protocol=protocol, host=host, ev3_obj=ev3_obj)
        self._volume = 1
        self._temperament = 440
        self._pos_tone = None
        self._plays = False

    @property
    def volume(self):
        return self._volume
    @volume.setter
    def volume(self, value:int):
        self._volume = value

    @property
    def temperament(self):
        return self._temperament
    @temperament.setter
    def temperament(self, value:float):
        self._temperament = value

    def play_tone(self, tone: str, duration: float=0) -> None:
        volume = self._volume
        if tone == "p":
            self.stop()
            return
        elif tone.startswith("c"):
            freq = self._temperament * 2**(-9/12)
        elif tone.startswith("d"):
            freq = self._temperament * 2**(-7/12)
        elif tone.startswith("e"):
            freq = self._temperament * 2**(-5/12)
        elif tone.startswith("f"):
            freq = self._temperament * 2**(-4/12)
        elif tone.startswith("g"):
            freq = self._temperament * 2**(-2/12)
        elif tone.startswith("a"):
            freq = self._temperament
        elif tone.startswith("b"):
            freq = self._temperament * 2**(2/12)
        else:
            raise AttributeError('unknown Tone: ' + tone)

        if len(tone) > 1:
            if tone[1] == "#":
                freq *= 2**(1/12)
            elif tone[1] == "b":
                freq /= 2**(1/12)

        if tone.endswith("'''"):
            freq *= 4
        elif tone.endswith("''"):
            freq *= 2
        elif tone.endswith("'"):
            pass
        else:
            freq /= 2
        ops = b''.join([
            ev3.opSound,
            ev3.TONE,
            ev3.LCX(volume),
            ev3.LCX(round(freq)),
            ev3.LCX(round(1000*duration))
        ])
        self.send_direct_cmd(ops)

    def stop(self) -> None:
        self.send_direct_cmd(ev3.opSound + ev3.BREAK)
        self._plays = False

    def _init_tone(self) -> None:
        self._pos_tone = 0
        self._plays = True

    def _next_tone(self, song) -> float:
        if self._pos_tone == len(song["tones"]):
            return -1
        tone, beats = song["tones"][self._pos_tone]
        self.play_tone(tone)
        self._pos_tone += 1
        return 60 * beats / song["tempo"]

    def play_song(self, song:dict) -> None:
        self._init_tone()
        while self._plays:
            duration = self._next_tone(song)
            if duration == -1:
                break
            time.sleep(duration)
        if self._plays:
            self._plays = False
            self.stop()

Remarks:

You already know all the operations from lesson 2.
The frequencies of the tones are calculated in the 12 tone equal temperament and then rounded to integers.
The object attribute _plays is a flag, that signals, that actually a song is played. This class reacts correctly, if method stop is called, while a song is played. It stops playing.
Method play_song is time consuming but does not block the EV3 device. From this point of view, it behaves like methods drive_straight, drive_turn, rotate_to and drive_to of class TwoWheelVehicle.
The songs are defined as JSON objects, which is like a poor mans midi notation.
As all our classes, Jukebox is a layer of abstraction. This one encapsulates the playing of music.

The documentation of module ev3_sound:


Help on module ev3_sound:

NAME
    ev3_sound

CLASSES
    ev3.EV3(builtins.object)
        Jukebox
    
    class Jukebox(ev3.EV3)
     |  plays tones and songs
     |  
     |  Method resolution order:
     |      Jukebox
     |      ev3.EV3
     |      builtins.object
     |  
     |  Methods defined here:
     |  
     |  __init__(self, protocol:str=None, host:str=None, ev3_obj=None)
     |      Establish a connection to a LEGO EV3 device
     |      
     |      Keyword Arguments (either protocol and host or ev3_obj):
     |      protocol: None, 'Bluetooth', 'Usb' or 'Wifi'
     |      host: None or mac-address of the LEGO EV3 (f.i. '00:16:53:42:2B:99')
     |      ev3_obj: None or an existing EV3 object (its connections will be used)
     |  
     |  play_song(self, song:dict) -> None
     |      plays a song
     |      
     |      example:
     |      jukebox = ev3_sound.Jukebox(protocol=ev3.BLUETOOTH, host='00:16:53:42:2B:99')
     |      jukebox.play_song(ev3_sound.HAPPY_BIRTHDAY)
     |  
     |  play_tone(self, tone:str, duration:float=0) -> None
     |      plays a tone
     |      
     |      Attributes:
     |      tone: name of tone f.i. "c'", "cb''", "c#"
     |      
     |      Keyword Attributes:
     |      duration: length (sec.) of the tone (value 0 means forever)
     |  
     |  stop(self) -> None
     |      stops the sound
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors defined here:
     |  
     |  temperament
     |      temperament of the tones (delfault: 440 Hz)
     |  
     |  volume
     |      volume of sound [0 - 100] (default: 1)
     |  
     |  ----------------------------------------------------------------------
     |  Methods inherited from ev3.EV3:
     |  
     |  __del__(self)
     |      closes the connection to the LEGO EV3
     |  
     |  send_direct_cmd(self, ops:bytes, local_mem:int=0, global_mem:int=0) -> bytes
     |      Send a direct command to the LEGO EV3
     |      
     |      Arguments:
     |      ops: holds netto data only (operations), the following fields are added:
     |        length: 2 bytes, little endian
     |        counter: 2 bytes, little endian
     |        type: 1 byte, DIRECT_COMMAND_REPLY or DIRECT_COMMAND_NO_REPLY
     |        header: 2 bytes, holds sizes of local and global memory
     |      
     |      Keyword Arguments:
     |      local_mem: size of the local memory
     |      global_mem: size of the global memory
     |      
     |      Returns: 
     |        sync_mode is STD: reply (if global_mem > 0) or message counter
     |        sync_mode is ASYNC: message counter
     |        sync_mode is SYNC: reply of the LEGO EV3
     |  
     |  wait_for_reply(self, counter:bytes) -> bytes
     |      Ask the LEGO EV3 for a reply and wait until it is received
     |      
     |      Arguments:
     |      counter: is the message counter of the corresponding send_direct_cmd
     |      
     |      Returns:
     |      reply to the direct command
     |  
     |  ----------------------------------------------------------------------
     |  Data descriptors inherited from ev3.EV3:
     |  
     |  __dict__
     |      dictionary for instance variables (if defined)
     |  
     |  __weakref__
     |      list of weak references to the object (if defined)
     |  
     |  sync_mode
     |      sync mode (standard, asynchronous, synchronous)
     |      
     |      STD:   Use DIRECT_COMMAND_REPLY if global_mem > 0,
     |             wait for reply if there is one.
     |      ASYNC: Use DIRECT_COMMAND_REPLY if global_mem > 0,
     |             never wait for reply (it's the task of the calling program).
     |      SYNC:  Always use DIRECT_COMMAND_REPLY and wait for reply.
     |      
     |      The general idea is:
     |      ASYNC: Interruption or EV3 device queues direct commands,
     |             control directly comes back.
     |      SYNC:  EV3 device is blocked until direct command is finished,
     |             control comes back, when direct command is finished.               
     |      STD:   NO_REPLY like ASYNC with interruption or EV3 queuing,
     |             REPLY like SYNC, synchronicity of program and EV3 device.
     |  
     |  verbosity
     |      level of verbosity (prints on stdout).

DATA
    HAPPY_BIRTHDAY = {'tempo': 100, 'tones': [["d'", 0.75], ["d'", 0.25], ...
    TRIAS = {'tempo': 80, 'tones': [["c'", 1], ["e'", 1], ["g'", 1], ["c''"...

We write this little program to test it:


#!/usr/bin/env python3

import ev3, ev3_sound

jukebox = ev3_sound.Jukebox(protocol=ev3.BLUETOOTH, host='00:16:53:42:2B:99')
jukebox.verbosity = 1
jukebox.play_song(ev3_sound.HAPPY_BIRTHDAY)

Its output:


20:17:14.956954 Sent 0x|0C:00|2A:00|80|00:00|94:01:01:82:26:01:00|
20:17:15.408330 Sent 0x|0C:00|2B:00|80|00:00|94:01:01:82:26:01:00|
20:17:15.559332 Sent 0x|0C:00|2C:00|80|00:00|94:01:01:82:4A:01:00|
20:17:16.160783 Sent 0x|0C:00|2D:00|80|00:00|94:01:01:82:26:01:00|
20:17:16.762240 Sent 0x|0C:00|2E:00|80|00:00|94:01:01:82:88:01:00|
20:17:17.363682 Sent 0x|0C:00|2F:00|80|00:00|94:01:01:82:72:01:00|
20:17:18.565853 Sent 0x|0C:00|30:00|80|00:00|94:01:01:82:26:01:00|
20:17:19.017158 Sent 0x|0C:00|31:00|80|00:00|94:01:01:82:26:01:00|
20:17:19.168137 Sent 0x|0C:00|32:00|80|00:00|94:01:01:82:4A:01:00|
20:17:19.769569 Sent 0x|0C:00|33:00|80|00:00|94:01:01:82:26:01:00|
20:17:20.371090 Sent 0x|0C:00|34:00|80|00:00|94:01:01:82:B8:01:00|
20:17:20.972644 Sent 0x|0C:00|35:00|80|00:00|94:01:01:82:88:01:00|
20:17:22.174288 Sent 0x|0C:00|36:00|80|00:00|94:01:01:82:26:01:00|
20:17:22.625924 Sent 0x|0C:00|37:00|80|00:00|94:01:01:82:26:01:00|
20:17:22.777000 Sent 0x|0C:00|38:00|80|00:00|94:01:01:82:4B:02:00|
20:17:23.378498 Sent 0x|0C:00|39:00|80|00:00|94:01:01:82:EE:01:00|
20:17:23.980124 Sent 0x|0C:00|3A:00|80|00:00|94:01:01:82:88:01:00|
20:17:24.581646 Sent 0x|0C:00|3B:00|80|00:00|94:01:01:82:72:01:00|
20:17:25.183178 Sent 0x|0C:00|3C:00|80|00:00|94:01:01:82:4A:01:00|
20:17:25.784707 Sent 0x|0C:00|3D:00|80|00:00|94:01:01:82:0B:02:00|
20:17:26.236062 Sent 0x|0C:00|3E:00|80|00:00|94:01:01:82:0B:02:00|
20:17:26.387118 Sent 0x|0C:00|3F:00|80|00:00|94:01:01:82:EE:01:00|
20:17:26.988627 Sent 0x|0C:00|40:00|80|00:00|94:01:01:82:88:01:00|
20:17:27.590142 Sent 0x|0C:00|41:00|80|00:00|94:01:01:82:B8:01:00|
20:17:28.191678 Sent 0x|0C:00|42:00|80|00:00|94:01:01:82:88:01:00|
20:17:29.393731 Sent 0x|07:00|43:00|80|00:00|94:00|

Combining multiple tasks without multithreading

Whe combine driving and playing a song:


#!/usr/bin/env python3

import ev3, ev3_sound, ev3_vehicle

jukebox = ev3_sound.Jukebox(protocol=ev3.BLUETOOTH, host='00:16:53:42:2B:99')
vehicle = ev3_vehicle.TwoWheelVehicle(
    0.02128,                 # radius_wheel
    0.1175,                  # tread
    ev3_obj=jukebox
)
vehicle.drive_turn(25, 0.2)
jukebox.play_song(ev3_sound.HAPPY_BIRTHDAY)
vehicle.stop()

This program does two independent things, it plays a song, which uses EV3's sound resource and it drives the vehicle, which uses two motors. Two independent actions and no need of multithreading, why that? This is a combination of actions with different character:

When calling drive_turn without setting an angle, this is an unlimited action and control comes back directly.
play_song does the timing. Control comes back when the song is finished.
The call of method stop ends the unlimited movement by interruption. Control directly comes back.

This says, we can split our action in three parts:

Immediate return:
```
vehicle.drive_turn(25, 0.2)
 
```

Time consuming:


jukebox.play_song(ev3_sound.HAPPY_BIRTHDAY)

Immediate return:
```
vehicle.stop()
 
```

The timing is done by the time consuming parts of an action! Please take a look at the program The depressed giraffe of lesson 6 and identify the time consuming actions.

Let's come to a first conclusion. Executing multiple tasks does not necessarily need multithreading. Often it's possible to combine the actions in a sequence so that the correct timing is given and all tasks are done as desired. But this needs a clear understanding of the time consumption and the dependencies. The result is a sequence of actions, some of them are time consuming, others return immediately and are grouped around the time consuming actions, which do the timing.

This does not allow to run time consuming actions parallel! The program always waits until control is back.

Multithreading

Now we use multithreading to execute two independent actions. First we add some information to song HAPY_BIRTHDAY:


HAPPY_BIRTHDAY = {
    "tempo": 100,
    "beats_per_bar": 3,
    "upbeat": 1,
    "led_sequence": [ev3.LED_ORANGE, ev3.LED_GREEN, ev3.LED_RED, ev3.LED_GREEN],
    "tones": [
        ...

Then we run this program:


#!/usr/bin/env python3

import ev3, ev3_sound, threading, time

jukebox = ev3_sound.Jukebox(protocol=ev3.BLUETOOTH, host='00:16:53:42:2B:99')
jukebox.verbosity = 1

def change_color(led_pattern: bytes) -> None:
    ops = b''.join([
        ev3.opUI_Write,
        ev3.LED,
        led_pattern
    ])
    jukebox.send_direct_cmd(ops)

def colors(song: dict) -> None:
    if "upbeat" in song:
        time.sleep(60 * song["upbeat"] / song["tempo"])
    pos_led = 0
    while plays:
        change_color(song["led_sequence"][pos_led])
        pos_led += 1
        pos_led %= len(song["led_sequence"])
        time.sleep(60 * song["beats_per_bar"] / song["tempo"])
    change_color(ev3.LED_GREEN)

plays = True  
threading.Thread(
    target=colors,
    args=(ev3_sound.HAPPY_BIRTHDAY,)
).start()
jukebox.play_song(ev3_sound.HAPPY_BIRTHDAY)
plays = False

Remarks:

Both, function colors and method play_song are time consuming. Both run parallel, but play_song does the timing, it stops colors when the song is finished.
Class Thread allows to run any callable in its own thread. This says, control immediately comes back.
We call colors with an argument. If you want its return value, this needs some more logic, method start does not return it. The execution is asynchronous and handling return values is one of the drawbacks. We are lucky, there is no return value.
Please consult the documentation of module threading for details.

This programs output:


20:15:20.846839 Sent 0x|0C:00|2A:00|80|00:00|94:01:01:82:26:01:00|
20:15:21.298260 Sent 0x|0C:00|2B:00|80|00:00|94:01:01:82:26:01:00|
20:15:21.447336 Sent 0x|08:00|2C:00|80|00:00|82:1B:03|
20:15:21.449398 Sent 0x|0C:00|2D:00|80|00:00|94:01:01:82:4A:01:00|
20:15:22.050950 Sent 0x|0C:00|2E:00|80|00:00|94:01:01:82:26:01:00|
20:15:22.652346 Sent 0x|0C:00|2F:00|80|00:00|94:01:01:82:88:01:00|
20:15:23.249905 Sent 0x|08:00|30:00|80|00:00|82:1B:01|
20:15:23.253805 Sent 0x|0C:00|31:00|80|00:00|94:01:01:82:72:01:00|
20:15:24.455873 Sent 0x|0C:00|32:00|80|00:00|94:01:01:82:26:01:00|
20:15:24.907189 Sent 0x|0C:00|33:00|80|00:00|94:01:01:82:26:01:00|
20:15:25.052517 Sent 0x|08:00|34:00|80|00:00|82:1B:02|
20:15:25.058203 Sent 0x|0C:00|35:00|80|00:00|94:01:01:82:4A:01:00|
20:15:25.659172 Sent 0x|0C:00|36:00|80|00:00|94:01:01:82:26:01:00|
20:15:26.260541 Sent 0x|0C:00|37:00|80|00:00|94:01:01:82:B8:01:00|
20:15:26.855334 Sent 0x|08:00|38:00|80|00:00|82:1B:01|
20:15:26.862090 Sent 0x|0C:00|39:00|80|00:00|94:01:01:82:88:01:00|
20:15:28.064249 Sent 0x|0C:00|3A:00|80|00:00|94:01:01:82:26:01:00|
20:15:28.515525 Sent 0x|0C:00|3B:00|80|00:00|94:01:01:82:26:01:00|
20:15:28.657901 Sent 0x|08:00|3C:00|80|00:00|82:1B:03|
20:15:28.666590 Sent 0x|0C:00|3D:00|80|00:00|94:01:01:82:4B:02:00|
20:15:29.268142 Sent 0x|0C:00|3E:00|80|00:00|94:01:01:82:EE:01:00|
20:15:29.869902 Sent 0x|0C:00|3F:00|80|00:00|94:01:01:82:88:01:00|
20:15:30.460544 Sent 0x|08:00|40:00|80|00:00|82:1B:01|
20:15:30.471457 Sent 0x|0C:00|41:00|80|00:00|94:01:01:82:72:01:00|
20:15:31.072930 Sent 0x|0C:00|42:00|80|00:00|94:01:01:82:4A:01:00|
20:15:31.674404 Sent 0x|0C:00|43:00|80|00:00|94:01:01:82:0B:02:00|
20:15:32.125746 Sent 0x|0C:00|44:00|80|00:00|94:01:01:82:0B:02:00|
20:15:32.263208 Sent 0x|08:00|45:00|80|00:00|82:1B:02|
20:15:32.276589 Sent 0x|0C:00|46:00|80|00:00|94:01:01:82:EE:01:00|
20:15:32.878185 Sent 0x|0C:00|47:00|80|00:00|94:01:01:82:88:01:00|
20:15:33.479785 Sent 0x|0C:00|48:00|80|00:00|94:01:01:82:B8:01:00|
20:15:34.065854 Sent 0x|08:00|49:00|80|00:00|82:1B:01|
20:15:34.081116 Sent 0x|0C:00|4A:00|80|00:00|94:01:01:82:88:01:00|
20:15:35.282992 Sent 0x|07:00|4B:00|80|00:00|94:00|
20:15:35.868486 Sent 0x|08:00|4C:00|80|00:00|82:1B:01|

Both of them run independently but are thought to work synchronized. We take a closer look to the synchronization and realize, that the synchronization becomes worse. If we played a longer song, we could see and hear the growing time shift. Every command needs some time to execute. And these small durations add up. There are more tones than color changes, this makes that tones fall behind colors.

Class Jukebox with colors

We add colors to class Jukebox. This needs two more methods, change_color and _colors:


    def change_color(self, led_pattern: bytes) -> None:
        ops = b''.join([
            ev3.opUI_Write,
            ev3.LED,
            led_pattern
        ])
        self.send_direct_cmd(ops)

    def _colors(self, song: dict) -> None:
        if "upbeat" in song:
            time.sleep(60 * song["upbeat"] / song["tempo"])
        pos_led = 0
        while self._plays:
            self.change_color(song["led_sequence"][pos_led])
            pos_led += 1
            pos_led %= len(song["led_sequence"])
            time.sleep(60 * song["beats_per_bar"] / song["tempo"])
        self.change_color(ev3.LED_GREEN)

We modify method play_song:


    def play_song(self, song:dict) -> None:
        self._init_tone()
        threading.Thread(
            target=self._colors,
            args=(song,)
        ).start()
        while self._plays:
            duration = self._next_tone(song)
            if duration == -1:
                break
            time.sleep(duration)
        if self._plays:
            self._plays = False
            self.stop()

We test it:


#!/usr/bin/env python3

import ev3, ev3_sound

jukebox = ev3_sound.Jukebox(protocol=ev3.BLUETOOTH, host='00:16:53:42:2B:99')
jukebox.play_song(ev3_sound.HAPPY_BIRTHDAY)

We test the stopping:


#!/usr/bin/env python3

import ev3, ev3_sound, time, threading

jukebox = ev3_sound.Jukebox(protocol=ev3.BLUETOOTH, host='00:16:53:42:2B:99')
threading.Thread(
    target=jukebox.play_song,
    args=(ev3_sound.HAPPY_BIRTHDAY,)
).start()
time.sleep(5)
jukebox.stop()

This program runs three threads. One changes the colors, one the tones and the base thread stops them.

Exact timing

For a better timing we improve class Jukebox and modify method _colors:


    def _colors(self, song: dict) -> None:
        time_action = time.time()
        if "upbeat" in song:
            time_action += 60 * song["upbeat"] / song["tempo"]
            gap = time_action - time.time()
            if gap > 0:
                time.sleep(gap)
        pos_led = 0
        while self._plays:
            self.change_color(song["led_sequence"][pos_led])
            pos_led += 1
            pos_led %= len(song["led_sequence"])
            time_action += 60 * song["beats_per_bar"] / song["tempo"]
            gap = time_action - time.time()
            if gap > 0:
                time.sleep(gap)
        self.change_color(ev3.LED_GREEN)

and method play_song:


    def play_song(self, song:dict) -> None:
        self._init_tone()
        threading.Thread(
            target=self._colors,
            args=(song,)
        ).start()
        time_action = time.time()
        while self._plays:
            duration = self._next_tone(song)
            if duration == -1:
                break
            time_action += duration
            gap = time_action - time.time()
            if gap > 0:
                time.sleep(gap)
        if self._plays:
            self._plays = False
            self.stop()

we test it with this program:


#!/usr/bin/env python3

import ev3, ev3_sound

jukebox = ev3_sound.Jukebox(protocol=ev3.BLUETOOTH, host='00:16:53:42:2B:99')
jukebox.verbosity = 1
jukebox.play_song(ev3_sound.HAPPY_BIRTHDAY)

the output:


09:39:22.873872 Sent 0x|0C:00|2A:00|80|00:00|94:01:01:82:26:01:00|
09:39:23.324479 Sent 0x|0C:00|2B:00|80|00:00|94:01:01:82:26:01:00|
09:39:23.474193 Sent 0x|0C:00|2C:00|80|00:00|94:01:01:82:4A:01:00|
09:39:23.474699 Sent 0x|08:00|2D:00|80|00:00|82:1B:03|
09:39:24.074629 Sent 0x|0C:00|2E:00|80|00:00|94:01:01:82:26:01:00|
09:39:24.674645 Sent 0x|0C:00|2F:00|80|00:00|94:01:01:82:88:01:00|
09:39:25.274656 Sent 0x|0C:00|30:00|80|00:00|94:01:01:82:72:01:00|
09:39:25.275207 Sent 0x|08:00|31:00|80|00:00|82:1B:01|
09:39:26.475232 Sent 0x|0C:00|32:00|80|00:00|94:01:01:82:26:01:00|
09:39:26.924492 Sent 0x|0C:00|33:00|80|00:00|94:01:01:82:26:01:00|
09:39:27.074212 Sent 0x|0C:00|34:00|80|00:00|94:01:01:82:4A:01:00|
09:39:27.074697 Sent 0x|08:00|35:00|80|00:00|82:1B:02|
09:39:27.674645 Sent 0x|0C:00|36:00|80|00:00|94:01:01:82:26:01:00|
09:39:28.274532 Sent 0x|0C:00|37:00|80|00:00|94:01:01:82:B8:01:00|
09:39:28.874644 Sent 0x|0C:00|38:00|80|00:00|94:01:01:82:88:01:00|
09:39:28.875532 Sent 0x|08:00|39:00|80|00:00|82:1B:01|
09:39:30.075237 Sent 0x|0C:00|3A:00|80|00:00|94:01:01:82:26:01:00|
09:39:30.524485 Sent 0x|0C:00|3B:00|80|00:00|94:01:01:82:26:01:00|
09:39:30.674126 Sent 0x|0C:00|3C:00|80|00:00|94:01:01:82:4B:02:00|
09:39:30.674611 Sent 0x|08:00|3D:00|80|00:00|82:1B:03|
09:39:31.274652 Sent 0x|0C:00|3E:00|80|00:00|94:01:01:82:EE:01:00|
09:39:31.874653 Sent 0x|0C:00|3F:00|80|00:00|94:01:01:82:88:01:00|
09:39:32.474641 Sent 0x|0C:00|40:00|80|00:00|94:01:01:82:72:01:00|
09:39:32.475321 Sent 0x|08:00|41:00|80|00:00|82:1B:01|
09:39:33.074650 Sent 0x|0C:00|42:00|80|00:00|94:01:01:82:4A:01:00|
09:39:33.674640 Sent 0x|0C:00|43:00|80|00:00|94:01:01:82:0B:02:00|
09:39:34.124485 Sent 0x|0C:00|44:00|80|00:00|94:01:01:82:0B:02:00|
09:39:34.274186 Sent 0x|0C:00|45:00|80|00:00|94:01:01:82:EE:01:00|
09:39:34.274928 Sent 0x|08:00|46:00|80|00:00|82:1B:02|
09:39:34.874565 Sent 0x|0C:00|47:00|80|00:00|94:01:01:82:88:01:00|
09:39:35.474633 Sent 0x|0C:00|48:00|80|00:00|94:01:01:82:B8:01:00|
09:39:36.074662 Sent 0x|0C:00|49:00|80|00:00|94:01:01:82:88:01:00|
09:39:36.075217 Sent 0x|08:00|4A:00|80|00:00|82:1B:01|
09:39:37.275170 Sent 0x|07:00|4B:00|80|00:00|94:00|
09:39:37.875666 Sent 0x|08:00|4C:00|80|00:00|82:1B:01|

This solved the problem! We changed from netto to brutto timing. Now the time distances include the time for execution.

Communication between threads

The base thread and all threads it starts, use the same global data. This allows communication. Every thread can read data of another one. The communication is asynchronous, which is no problem, we are used to asynchronous communication. Think of mails or emails. We already have seen it working. Attribute _plays was used for communication between different threads. If one thread sets this flag, the others read it and react.

Locking

The communication between threads sometimes shows unexpected results. Let's look at a snippet of code, that runs in its own thread. This thread uses a global variable state to tell its actual state to the rest of the world:


STATE_TO_STOP = "TO_STOP"
STATE_STOPPED = "STOPPED"
STATE_FINISHED = "FINISHED"

def finished_or_stopped():
    global state
    if state == STATE_TO_STOP:
        state = STATE_STOPPED
    if state == STATE_STOPPED and not next:
        state = STATE_FINISHED

state = STATE_TO_STOP
next = False
finished_or_stopped()

We expect, that there are only two combinations of state and next:

Before the function is called: state == STATE_TO_STOP and next == False
After the call of the function: state == STATE_FINISHED and next == False

The experience will show, this is correct in about 99.9999 % of all situations, but not really for all. It may happen, that a foreign thread asks about variable state just after it was changed to value STATE_STOPPED. This seldom case shows a new combination:

While the function is executed: state == STATE_STOPPED and next == False

The locking mechanism prevents this. We change our function to:


def finished_or_stopped():
    global state
    lock.acquire()
    if state == STATE_TO_STOP:
        state = STATE_STOPPED
    if state == STATE_STOPPED and not next:
        state = STATE_FINISHED
    lock.release()

The lock object was created by

lock =
      threading.Lock()

and the foreign thread, which asks about the state also must know and use it:


    lock.acquire()
    if state == STATE_STOPPED:
        print("This never happens")
    lock.release()

The lock object guaranties, that a second call of method acquire will wait until method release was called (maybe by anyone else). This says either function finished_or_stopped has to wait until the foreign thread has finished its if statement or the foreign thread has to wait until the change of valiable state in function finished_or_stopped is done.

Locking is a very common thechnique. Databases use it to prevent concurrent updates of the same data. Operating systems use locking to manage the usage of hardware resources and so on. But it's never fun to code it. If one forgets a single call of method release, the resource is blocked forever. The good news is, that encapsulation allows to do all this behind the scene. This says, the execution (f.i. usage of the resources) is done through well defined methods, which implement the locking mechanism.

Error handling

Another drawback of asynchronous processing is error handling. If an error is thrown inside a thread, this will not reach the other threads (and not the base thread). Let' look at an example:


#!/usr/bin/env python3

import ev3, ev3_sound, time, threading

jukebox = ev3_sound.Jukebox(protocol=ev3.BLUETOOTH, host='00:16:53:42:2B:99')
jukebox.verbosity = 1

def tone(intervall) -> None:
    time_action = time.time()
    for i in range(4):
        jukebox.play_tone("c", 0.1)
        time.sleep(0.5)
        jukebox.play_tone("c'", 0.05)
        time.sleep(0.5)
        jukebox.play_tone("c'", 0.05)
        time.sleep(0.5)
        jukebox.play_tone("c'", 0.05)
        time_action += intervall
        gap = time_action - time.time()
        time.sleep(max(0, gap))
    jukebox.stop()

def led(intervall) -> None:
    time_action = time.time()
    for i in range(2):
        jukebox.change_color(ev3.LED_RED)
        time.sleep(2)
        raise Exception('Something happened')
        jukebox.change_color(ev3.LED_GREEN)
        time_action += intervall
        gap = time_action - time.time()
        time.sleep(max(0, gap))

threading.Thread(target=led, args=(4,)).start()
tone(2)

its output:


08:40:18.292905 Sent 0x|08:00|2A:00|80|00:00|82:1B:02|
08:40:18.293778 Sent 0x|0D:00|2B:00|80|00:00|94:01:01:82:83:00:81:64|
08:40:18.795806 Sent 0x|0D:00|2C:00|80|00:00|94:01:01:82:06:01:81:32|
08:40:19.297941 Sent 0x|0D:00|2D:00|80|00:00|94:01:01:82:06:01:81:32|
08:40:19.800143 Sent 0x|0D:00|2E:00|80|00:00|94:01:01:82:06:01:81:32|
08:40:20.294214 Sent 0x|0D:00|2F:00|80|00:00|94:01:01:82:83:00:81:64|
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.4/threading.py", line 868, in run
    self._target(*self._args, **self._kwargs)
  File "./test_01.py", line 28, in led
    raise Exception('Something happened')
Exception: Something happend

08:40:20.796906 Sent 0x|0D:00|30:00|80|00:00|94:01:01:82:06:01:81:32|
08:40:21.299067 Sent 0x|0D:00|31:00|80|00:00|94:01:01:82:06:01:81:32|
08:40:21.801280 Sent 0x|0D:00|32:00|80|00:00|94:01:01:82:06:01:81:32|
08:40:22.294128 Sent 0x|0D:00|33:00|80|00:00|94:01:01:82:83:00:81:64|
08:40:22.796186 Sent 0x|0D:00|34:00|80|00:00|94:01:01:82:06:01:81:32|
08:40:23.298252 Sent 0x|0D:00|35:00|80|00:00|94:01:01:82:06:01:81:32|
08:40:23.800432 Sent 0x|0D:00|36:00|80|00:00|94:01:01:82:06:01:81:32|
08:40:24.294131 Sent 0x|0D:00|37:00|80|00:00|94:01:01:82:83:00:81:64|
08:40:24.796197 Sent 0x|0D:00|38:00|80|00:00|94:01:01:82:06:01:81:32|
08:40:25.298273 Sent 0x|0D:00|39:00|80|00:00|94:01:01:82:06:01:81:32|
08:40:25.800424 Sent 0x|0D:00|3A:00|80|00:00|94:01:01:82:06:01:81:32|
08:40:26.294006 Sent 0x|07:00|3B:00|80|00:00|94:00|

The exception was handled inside Thread-1. The base thread did not recognize the exception. Maybe this is what we want, maybe not. I prefer a hard stop of everything as the default reaction. Error handling is communication, we add a variable error:


#!/usr/bin/env python3

import ev3, ev3_sound, time, threading, traceback, sys

jukebox = ev3_sound.Jukebox(protocol=ev3.BLUETOOTH, host='00:16:53:42:2B:99')
jukebox.verbosity = 1
error = False

def tone(intervall) -> None:
    global error
    time_action = time.time()
    for i in range(4):
        if error:
            sys.exit()
        jukebox.play_tone("c", 0.1)
        time.sleep(0.5)
        jukebox.play_tone("c'", 0.05)
        time.sleep(0.5)
        jukebox.play_tone("c'", 0.05)
        time.sleep(0.5)
        jukebox.play_tone("c'", 0.05)
        time_action += intervall
        gap = time_action - time.time()
        time.sleep(max(0, gap))
    jukebox.stop()

def led(intervall) -> None:
    global error
    try:
        time_action = time.time()
        for i in range(2):
            jukebox.change_color(ev3.LED_RED)
            time.sleep(2)
            raise Exception('Something happened')
            jukebox.change_color(ev3.LED_GREEN)
            time_action += intervall
            gap = time_action - time.time()
            time.sleep(max(0, gap))
    except Exception:
        error = True
        raise

threading.Thread(target=led, args=(4,)).start()
tone(2)

the output:


09:05:21.509790 Sent 0x|08:00|2A:00|80|00:00|82:1B:02|
09:05:21.510744 Sent 0x|0D:00|2B:00|80|00:00|94:01:01:82:83:00:81:64|
09:05:22.012858 Sent 0x|0D:00|2C:00|80|00:00|94:01:01:82:06:01:81:32|
09:05:22.515052 Sent 0x|0D:00|2D:00|80|00:00|94:01:01:82:06:01:81:32|
09:05:23.017140 Sent 0x|0D:00|2E:00|80|00:00|94:01:01:82:06:01:81:32|
09:05:23.511133 Sent 0x|0D:00|2F:00|80|00:00|94:01:01:82:83:00:81:64|
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.4/threading.py", line 920, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.4/threading.py", line 868, in run
    self._target(*self._args, **self._kwargs)
  File "./test_01.py", line 31, in led
    raise Exception('Something happened')
Exception: Something happened

09:05:24.013019 Sent 0x|0D:00|30:00|80|00:00|94:01:01:82:06:01:81:32|
09:05:24.516006 Sent 0x|0D:00|31:00|80|00:00|94:01:01:82:06:01:81:32|
09:05:25.018296 Sent 0x|0D:00|32:00|80|00:00|94:01:01:82:06:01:81:32|

Multiplying the code in function tone would prevent the last three beats. Fact is, that function tone has to ask if an error occured, there is no automatic mechanism. This is asynchronous communication.

Coordinating actions of unknown duration

Sometimes, you have a number of parallel actions, but there is no clear responsibility for the timing. You want your program to wait until all of the actions are finished. This also can be solved with multithreading. A Thread object has a join method, which waits until the thread is finished:


#!/usr/bin/env python3

import ev3, ev3_sound, time, datetime, threading

jukebox = ev3_sound.Jukebox(protocol=ev3.BLUETOOTH, host='00:16:53:42:2B:99')
jukebox.verbosity = 1

def tone() -> None:
    global jukebox
    jukebox.play_tone("c'", 0.3)
    time.sleep(1)

def led() -> None:
    global jukebox
    jukebox.change_color(ev3.LED_RED)
    time.sleep(2)
    jukebox.change_color(ev3.LED_GREEN)

t1 = threading.Thread(target=tone)
t2 = threading.Thread(target=led)
t1.start()
t2.start()
t1.join()
t2.join()
now = datetime.datetime.now().strftime('%H:%M:%S.%f')
print(now, "all done")

The output:


11:19:37.801899 Sent 0x|0E:00|2A:00|80|00:00|94:01:01:82:06:01:82:2C:01|
11:19:37.803362 Sent 0x|08:00|2B:00|80|00:00|82:1B:02|
11:19:39.808145 Sent 0x|08:00|2C:00|80|00:00|82:1B:01|
11:19:39.808878 all done

Events

Events allow that one thread signals an event and other threads wait on it. Here is an example:


#!/usr/bin/env python3

import ev3, ev3_sound, threading, time

jukebox = ev3_sound.Jukebox(protocol=ev3.BLUETOOTH, host='00:16:53:42:2B:99')
jukebox.verbosity = 1

lock = threading.Lock()

def task1():
    jukebox.play_tone("c", 0.1)
    lock.acquire()
    lock.release()
    jukebox.play_tone("c", 0.1)
    
def task2():
    jukebox.change_color(ev3.LED_RED)
    lock.acquire()
    lock.release()
    jukebox.change_color(ev3.LED_GREEN)

lock.acquire()
threading.Thread(target=task1).start()
threading.Thread(target=task2).start()
time.sleep(5)
lock.release()

The base thread and the threads of task1 and task2 all use the same Lock object lock. The event is signaled by the base threads call lock.release(). This allows both tasks to continue their work.

This is a very common case and the above presented code is hard to read. It is a common practice to use class Event instead, which is syntactic sugar, but reads easier. We change the program:


#!/usr/bin/env python3

import ev3, ev3_sound, threading, time

jukebox = ev3_sound.Jukebox(protocol=ev3.BLUETOOTH, host='00:16:53:42:2B:99')
jukebox.verbosity = 1

event = threading.Event()

def task1():
    jukebox.play_tone("c", 0.1)
    event.wait()
    jukebox.play_tone("c", 0.1)
    
def task2():
    jukebox.change_color(ev3.LED_RED)
    event.wait()
    jukebox.change_color(ev3.LED_GREEN)

threading.Thread(target=task1).start()
threading.Thread(target=task2).start()
time.sleep(5)
event.set()

Both versions produce the same output:


09:47:08.032849 Sent 0x|0D:00|2A:00|80|00:00|94:01:01:82:83:00:81:64|
09:47:08.034327 Sent 0x|08:00|2B:00|80|00:00|82:1B:02|
09:47:13.040731 Sent 0x|0D:00|2C:00|80|00:00|94:01:01:82:83:00:81:64|
09:47:13.041566 Sent 0x|08:00|2D:00|80|00:00|82:1B:01|

Timers

There is a special subclass of Thread, that allows to start a thread after some waiting time. Here an example:


#!/usr/bin/env python3

import ev3, ev3_sound, threading, time

jukebox = ev3_sound.Jukebox(protocol=ev3.BLUETOOTH, host='00:16:53:42:2B:99')
jukebox.verbosity = 1

def task():
    jukebox.play_tone("c", 0.1)

task()
threading.Timer(5, task).start()

The ouput:


09:56:16.543277 Sent 0x|0D:00|2A:00|80|00:00|94:01:01:82:83:00:81:64|
09:56:21.545055 Sent 0x|0D:00|2B:00|80|00:00|94:01:01:82:83:00:81:64|

When a Timer is still waiting, it can be canceled:


#!/usr/bin/env python3

import ev3, ev3_sound, threading, time

jukebox = ev3_sound.Jukebox(protocol=ev3.BLUETOOTH, host='00:16:53:42:2B:99')
jukebox.verbosity = 1

def task():
    jukebox.play_tone("c", 0.1)

task()
timer = threading.Timer(5, task)
timer.start()
time.sleep(2)
timer.cancel()

Its output:


09:59:56.989692 Sent 0x|0D:00|2A:00|80|00:00|94:01:01:82:83:00:81:64|

The second call of task never took place, because the Timer was cancelled while it was waiting.

Interruptable Sleeping

We use class Condition to code an interruptable sleeper:


#!/usr/bin/env python3

import threading, time, datetime
lock = threading.Lock()
cond = threading.Condition(lock)

def task():
    lock.acquire()
    now = datetime.datetime.now().strftime('%H:%M:%S.%f')
    print(now, "task started")
    cond.wait(3)
    now = datetime.datetime.now().strftime('%H:%M:%S.%f')
    print(now, "task ended")
    lock.release()
    

threading.Thread(target=task).start()
time.sleep(1)
lock.acquire()
now = datetime.datetime.now().strftime('%H:%M:%S.%f')
print(now, "notify task1")
cond.notify()
time.sleep(1)
now = datetime.datetime.now().strftime('%H:%M:%S.%f')
print(now, "lock will be released")
lock.release()

The output:


10:10:32.073769 task started
10:10:33.075199 notify task1
10:10:34.076750 lock will be released
10:10:34.077305 task ended

Remarks:

The base thread interrupts the sleeping of function task, which runs in its own thread. This is possible because both threads use the same Condition object.
Every condition is bound to a lock.
Method wait implicitly releases the lock, which allows another thread to acquire it.
The base thread calls method notify which wakes up the waiting thread task.
When notified, task tries to acquire the lock.
Our example, where the base thread holds the lock for another sec. after notifying task is unusual but demonstrates the role of the lock.
If no notification takes place, cond.wait(3) waits for three sec. and meanwhile releases the lock.

Modify class EV3

It needs some modifications of class EV3 to prepare it for parallel execution of multiple tasks.

Locking

For the moment, the message counter is the only class attribute. It is the common identity of a direct command and its reply. If we use multiple instances of class EV3 parallel, we need a locking mechanism when changing class attributes.


class EV3:
    _msg_cnt = 41
    _lock = threading.Lock()

and we modify method _complete_direct_cmd:


    def _complete_direct_cmd(self, ops:bytes,
                             local_mem:int,
                             global_mem:int) -> bytes:
        if global_mem > 0  or self._sync_mode == SYNC:
            cmd_type = _DIRECT_COMMAND_REPLY
        else:
            cmd_type = _DIRECT_COMMAND_NO_REPLY
        self._lock.acquire()
        if self._msg_cnt < 65535:
            self._msg_cnt += 1
        else:
            self._msg_cnt = 1
        msg_cnt = self._msg_cnt
        self._lock.release()
        return b''.join([
            struct.pack('<hh', len(ops) + 5, msg_cnt),
            cmd_type,
            struct.pack('<h', local_mem * 1024 + global_mem),
            ops
        ])

This guaranties, that the message counters are distinct until they are reused after 65.535 direct commands.

Foreign replies

Sometimes it may happen, that two direct commands (both with reply) do not hold the sequence: send cmd_1, receive reply_1, send cmd_2, receive reply_2. Instead we see the sequence: send cmd_1, send cmd_2, receive reply_1, receive reply_2. We don't prevent that because we have independent parallel tasks and we want the communication as fast as possible. This says task_1 may send cmd_1, but get reply_2:


12:15:23.903970 Sent 0x|15:00|1C:02|00|08:00|99:1C:00:13:07:01:01:60:99:1C:00:10:07:00:01:64|
12:15:23.910924 Sent 0x|0E:00|1D:02|00|04:00|99:1C:00:00:81:21:00:01:60|
12:15:23.953569 Recv 0x|0B:00|1C:02|02|98:70:00:00:4A:78:00:00|
12:15:23.954865 Recv 0x|07:00|1D:02|02|0D:00:00:00|

We solve this problem with a dictionary of foreign replies and add another class attribute to class EV3:


class EV3:
    _msg_cnt = 41
    _lock = threading.Lock()
    _foreign = {}

We add two protected methods:


    def _put_foreign_reply(self, counter: bytes, reply: bytes) -> None:
        if counter in self._foreign:
            raise ValueError('reply with counter ' + counter + ' already exists')
        else:
            self._foreign[counter] = reply

    def _get_foreign_reply(self, counter: bytes) -> bytes:
        if counter in self._foreign:
            reply = self._foreign[counter]
            del self._foreign[counter]
            return reply
        else:
            return None

The first adds a reply to the dictionary with its counter as key. The second looks, if the dictionary contains a reply with a given key. If so, it returns the reply and deletes it from the dictionary. We add some code to method wait_for_reply:


    def wait_for_reply(self, counter: bytes) -> bytes:
        self._lock.acquire()
        reply = self._get_foreign_reply(counter)
        if reply:
            self._lock.release()
            if reply[4:5] != _DIRECT_REPLY:
                raise DirCmdError(
                    "direct command {:02X}:{:02X} replied error".format(
                        reply[2],
                        reply[3]
                    )
                )
            return reply
        while True:
            if self._protocol in [BLUETOOTH, WIFI]:
                reply = self._socket.recv(1024)
            else:
                reply = bytes(self._device.read(EP_IN, 1024, 0))
            len_data = struct.unpack('<H', reply[:2])[0] + 2
            reply_counter = reply[2:4]
            if self._verbosity >= 1:
                ...
            if counter != reply_counter:
                self._put_foreign_reply(reply_counter, reply[:len_data])
            else:
                self._lock.release()
                if reply[4:5] != _DIRECT_REPLY:
                    raise DirCmdError(
                        "direct command {:02X}:{:02X} replied error".format(
                            reply[2],
                            reply[3]
                        )
                    )
                return reply[:len_data]

The logic:

It first looks, if the reply is already in the dictionary. If so, it does not communicate with the EV3 device.
If not, it reads reply for reply until it gets the one it looks for. All foreign replies are put into the dictionary.
The standard situation is an empty dictionary.
The locking guaranties an exclusive access to the dictionary.

The dancing robot

This lesson ends with a program, that combines three independent actions:


#!/usr/bin/env python3

import ev3, ev3_sound, ev3_vehicle, threading, time

jukebox = ev3_sound.Jukebox(protocol=ev3.BLUETOOTH, host='00:16:53:42:2B:99')
jukebox.volume = 5
vehicle = ev3_vehicle.TwoWheelVehicle(0.02128, 0.1175, ev3_obj=jukebox)

def drive(song):
    time_action = time.time()
    if "upbeat" in song:
        duration = 60 * song["upbeat"] / song["tempo"]
        time_action += duration
        gap = time_action - time.time()
        time.sleep(max(0, gap))
    duration = 2 * 60 * song["beats_per_bar"] / song["tempo"]
    while driving:
        vehicle.drive_turn(speed, 0.2)
        time_action += duration
        gap = time_action - time.time()
        time.sleep(max(0, gap))
        if not driving: break
        vehicle.drive_turn(-speed, -0.2)
        time_action += duration
        gap = time_action - time.time()
        time.sleep(max(0, gap))
    vehicle.stop()

song = ev3_sound.HAPPY_BIRTHDAY
speed = 30
driving = True
threading.Thread(
    target=drive,
    args=(song,)
).start()
jukebox.play_song(song)
driving = False

These are three parallel actions, tones, colors, movements. All of them have their own timing. The coordination results from the ratios of the timings, which all are determined by the rhythm of the music. Movement changes every second bar, color changes per bar and the tones fit into bars.

Conclusion

This lesson layed the foundations of multitasking. We coded a sublass of EV3: Jukebox, which uses multithreading. We have done a sightseeing tour, that showed us a number of aspects, we need to take into account. We were no passive visitors, no we wrote little programs and got familiar with multitasking and multithreading. We learned, that the control of time is an important aspect.

We modified class EV3 to prepare it for multiple concurrent tasks. This needed a locking mechanism and some common resources.

Next lesson we will realize some tools, that help to organize and handle multiple tasks. Here is a first specification:

We want to start, stop and continue tasks.
We want an easy API for repeated and periodic tasks with exact timing.
We want to organize tasks as chains of tasks.
We need some help for locking and error handling.

EV3 Direct Commands

Friday, 29 April 2016

Lesson 7 - Multithreading, doing things parallel

Introduction

Class Jukebox

Combining multiple tasks without multithreading

Multithreading

Class Jukebox with colors

Exact timing

Communication between threads

Locking

Error handling

Coordinating actions of unknown duration

Events

Timers

Interruptable Sleeping

Modify class EV3

Locking

Foreign replies

The dancing robot

Conclusion

1 comment: