Why is controlling Pis so difficult?

As discussed earlier, the game is run by a distributed network with one main client PC and multiple Raspberry Pi servers. It would be very painful to have to SSH manually into each Raspberry Pi to start their respective programs each time we wanted to initiate the game. Automating this process would prove to be very useful in the future as we continue developing the game.

There is one very obvious way to handle this. Raspberry Pis can be told to start with a specific script on boot. This is fine, but there is very little feedback from the Pis to know if it was even successful or not. I thought we could do this in a much better way. And this was an adventure that I did not plan on going on.

There were many considerations. It would be impractical to have the main PC maintain an SSH terminal for every Pi if we already have a low-level socket communication system. So that means we can't just SSH and call the python program from that terminal since we want to disconnect after it boots the program. The obvious choice here was to use some terminal emulator on the Raspberry Pi to "spawn" a new terminal through SSH that could remain open when the SSH connection is disconnected. This could be easily encompassed into a bash script on the Raspberry Pi that we could then call from the PC. Famous last words.

I wanted to have a check that the pygame outputs "Server is listening" to show that it booted correctly. lxterminal on Linux ( the terminal emulator that Pi comes with ) does not pipe output easily. Thus we would need an output log to store all the pygame prints. After juggling with a million commands, I found this command which would open up a new terminal, run the script, and log the output.

lxterminal -e sh -c ". venv/bin/activate; python3 $pygame_script > $log_file 2>&1"

Now on the PC side. Using Putty, I was able to call the script file but nothing opened up on the Pi. We need the pygame GUI for the back wall game so this wouldn't suffice. Turns out we need to specify the DISPLAY to export to. Fixing that:

lxterminal -e sh -c "export DISPLAY=:0; . venv/bin/activate; python3 $pygame_script > $log_file 2>&1"

Now it works through Putty. In Python on the client code, I used Paramiko to initiate the SSH connection to run the script. All should be fine. But, it failed every time. SSH has this X11 Forwarding feature that needs to be used to allow for GUI programs to output from an SSH terminal. Paramiko does not support it without some serious work.

I was pretty sick of Python's SSH and shell packages at this point because everything just was not working. But I knew that PowerShell on Windows would work.

Using Window's default SSH client, you can just use the

-X

command to allow X11 Forwarding. A second script was written for the PC that would SSH to the Pi, do all the X11 Forwarding and check that the other script has outputted "Success".

$sshCommand = "ssh -F $env:USERPROFILE\.ssh\config -X -t $sshConnectionName 'export DISPLAY=:0; cd $remoteDirectory; nohup ./pygame_run.sh'"

Now, Python's only job is just to call this file and check for "Success". I'm glad it could handle it. Now that this works, I added some kill commands for the terminal process on the Pi side to stop multiple instances of the code from running. Everything can finally be initiated and controlled from the main PC, no more manually starting programs on the Pis!