Building a wireless remote volume control with a rotary knob for Raspberry pi - help needed with Python

[mdsimon2]

Maybe trying running the script with sudo? I've seen issues where pycamilladsp is installed with sudo and you attempt to run it as a normal user.

EDIT: Other potential issue is if you are running camillaDSP V2 the pycamilladsp nomenclature has completely changed.

Michael

 

[changer]

so from the screenshot in my second last tweet, the location of this install is:
/usr/local/camillagui/environment/lib/python3.8/site-packages/camilladsp/camilladsp.py
does this play a role, where it is located at?

Edit: git says it’s CamillaDSP 1.0.3

 

[mdsimon2]

Hmm...mine is installed in /usr/local/lib/python3.11/dist-packages/camilladsp/camilladsp.py. Looking at install_cdsp.py the pycamilladsp installation it looks like a bit custom.

### Install CamillaGUI

install_if_missing python3.8
install_temporarily_if_missing python3.8-pip
$use32bit && install_temporarily_if_missing python3.8-dev
sudo mkdir -m 775 /usr/local/camillagui
sudo chown root:staff /usr/local/camillagui
cd /usr/local/camillagui
python3 -m venv environment
(tr -d '\r' < environment/bin/activate) > environment/bin/activate_new # Create fixed version of the activate script. See https://stackoverflow.com/a/44446239
mv -f environment/bin/activate_new environment/bin/activate
source environment/bin/activate # activate custom python environment
python3 -m pip install --upgrade pip
pip install websocket_client aiohttp jsonschema setuptools
pip install git+https://github.com/HEnquist/[email protected]
pip install git+https://github.com/HEnquist/[email protected]
deactivate # deactivate custom python environment
wget https://github.com/HEnquist/camillagui-backend/releases/download/v1.0.1/camillagui.zip
unzip camillagui.zip
rm -f camillagui.zip
echo '
---
camilla_host: "127.0.0.1"
camilla_port: 1234
port: 5000
config_dir: "/mnt/mmcblk0p2/tce/camilladsp/configs"
coeff_dir: "/mnt/mmcblk0p2/tce/camilladsp/coeffs"
default_config: "/mnt/mmcblk0p2/tce/camilladsp/default_config.yml"
active_config: "/mnt/mmcblk0p2/tce/camilladsp/active_config"
active_config_txt: "/mnt/mmcblk0p2/tce/camilladsp/active_config.txt"
log_file: "/tmp/camilladsp.log"
update_config_symlink: true
update_config_txt: false
on_set_active_config: null
on_get_active_config: null
supported_capture_types: ["Stdin"]
supported_playback_types: ["Alsa"]
' > config/camillagui.yml

Complete shot in the dark but maybe try running the script out of /usr/loca/camillagui/? I assume your GUI works correctly? If so, it uses the same "from camilladsp import CamillaClient" but is located in /usr/local/camillagui/.

Michael

 

[somebodyelse]

I think the issue might be that the script installs pycamilladsp inside a virtual environment. Before running something that uses the virtual environment you need to activate it:

source /usr/local/camillagui/environment/bin/activate

You can see an example of usage in the usr/local/tce.installed/piCoreCDSP script it creates (line 207 of the install script.)

 

[changer]

Thanks.

None of the suggestions work, and I am waiting for a reply of the creator of the approach I was using https://github.com/JWahle/piCoreCDSP

Maybe he has the time to explain how to do get things going. If not, I might need to find out how to installt CamillaDSP on TinyCore a different way.

Edit: Some things I cannot implement currently. Is it necessary to have zigbee2mqtt running as a daemon for this approach? The guide is not specific about this. On TinyCore, there is no systemctl so I cannot create *.service.

 

[changer]

I managed to run tonecontrol.py (MCH’s script) from within the environment, beyond line 5:

- I started zigbee2mqtt bridge from within the environment in a separate window:

source /usr/local/camillagui/environment/bin/activate
cd /home/tc/zigbee2mqtt
npm start

And paired the knob anew: It happily logs commands:

Zigbee2MQTT:info 2023-09-19 08:41:51: MQTT publish: topic 'zigbee2mqtt/0xdc8e95fffeb8da70', payload '{"action":"brightness_step_up","action_step_size":13,"action_transition_time":0.01,"battery":100,"linkquality":23,"operation_mode":"command","voltage":3000}'

- The mosquito broker was downloaded as an extension in TinyCore from and installed:

tinycorelinux.net/13.x/armv7/tcz/mosquitto.tcz

I placed a config file with the following lines in /home/tc/mosquitto.conf

allow_anonymous true
listener 1883

And set the config file:

mosquitto -c /home/tc/mosquitto.conf

If I check, it still presents the link to config that is loaded from the extension, but maybe this is just a display issue:

tc@pCP:~$ ps ax | grep mosq
2162 mosquitt 0:02 mosquitto --daemon --config-file /usr/local/etc/mosquitto/mosquitto.conf
9380 tc 0:00 grep mosq

- I installed paho from within the environment and into the environment's folder:

source /usr/local/camillagui/environment/bin/activate
sudo pip3 install paho-mqtt

and checked it:

(environment) tc@pCP:~$ pip show paho-mqtt
Name: paho-mqtt
Version: 1.6.1
Summary: MQTT version 5.0/3.1.1 client class
Home-page: http://eclipse.org/paho
Author: Roger Light
Author-email: [email protected]
License: Eclipse Public License v2.0 / Eclipse Distribution License v1.0
Location: /usr/local/camillagui/environment/lib/python3.8/site-packages

I don't do anything else here, right?

- finally, I start tonecontrol.py from within the environment from a separate window (the other is occupied with zigbee2mqtt) after I changed the line which contains the ID of the knob:

client.subscribe("zigbee2mqtt/0xdc8e95fffeb8da70",0)

tc@pCP:~$ source /usr/local/camillagui/environment/bin/activate
(environment) tc@pCP:~$ python3 /home/tc/tonecontrol.py

(... cursor is blinking, console stuck)
(after ctrl+c to abort, gives the following output: )

Traceback (most recent call last):
File "/home/tc/tonecontrol.py", line 78, in <module>
client.loop_forever()
File "/usr/local/camillagui/environment/lib/python3.8/site-packages/paho/mqtt/client.py", line 1756, in loop_forever
rc = self._loop(timeout)
File "/usr/local/camillagui/environment/lib/python3.8/site-packages/paho/mqtt/client.py", line 1150, in _loop
socklist = select.select(rlist, wlist, [], timeout)
KeyboardInterrupt

Obviously, Line 78 is the end of tonecontrol.py and gives:

client.loop_forever()

I looked at the lines in paho's client.py, but I don't understand why the script is hanging.

Line 1150 (highlighted) is part of this def:

def loop(self, timeout=1.0, max_packets=1):
"""Process network events.

It is strongly recommended that you use loop_start(), or
loop_forever(), or if you are using an external event loop using
loop_read(), loop_write(), and loop_misc(). Using loop() on it's own is
no longer recommended.

This function must be called regularly to ensure communication with the
broker is carried out. It calls select() on the network socket to wait
for network events. If incoming data is present it will then be
processed. Outgoing commands, from e.g. publish(), are normally sent
immediately that their function is called, but this is not always
possible. loop() will also attempt to send any remaining outgoing
messages, which also includes commands that are part of the flow for
messages with QoS>0.

timeout: The time in seconds to wait for incoming/outgoing network
traffic before timing out and returning.
max_packets: Not currently used.

Returns MQTT_ERR_SUCCESS on success.
Returns >0 on error.

A ValueError will be raised if timeout < 0"""

if self._sockpairR is None or self._sockpairW is None:
self._reset_sockets(sockpair_only=True)
self._sockpairR, self._sockpairW = _socketpair_compat()

return self._loop(timeout)

def _loop(self, timeout=1.0):
if timeout < 0.0:
raise ValueError('Invalid timeout.')

try:
packet = self._out_packet.popleft()
self._out_packet.appendleft(packet)
wlist = [self._sock]
except IndexError:
wlist = []

# used to check if there are any bytes left in the (SSL) socket
pending_bytes = 0
if hasattr(self._sock, 'pending'):
pending_bytes = self._sock.pending()

# if bytes are pending do not wait in select
if pending_bytes > 0:
timeout = 0.0

# sockpairR is used to break out of select() before the timeout, on a
# call to publish() etc.
if self._sockpairR is None:
rlist = [self._sock]
else:
rlist = [self._sock, self._sockpairR]

try:
socklist = select.select(rlist, wlist, [], timeout)
except TypeError:
# Socket isn't correct type, in likelihood connection is lost
return MQTT_ERR_CONN_LOST
except ValueError:
# Can occur if we just reconnected but rlist/wlist contain a -1 for
# some reason.
return MQTT_ERR_CONN_LOST
except Exception:
# Note that KeyboardInterrupt, etc. can still terminate since they
# are not derived from Exception
return MQTT_ERR_UNKNOWN

if self._sock in socklist[0] or pending_bytes > 0:
rc = self.loop_read()
if rc or self._sock is None:
return rc

if self._sockpairR and self._sockpairR in socklist[0]:
# Stimulate output write even though we didn't ask for it, because
# at that point the publish or other command wasn't present.
socklist[1].insert(0, self._sock)
# Clear sockpairR - only ever a single byte written.
try:
# Read many bytes at once - this allows up to 10000 calls to
# publish() inbetween calls to loop().
self._sockpairR.recv(10000)
except BlockingIOError:
pass

if self._sock in socklist[1]:
rc = self.loop_write()
if rc or self._sock is None:
return rc

return self.loop_misc()

And Line 1756 (highlighted) of this def:

def loop_forever(self, timeout=1.0, max_packets=1, retry_first_connection=False):
"""This function calls the network loop functions for you in an
infinite blocking loop. It is useful for the case where you only want
to run the MQTT client loop in your program.

loop_forever() will handle reconnecting for you if reconnect_on_failure is
true (this is the default behavior). If you call disconnect() in a callback
it will return.


timeout: The time in seconds to wait for incoming/outgoing network
traffic before timing out and returning.
max_packets: Not currently used.
retry_first_connection: Should the first connection attempt be retried on failure.
This is independent of the reconnect_on_failure setting.

Raises OSError/WebsocketConnectionError on first connection failures unless retry_first_connection=True
"""

run = True

while run:
if self._thread_terminate is True:
break

if self._state == mqtt_cs_connect_async:
try:
self.reconnect()
except (OSError, WebsocketConnectionError):
self._handle_on_connect_fail()
if not retry_first_connection:
raise
self._easy_log(
MQTT_LOG_DEBUG, "Connection failed, retrying")
self._reconnect_wait()
else:
break

while run:
rc = MQTT_ERR_SUCCESS
while rc == MQTT_ERR_SUCCESS:
rc = self._loop(timeout)
# We don't need to worry about locking here, because we've
# either called loop_forever() when in single threaded mode, or
# in multi threaded mode when loop_stop() has been called and
# so no other threads can access _out_packet or _messages.
if (self._thread_terminate is True
and len(self._out_packet) == 0
and len(self._out_messages) == 0):
rc = 1
run = False

def should_exit():
return self._state == mqtt_cs_disconnecting or run is False or self._thread_terminate is True

if should_exit() or not self._reconnect_on_failure:
run = False
else:
self._reconnect_wait()

if should_exit():
run = False
else:
try:
self.reconnect()
except (OSError, WebsocketConnectionError):
self._handle_on_connect_fail()
self._easy_log(
MQTT_LOG_DEBUG, "Connection failed, retrying")

return rc

Again, thank you for reading and any suggestions.

 

[MCH]

@Charger I am glad you are making progress. Your issue goes well above my pay grade but I will try to have a look and see if I can help. Will tag @Daverz he help me with the original script, maybe he chime in with some clues.

 

[changer]

News: We have it mostly running, only that the gui does not mirror the commands.

Edit: Now I re-read the thread again and learned that the capability was never achieved to reflected the changes in the gui.

Has anyone ever tried to solve this since? As I don't use a display, this could be helpful.

 

[MCH]

News: We have it mostly running, only that the gui does not mirror the commands.

Edit: Now I re-read the thread again and learned that the capability was never achieved to reflected the changes in the gui.

Has anyone ever tried to solve this since? As I don't use a display, this could be helpful.

Great!
Regarding the GUI, well, as you can read, it was not considered a problem, just "the way it is". I disagree with that, but no, never found a solution.
What I don't know is if the recently released version (that goes with CamillaDSP v 2.0) behaves the same. I have not updated yet because I need to change several scripts but maybe @HenrikEnquist can tell.

 

[changer]

Thanks to everyone for the help and your patience with me spamming the thread.

For TinyCoreLinux and piCorePlayer, the script works from the shelf, if:

- Paho-mqtt is installed in the environment, which requires to install it everytime with bootup anew, because the content of the environment folders is in RAM. We did this with a shell script placed in /home/tc that starts paho-mqtt in the background. The shell script is started fom bootlocal.sh and includes 30 seconds delay in order to have an active internet connection before pip attempts to install the script.
- The tonecontrol script is loaded in the environment and run in the background, done with an entry in the same shell script as paho-mqtt.
- Node can be installed as a binary from nodejs.org to /home/tc, which is not a clean tinycore approach, because it is much data loaded into RAM. But it works. Due to whatever reason, npm does not start from the uncompressed directory, while Node does. So we delete the folder, uncompress the tarball, install it and move it to /usr/local each time, before starting it (Maybe moving it again to /usr/local is enough, need to check this). Done from bootlocal.sh.
- zigbee2mqtt can be saved to /home/tc and is started with a shell script, that has the same delay to wait for the installation of the other components. The shell script is started from bootlocal.sh.
- mosquitto is used via an extension loaded from the armv7 tinycore repository. The configuration file can be supposedly be changed via mosquitto -c ..., and we did, but it is not reflected in the console. It works though.
- Obviously, the tonecontrol.py script must contain the IP of the Pi and the ID of the knob.

This is how it works. I would however love to have the tonecontroll script behave differently. If the actions brightness_step_up and brightness_step_down could be used for i.e. treble and color_temperature_step_up and color_temperature_step_down could be used for bass, there is no chance to mix them up, ass the latter requires to turn-press. The action toggle could then be used to reset values to 0. With this configuration, a blind use of the knob is reliable, as after resetting the value to zero it is easy to change bass and treble precisely, without needing to know its current value and whether the knob is toggled to bass or treble manipulation. However, the scripts function revolves around the category tone_db_change whose effect is controlled via the changing of a value via toggle. So I don't know how to to it at the moment, this requires to understand camilladsp classes and python.

 

[changer]

I almost forgot: a huge thanks to @MCH, @mdsimon2, @Daverz and all other contributors! I can now listen with a loudness curve for low level listening and be gentle to my neighbours when listening to bass heavy music.

With Camilla, that has physical controls, the cheapskate solution of a Pi4 music server and streamer, SPDIF digital transport hat into the Fusion Amps digital inputs is complete.

Now I play with the idea to place the fusion amp modules in a case instead, together with the Pi. If I find this within my technical abilities... And maybe get another zigbee knob or two, for even more tone curves.

 

[MCH]

I almost forgot: a huge thanks to @MCH, @mdsimon2, @Daverz and all other contributors! I can now listen with a loudness curve for low level listening and be gentle to my neighbours when listening to bass heavy music.

With Camilla, that has physical controls, the cheapskate solution of a Pi4 music server and streamer, SPDIF digital transport hat into the Fusion Amps digital inputs is complete.

Great to hear, enjoy!!
It is indeed a great and worthy project. Costs next to nothing (the knob+sniffer) and works perfectly fine. I am using mine since beginning of the year as main volume control (btw, still on the first battery) and can't be happier, no searching for the remote in the dark anymore

Now I play with the idea to place the fusion amp modules in a case instead, together with the Pi. If I find this within my technical abilities... And maybe get another zigbee knob or two, for even more tone curves.

Hmmmm... that could be a good idea...

 

[HenrikEnquist]

What I don't know is if the recently released version (that goes with CamillaDSP v 2.0) behaves the same.

The new version behaves the same.

 

[MCH]

short update.

I saw a new zigbee knob in Aliexpress.

https://www.aliexpress.com/item/1005006096963728.html?spm=a2g0o.order_list.order_list_main.137.34381802s19Sfs

I liked the form factor (more flat and wide than the regular one) and the fact that it is white (easier to see in the dark). It costed 6.4 euros, significantly less than the other so i ordered one.

As you can see it also has the LED on the top instead of the bottom which i don't really mind, as it only blinks briefly while you turn it, what some might like but it is definitely not disturbing to me.

I paired it to my system, what was fast and worked at the first try:

Zigbee2MQTT:info 2023-12-19 20:46:22: Device '0xa4c138c83db0f863' joined Zigbee2MQTT:info 2023-12-19 20:46:22: MQTT publish: topic 'zigbee2mqtt/bridge/event', payload '{"data":{"friendly_name":"0xa4c138c83db0f863","ieee_address":"0xa4c138c83db0f863"},"type":"device_joined"}' Zigbee2MQTT:info 2023-12-19 20:46:22: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"message":{"friendly_name":"0xa4c138c83db0f863"},"type":"device_connected"}' Zigbee2MQTT:info 2023-12-19 20:46:22: Starting interview of '0xa4c138c83db0f863' Zigbee2MQTT:info 2023-12-19 20:46:22: MQTT publish: topic 'zigbee2mqtt/bridge/event', payload '{"data":{"friendly_name":"0xa4c138c83db0f863","ieee_address":"0xa4c138c83db0f863","status":"started"},"type":"device_interview"}' Zigbee2MQTT:info 2023-12-19 20:46:22: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"message":"interview_started","meta":{"friendly_name":"0xa4c138c83db0f863"},"type":"pairing"}' Zigbee2MQTT:info 2023-12-19 20:46:23: MQTT publish: topic 'zigbee2mqtt/bridge/event', payload '{"data":{"friendly_name":"0xa4c138c83db0f863","ieee_address":"0xa4c138c83db0f863"},"type":"device_announce"}' Zigbee2MQTT:info 2023-12-19 20:46:23: MQTT publish: topic 'zigbee2mqtt/bridge/log', payload '{"message":"announce","meta":{"friendly_name":"0xa4c138c83db0f863"},"type":"device_announced"}' Zigbee2MQTT:info 2023-12-19 20:46:29: Successfully interviewed '0xa4c138c83db0f863', device has successfully been paired

but unfortunately the device has one problem and one flaw:

The problem is that it is not yet supported by zigbee2MQTT:

Zigbee2MQTT:warn 2023-12-19 20:46:53: Received message from unsupported device with Zigbee model 'TS004F' and manufacturer name '_TZ3000_abrsvsou' Zigbee2MQTT:warn 2023-12-19 20:46:53: Please see: https://www.zigbee2mqtt.io/advanced/support-new-devices/01_support_new_devices.html

But i guess this is a matter of time or of doing it yourself if nobody else does it. There are not so many knobs like this in the market (i only know these two) so i would be surprised if it doesn't get supported at some time.

And the flaw, is.... what i see as a very stupid design decision. What turns, instead of the outer part of the knob, is the inner one!! why?? I am sure there is a use case for this, but it is definitely not how i would like to use it. So unfortunately this one goes directly to the drawer until i find another use for it. :-/

My recommendation is, unless you are ok with this way of turning it, don't order it.

 

[somebodyelse]

And the flaw, is.... what i see as a very stupid design decision. What turns, instead of the outer part of the knob, is the inner one!! why??

I think you found the reason earlier - manufacturing cost.

It costed 6.4 euros, significantly less than the other

 

[MCH]

I think you found the reason earlier - manufacturing cost.

Don't know, maybe I crank it open and see if it is a much simpler build inside. I don't see an obvious reason for it to be though.
Could also be they realised they had a flawed design and they just need to sell it cheaper to be able to sell it at all.
My theory is that there is always a use case, for instance, this design is much more difficult to turn by mistake, or that a small child does it. So if it controls something that you don't need to adjust often and you have kids, maybe you are better with something like this. But as a volume knob, it is not for me.

 

[paalj]

I was also inspired by the thread My Kingdom For A Remote Volume Knob!

So, if anyone is interested I've put together an open source web socket server for the Surface Dial. I'm currently using this in conjunction with a display app I built for BluOS streamers like the Node. The display app runs entirely within a browser. I use an old Android tablet I had lying around.

The web socket server should run on pretty much any Linux box with Bluetooth. I'm running it on a Raspberry Pi which also serves up the web page for the Android tablet. But you don't have to use the Surface Dial server with the display app. It can be used on its own to control another streamer, preamp, etc. That's why I wrote it as a generic web socket server. It provides a stream of rotations or clicks that could also be used for bass/treble or other adjustments or you can use them to do whatever you want within the limits of having just a few gestures. Example output:

{ "button" : "down" }
{ "button" : "up" }
{ "degrees" : "3.1" }

It's working well for volume, mute and pause/resume functions. The only gotcha is the Surface Dial falls asleep too quickly and you have to interact with it (click or rotate) to wake it up before it starts working again. Lots of people talking about that but I haven't found a work-around yet.

I don't have a photo of the display along with the Surface Dial next to it but I'm sure you get the idea from this one. The RPi is hidden under the bench.

View attachment 272728

Hi,
Thanks for this.
I have been fiddeling around with a microsoft surface dial and a rpi5 with camilla and your dialserver and finally I got the ws server running.

{"degrees":14.4}
{"degrees":2.2}
{"degrees":-7.3}
{"degrees":-19.5}
{"degrees":-11.6}
{"degrees":-1.7}
{"degrees":-12.7}
{"degrees":1.2}
{"button":"up"}
{"degrees":0.5}
{"button":"down"}

I am not familiar with python other than editing the scripts, but since camilla is talking ws and the server talks ws there might be a possibility? I would be grateful if someone could guide me through a volume control script.

 

[MCH]

Hi,
Thanks for this.
I have been fiddeling around with a microsoft surface dial and a rpi5 with camilla and your dialserver and finally I got the ws server running.

{"degrees":14.4}
{"degrees":2.2}
{"degrees":-7.3}
{"degrees":-19.5}
{"degrees":-11.6}
{"degrees":-1.7}
{"degrees":-12.7}
{"degrees":1.2}
{"button":"up"}
{"degrees":0.5}
{"button":"down"}

I am not familiar with python other than editing the scripts, but since camilla is talking ws and the server talks ws there might be a possibility? I would be grateful if someone could guide me through a volume control script.

I based the script for my remote knob on the one @mdsimon2 published in his tutorial for the flirc remote. Maybe you can adapt yours from it. Well, you will have to modify it a lot as the dial report angles, you will have to be creative on a way to translate that so that the volume changes smoothly...
I guess you know it but in case:

RPi4 + CamillaDSP Tutorial

Intent of this thread is to provide a tutorial for setting up CamillaDSP on a RPi4. There is a lot of good information scattered through ASR, DIYAudio and the CamillaDSP GitHub but there also appears to be a lot of apprehension. My goal is to give concrete examples of how to use CamillaDSP with...

www.audiosciencereview.com

Good luck, it is great to be able to get the dial working with camilladsp

 

[paalj]

Thanks MCH, I will have a look if I can understand what I need to add/remove. Guess I need to import the dialserver into the script. I will have to read the docs and give it a try ;-)
Yes, I am familiar with the great tutorial. Thanks to @mdsimon2