• WANTED: Happy members who like to discuss audio and other topics related to our interest. Desire to learn and share knowledge of science required. There are many reviews of audio hardware and expert members to help answer your questions. Click here to have your audio equipment measured for free!

Building a wireless remote volume control with a rotary knob for Raspberry pi - help needed with Python

anotherhobby

Addicted to Fun and Learning
Forum Donor
Joined
Dec 17, 2021
Messages
623
Likes
1,319
I'll be curious to see how this works out for you. I did some very similar things to bridge MQTT to volume control for my Denon using several different tools (zigbee2qmtt, Home Assistant, NodeRed, etc), but no matter what I did, the lag was too much to be a good user experience for me. Between MQTT, TCP, several processors, and several application hand-offs all in the middle took too long, and made the volume control very rubber band like and hard to predict. I'll be watching this thread to see if your solution manages to avoid that problem, and then I may take another crack at it. I probably had more parts in the middle slowing me down, and I wish you good luck sir!
 
OP
M

MCH

Major Contributor
Joined
Apr 10, 2021
Messages
2,581
Likes
2,197
I'll be curious to see how this works out for you. I did some very similar things to bridge MQTT to volume control for my Denon using several different tools (zigbee2qmtt, Home Assistant, NodeRed, etc), but no matter what I did, the lag was too much to be a good user experience for me. Between MQTT, TCP, several processors, and several application hand-offs all in the middle took too long, and made the volume control very rubber band like and hard to predict. I'll be watching this thread to see if your solution manages to avoid that problem, and then I may take another crack at it. I probably had more parts in the middle slowing me down, and I wish you good luck sir!
Agree, that is to be seen.
The only reference i can give you at the moment, that at the same time is what makes me confident on a relative fast response, is that when I turn the knob, the message gets printed immediately in the RPI command window. And all the remaining processes, including the volume control itself, happen in the same rpi. I trust such internal processing in a raspberry pi 4b should be quite fast, even with camilladsp running in parallel, but let's see...
But unless the lag is really very long i don't think it will be a big problem for me, i have lived with a topping remote for several months :D
What i am less confident about is if the response to a sequence of several actions will be smooth or erratic. I would love to have a smooth volume increase when turning the knob slowly, but I really doubt it will happen. But well, as long as it is functional, it will be worth the 20eur total cost. @Daverz help is priceless, but so far it has been for free :)
 
Last edited:

anotherhobby

Addicted to Fun and Learning
Forum Donor
Joined
Dec 17, 2021
Messages
623
Likes
1,319
Agree, that is to be seen.
The only reference i can give you at the moment, that at the same time is what makes me confident on a relative fast response, is that when I turn the knob, the message gets printed immediately in the RPI command window. And all the remaining processes, including the volume control itself, happen in the same rpi. I trust such internal processing in a raspberry pi 4b should be quite fast, even with camilladsp running in parallel, but let's see...
But unless the lag is really very long i don't think it will be a big problem for me, i have lived with a topping remote for several months :D
What i am less confident about is if the response to a sequence of several actions will be smooth or erratic. I would love to have a smooth volume increase when turning the knob slowly, but I really doubt it will happen. But well, as long as it is functional, it will be worth the 2eur total cost. @Daverz help is priceless, but so far it has been for free :)
I suspect you may very well end up with low/reasonable lag. It'll certainly be better than mine as I had too many network hops between the control and device. You definitely have less hops that I did.
 
  • Like
Reactions: MCH

Capitol C

Active Member
Joined
May 21, 2021
Messages
164
Likes
189
Location
Washington, DC
OP
M

MCH

Major Contributor
Joined
Apr 10, 2021
Messages
2,581
Likes
2,197

Daverz

Major Contributor
Joined
Mar 17, 2019
Messages
1,294
Likes
1,451
in the meantime, i have found the paho mqtt tutorial below that is quite complete and starts from the very beginning. Now everything starts to make sense, even though it uses some terminology that is definitely not for a newcomer like me.


one doubt i have: i read pycamilladsp is a websocket client. Does this mean that paho-mqtt client should also be set up as websocket or should it be left as tcp?

I would think that paho-mqtt is totally independent of camilladsp, so I'd use whatever is easiest.
 
OP
M

MCH

Major Contributor
Joined
Apr 10, 2021
Messages
2,581
Likes
2,197
Were you able to get that oiginal paho script to run? The payload above looks like JSON, but the paho Python module may convert it to a Python dictionary. To get a better handle on what it is, you can add some more print statements to the callback:

Python:
print('payload type:', type(msg.payload))
print('payload attributes:', dir(msg.payload))

If it's a string, you can convert it using the built-in json module

Python:
import json

# ...

payload = json.loads(msg.payload)  # if it's a string and not already converted to a dict for you
step_size = payload['action_step_size']
# convert step_size to dB based, this would be your own function
db_change = step_size_to_db(current_volume, step_size)
if msg.action == 'brightness_step_down':
     db_change = -db_change
# ...
this is what your instructions afforded:

Connected with result code 0
zigbee2mqtt/0xdc8e95fffe8a7094 b'{"action":"brightness_step_up","action_step_size":13,"action_transition_time":0.01,"battery":100,"linkquality":65,"operation_mode":"command","voltage":3000}'
payload type: <class 'bytes'>
payload attributes: ['__add__', '__class__', '__contains__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__getnewargs__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__', '__lt__', '__mod__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__rmod__', '__rmul__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'capitalize', 'center', 'count', 'decode', 'endswith', 'expandtabs', 'find', 'fromhex', 'hex', 'index', 'isalnum', 'isalpha', 'isascii', 'isdigit', 'islower', 'isspace', 'istitle', 'isupper', 'join', 'ljust', 'lower', 'lstrip', 'maketrans', 'partition', 'removeprefix', 'removesuffix', 'replace', 'rfind', 'rindex', 'rjust', 'rpartition', 'rsplit', 'rstrip', 'split', 'splitlines', 'startswith', 'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill']
 
OP
M

MCH

Major Contributor
Joined
Apr 10, 2021
Messages
2,581
Likes
2,197
Eureka!! Finally works!


Still far from finished but the proof of concept is there. The range is enough (even with the receptor inside a cabinet with the door closed), the delay is very small.
I still need to work a lot on the python script and on the service setup (i don't manage to make it work as a daemon) but the problem is my zero knowledge of Linux, let alone python.
Thanks to everyone who commented, and special thanks to @Daverz
 

Daverz

Major Contributor
Joined
Mar 17, 2019
Messages
1,294
Likes
1,451
this is what your instructions afforded:

Connected with result code 0
zigbee2mqtt/0xdc8e95fffe8a7094 b'{"action":"brightness_step_up","action_step_size":13,"action_transition_time":0.01,"battery":100,"linkquality":65,"operation_mode":"command","voltage":3000}'
payload type: <class 'bytes'>
payload attributes: ['__add__', '__class__', '__contains__', '__delattr__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getitem__', '__getnewargs__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__len__', '__lt__', '__mod__', '__mul__', '__ne__', '__new__', '__reduce__', '__reduce_ex__', '__repr__', '__rmod__', '__rmul__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', 'capitalize', 'center', 'count', 'decode', 'endswith', 'expandtabs', 'find', 'fromhex', 'hex', 'index', 'isalnum', 'isalpha', 'isascii', 'isdigit', 'islower', 'isspace', 'istitle', 'isupper', 'join', 'ljust', 'lower', 'lstrip', 'maketrans', 'partition', 'removeprefix', 'removesuffix', 'replace', 'rfind', 'rindex', 'rjust', 'rpartition', 'rsplit', 'rstrip', 'split', 'splitlines', 'startswith', 'strip', 'swapcase', 'title', 'translate', 'upper', 'zfill']

Yup, it's just a string of raw bytes, thus the decode('utf8') step shown in the example to decode it to a UTF-8 JSON string.
 
  • Like
Reactions: MCH

Capitol C

Active Member
Joined
May 21, 2021
Messages
164
Likes
189
Location
Washington, DC
Hey Capitol, the objective here is a knob that you turn with your hand and changes a software volume control remotely. But I see your suggestion could be of interest for others. Your link is dead though.
I'm recovering from Covid followed by kidney-stone surgery. Or maybe I'm just getting old. Sorry for reading your description wrong, it is quite clear. Here is a link for anyone who wants the other gadget: https://www.ebay.com/itm/2850511613...UxFeF221kewoZszTePWDF+m8xX|tkp:Bk9SR_6QiISuYQ
 
  • Like
Reactions: MCH
OP
M

MCH

Major Contributor
Joined
Apr 10, 2021
Messages
2,581
Likes
2,197
Yup, it's just a string of raw bytes, thus the decode('utf8') step shown in the example to decode it to a UTF-8 JSON string.
Hey Daverz, all,

What i eventually did was to use the line:

payload = json.loads(msg.payload)

and as you can see, i have it working :)

I have been working a bit more on the python code this morning and i have now a pretty solid script that responds to faster/longer turns, has a top limit set up for safety and the button press action triggers mute/unmute. In this regard i think i am done for the moment, maybe i will just adjust the db changes size down the road (see demo video at the end to get an idea). Also, i am seeing the range of the knob is much better than i thought, and works even from a different room with the doors closed!

Where i am stacked now is in creating a service to have it always running in the background and to get it started on boot:
The service i created for zigbee2mqtt (set it up in /etc/systemd/system/zigbee2mqtt.service) works fine.
however, when i try to create a service for the knob "knob.service" either in /etc/systemd/system/ or in /lib/systemd/system/ it doesnt want to work...
i have the feeling that the problem might be on when to start. I have tried different possibilities but nothing worked. My current file is the following, starting after the zigbee2mqtt.service, thinking that it might need that one working before starting (??):

[Unit]
After=zigbee2mqtt.service
StartLimitIntervalSec=10
StartLimitBurst=10

[Service]
Type=simple
ExecStart=python3 /home/marcosch/knob.py
Restart=always
RestartSec=1
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=knob
User=root
Group=root

[Install]
WantedBy=graphical.target

and what i get when i try to start it from the command prompt is:

marcosch@raspcamilla:~$ sudo systemctl start knob
marcosch@raspcamilla:~$ systemctl status knob.service
● knob.service
Loaded: loaded (/lib/systemd/system/knob.service; enabled; preset: enabled)
Active: activating (auto-restart) (Result: exit-code) since Mon 2023-01-02 09:30:25 CET; 359ms ago
Process: 27810 ExecStart=python3 /home/marcosch/knob.py (code=exited, status=1/FAILURE)
Main PID: 27810 (code=exited, status=1/FAILURE)
CPU: 444ms

and if i check what is going on with:

sudo journalctl -u knob.service -f

i get this output continuously:

Jan 02 09:48:27 raspcamilla systemd[1]: Started knob.service.
Jan 02 09:48:27 raspcamilla knob[28541]: Traceback (most recent call last):
Jan 02 09:48:27 raspcamilla knob[28541]: File "/home/marcosch/knob.py", line 3, in <module>
Jan 02 09:48:27 raspcamilla knob[28541]: import paho.mqtt.client as mqtt
Jan 02 09:48:27 raspcamilla knob[28541]: ModuleNotFoundError: No module named 'paho'
Jan 02 09:48:27 raspcamilla systemd[1]: knob.service: Main process exited, code=exited, status=1/FAILURE
Jan 02 09:48:27 raspcamilla systemd[1]: knob.service: Failed with result 'exit-code'.
Jan 02 09:48:28 raspcamilla systemd[1]: knob.service: Scheduled restart job, restart counter is at 25169.
Jan 02 09:48:28 raspcamilla systemd[1]: Stopped knob.service.

highlighted lines are mine.
Any idea what could i try??
thanks!!

Latest progress:

 
Last edited:

mdsimon2

Major Contributor
Forum Donor
Joined
Oct 20, 2020
Messages
2,477
Likes
3,315
Location
Detroit, MI
Hey Daverz, all,

What i eventually did was to use the line:

payload = json.loads(msg.payload)

and as you can see, i have it working :)

I have been working a bit more on the python code this morning and i have now a pretty solid script that responds to faster/longer turns, has a top limit set up for safety and the button press action triggers mute/unmute. In this regard i think i am done for the moment, maybe i will just adjust the db changes size down the road (see demo video at the end to get an idea). Also, i am seeing the range of the knob is much better than i thought, and works even from a different room with the doors closed!

Where i am stacked now is in creating a service to have it always running in the background and to get it started on boot:
The service i created for zigbee2mqtt (set it up in /etc/systemd/system/zigbee2mqtt.service) works fine.
however, when i try to create a service for the knob "knob.service" either in /etc/systemd/system/ or in /lib/systemd/system/ it doesnt want to work...
i have the feeling that the problem might be on when to start. I have tried different possibilities but nothing worked. My current file is the following, starting after the zigbee2mqtt.service, thinking that it might need that one working before starting (??):

[Unit]
After=zigbee2mqtt.service
StartLimitIntervalSec=10
StartLimitBurst=10

[Service]
Type=simple
ExecStart=python3 /home/marcosch/knob.py
Restart=always
RestartSec=1
StandardOutput=syslog
StandardError=syslog
SyslogIdentifier=knob
User=root
Group=root

[Install]
WantedBy=graphical.target

and what i get when i try to start it from the command prompt is:

marcosch@raspcamilla:~$ sudo systemctl start knob
marcosch@raspcamilla:~$ systemctl status knob.service
● knob.service
Loaded: loaded (/lib/systemd/system/knob.service; enabled; preset: enabled)
Active: activating (auto-restart) (Result: exit-code) since Mon 2023-01-02 09:30:25 CET; 359ms ago
Process: 27810 ExecStart=python3 /home/marcosch/knob.py (code=exited, status=1/FAILURE)
Main PID: 27810 (code=exited, status=1/FAILURE)
CPU: 444ms

and if i check what is going on with:

sudo journalctl -u knob.service -f

i get this output continuously:

Jan 02 09:48:27 raspcamilla systemd[1]: Started knob.service.
Jan 02 09:48:27 raspcamilla knob[28541]: Traceback (most recent call last):
Jan 02 09:48:27 raspcamilla knob[28541]: File "/home/marcosch/knob.py", line 3, in <module>
Jan 02 09:48:27 raspcamilla knob[28541]: import paho.mqtt.client as mqtt
Jan 02 09:48:27 raspcamilla knob[28541]: ModuleNotFoundError: No module named 'paho'
Jan 02 09:48:27 raspcamilla systemd[1]: knob.service: Main process exited, code=exited, status=1/FAILURE
Jan 02 09:48:27 raspcamilla systemd[1]: knob.service: Failed with result 'exit-code'.
Jan 02 09:48:28 raspcamilla systemd[1]: knob.service: Scheduled restart job, restart counter is at 25169.
Jan 02 09:48:28 raspcamilla systemd[1]: Stopped knob.service.

highlighted lines are mine.
Any idea what could i try??
thanks!!

Latest progress:


When you were not using the service were you running the knob script as root?

The python package manager is a bit weird in that you need to install packages as root if you want to run python as root. It may be as simple as running “sudo pip3 install paho-mqtt”.

Michael
 
  • Like
Reactions: MCH
OP
M

MCH

Major Contributor
Joined
Apr 10, 2021
Messages
2,581
Likes
2,197
Hi Michael,
When you were not using the service were you running the knob script as root?
The knob script is in the home folder ( ~/knob.py). To run it manually i just do
python3 ~/knob.py
And it works perfectly this way. Is this what you mean?
The python package manager is a bit weird in that you need to install packages as root if you want to run python as root. It may be as simple as running “sudo pip3 install paho-mqtt”.
Oh! Good to know. I don't remember 100% but I believe I did:
pip install paho-mqtt
Do you think I can reinstall using
sudo pipi3 install paho-mqtt
as is or should I remove what is already installed before?
One more question Michael. What do you think about my service document? I don't yet understand the last line "WantedBy=graphical.target" and i am not sure it is the right option....
Thanks a lot!
 

mdsimon2

Major Contributor
Forum Donor
Joined
Oct 20, 2020
Messages
2,477
Likes
3,315
Location
Detroit, MI
The knob script is in the home folder ( ~/knob.py). To run it manually i just do
python3 ~/knob.py
And it works perfectly this way. Is this what you mean?

This means you are running the script as a normal user, not root. However your script is running as a root user.

Oh! Good to know. I don't remember 100% but I believe I did:
pip install paho-mqtt
Do you think I can reinstall using
sudo pipi3 install paho-mqtt
as is or should I remove what is already installed before?

At least on ubuntu, pip install and pip3 install are the same command and will install a python package for the user running the command. In order to install the package as root run sudo pip install paho-mqtt or sudo pip3 install paho-mqtt. This is a separate installation from your previous command so need to uninstall / reinstall anything.

One more question Michael. What do you think about my service document? I don't yet understand the last line "WantedBy=graphical.target" and i am not sure it is the right option....

I really do not know much in this area and have in general just copied the camilladsp.service for my other services. I imagine it will work as you have it.

Michael
 
  • Like
Reactions: MCH
OP
M

MCH

Major Contributor
Joined
Apr 10, 2021
Messages
2,581
Likes
2,197
This means you are running the script as a normal user, not root. However your script is running as a root user.



At least on ubuntu, pip install and pip3 install are the same command and will install a python package for the user running the command. In order to install the package as root run sudo pip install paho-mqtt or sudo pip3 install paho-mqtt. This is a separate installation from your previous command so need to uninstall / reinstall anything.



I really do not know much in this area and have in general just copied the camilladsp.service for my other services. I imagine it will work as you have it.

Michael
Hey you are right, it worked!!!

great, i think I can call the project almost finished. Thanks a lot guys!
 
OP
M

MCH

Major Contributor
Joined
Apr 10, 2021
Messages
2,581
Likes
2,197
Great to hear! This project looks really cool and offers some great functionality that is not common, well done.

Michael
Thanks Michael, i wouldn't be using camilladsp at all without your tutorial and help. :)
 
OP
M

MCH

Major Contributor
Joined
Apr 10, 2021
Messages
2,581
Likes
2,197
hello,
It is raining outside and i took the chance to add a bit more of features to my beloved knob (i rarely use the ir remote anymore). Taking profit from the fact that it reports % battery left, i wanted that my display shows a warning when the battery level gets too low. It kind of works, but i must be missing something because after the first movement of the knob the display gets all messed up....:

my display was built following the instructions of @mdsimon2 here:


i modified a bit the code of the python script of the display so that it gets subscribed to the knob mqtt signal and when the battery level reaches a thereshold, instead of showing the sample rate, shows the message "low batt". At the end of this post i have attached the modified part, that goes roughly from after defining oled_string until the end. Please note that for my tests i have set the threshold of "low batt" at 110%, because my battery is still showing 100% and i want to see how my mod works. The print lines are to help me troubleshooting while i write the code.

i believe the modified code works as intended, the problem is the following:
With the display blank (normal operation), i move the knob once and the display shows what is supposed to show, with the message "low batt" there, as intended
If before the display goes blank again i move again the knob, the display gets all messed up and only the message "low batt" can be clearly read. It looks like this:

1673711435556.png


I have the strong feeling that i need to add a line to refresh the display somewhere, but i dont know what such instruction looks like and where should be placed. Any suggestions?
thanks a lot!

code:

def on_connect(client, userdata, flags, rc, properties=None): print("Connected with result code "+str(rc)) broker="xxx.xxx.xxx.xx" client.subscribe("zigbee2mqtt/0xdc8e95fffe8a7094",1) def on_message(client, userdata, msg): print(msg.topic+" "+str(msg.payload)) payload = json.loads(msg.payload) global battery battery = payload['battery'] if battery > 110: oled_string2 = oled_string(0x1D,0x34,0x18,0x2B,20,font3,str(conf["devices"]["samplerate"]) + " Hz") elif battery <= 110: oled_string2 = oled_string(0x1D,0x34,0x18,0x2B,20,font3,"low batt") def oled_string2(): oled_string(0x1D,0x34,0x18,0x2B,20,font3,str(conf["devices"]["samplerate"]) + " Hz") def oled_string3(): oled_string(0x1D,0x37,0x2C,0x3F,20,font3,status) def dB(): oled_string(0x56,0x5C,0x1F,0x32,20,font3,"dB") # Initialize display def oled_init(): set_sleep_mode(0xAE) set_command_lock(0x12) set_display_clock_oscillator_frequency(0xA0) set_multiplex_ratio(0x3F) set_display_offset(0x00) function_selection(0x01) set_display_start_line(0x00) set_remap(0x15,0x11) master_contrast_control(0x0F) set_contrast_control(0x9F) set_phase_length(0xE2) set_precharge_voltage(0x17) enable_external_vsl(0xA0,0xFD) set_vcomh_voltage(0x04) display_mode(0xA6) exit_partial_display() display_enhancement(0xA2) set_gpio(0x00) default_grayscale_command() set_second_precharge_period(0x08) set_sleep_mode(0xAF) clearpixel() client = mqtt.Client(client_id="", clean_session=True, userdata=None, transport="tcp") client.on_connect = on_connect client.on_message = on_message client.connect("xxx.xxxx.xxx.xx") client.loop_start() if __name__ == '__main__': try: main() except KeyboardInterrupt: pass finally: clearpixel()
 
OP
M

MCH

Major Contributor
Joined
Apr 10, 2021
Messages
2,581
Likes
2,197
OK, it took me a lot of trial and error but i found the solution:
for some reason that i still don't understand, trying to change one existing line for other is not so straightforward, at least for me, so i decided to create a new "item" on the top right corner of the display, that it will print the message "LB" for Low Battery, when the battery level reaches the threshold. this works fine and now i realize it is a much better option as it is much easier to spot:

1673731498537.png
 
Top Bottom