Selecting audio input or output device

I created a webapp called Guitar Chord Recognition using Streamlit. It allows you to record a chord and then outputs the predicted chord. It also displays the spectrogram of the sound recorded. You can view the demo here. Also, you may view the GitHub repository.

I want to deploy this app publicly but I need a way to select the recording device in the app. I couldnā€™t find any particular way to achieve this. Is there any way to select the input or output device in the web app?

4 Likes

Hello @Ayush_Kumar_Shah, welcome to the forums !

This is a super use case you have here :smiley: I had a similar usecase where I wanted to record my voice via Streamlit and then make a ā€œchipymunkā€ version of it.

I think I remember why I had to put it down for now :

  • My initial take was to select the recording audio device through sounddevice, or in your case through pyaudio, but this selects the recording device on Streamlitā€™s server, not on the user facing web page. So if your app is deployed, selecting the recording device for the server is not very interestingā€¦
  • My second take would be to use some HTML5 input audio capture, but I think this needs some JS processing, and Streamlit does not support that yet.
    • That could be an interesting Custom Component to write.
    • If you a super courageous, though this is more of a hack that I have yet to test, you can run ipywidgets in Bokeh 2 so you could try ipywebrtc inside st.bokeh_chart, have the output file streamed in a temporary folder and have Streamlit watch that folderā€¦
  • You could cheat the system by pre-recording your guitar play and send it through the file uploader in the meantime (thatā€™s ultimately what I did when translating the DDSP Colab demo to Streamlitā€¦)
    • Did not skim the Streamlit File Uploader source code but maybe itā€™s possible to enhance it with audio capture capabilities ? There are github issues here and here, we can add our use cases there and see if more people are interested in this feature natively in Streamlit or as a custom component. I would be a big fan for audio/video input in native Streamlit though, because ā€œDeep Learningā€

Hope this helps a bit :upside_down_face:

2 Likes

Hello @andfanilo. Thank you for all the useful information. Here are my takes from each of it:

  • I realized the same thing and figured that I need a way to select the recording device on the userā€™s web page instead.
  • I tried using ipywebrtc but I couldnā€™t get it to record sound. The code snippet didnā€™t work and I couldnā€™t figure out whatā€™s wrong. The recorder didnā€™t execute and I got the error
    ValueError: No data, did you record anything?
    
  • The pre-recording option will not work for me as the users should be able to play and record the chord so that the system can return the predicted chord. So, the system should be able to predict a chord for every new recording the user does.
    • Yeah, it would be awesome if Streamlit would add the feature of audio/video input.

I am searching for any kind of hack that could do the job for me. Without the ability of recording sound by users, the app is of no use.

However, I greatly appreciate your help. If you find something, please inform me as well. Thanks.

2 Likes

Has anyone come up with a solution to this? I am trying to capture microphone input and manipulate the speakerā€™s voice. I have it working locally but Iā€™m having the same problems as above where I canā€™t capture the userā€™s microphone.

3 Likes

Him any update on this? Has anyone done something like this. Ability to use ipywebrtc inside streamlit would solve a lot of issues along with audio/ video recording

It should be possible now using New Component: streamlit-webrtc, a new way to deal with real-time media streams

If anyone wants to give this a try :slight_smile:
Fanilo

1 Like

Hi, iā€™m trying to use streamlit-webrtc to record only audio (im not interested in video).
since thereā€™s no documentation and the only resources online regard the video part not the audio, anyone in the forum can help me?

thanks in advance