Overview of My Submission
The goal is to provide extra way to input value. It could be really helpful for people with disabilities, anyone that have problems with typing on a keyboard. You can fill any input using pointer and voice! Cool!
Submission Category:
Accessibility
Link to Code on GitHub
DEV hackathon project, usage of Deepgram AI Speech Recognition, boost your app accessibility
It's example of integration with Deepgram using react.
Deepgram is external service to transcript speech from audio! (using AI, crazy stuff!)
Read more here: https://deepgram.com
Purpose is to use speech transcription to improve an react app accessibility. We provide extra way to input value!
Help to provide input for people with disabilities
Speed up a form filling
Share expierience accross any device
any device supporting modern browser
react-native (mobile, TV, dekstop) as well!
Project was made as submission to DEV hackathlon, read more here
Post here: post
Project is built from two parts deepgram-proxy and deepgram-react
We need some backend to upload audio file…
Additional Resources / Info
I am using react as a frontend app.
It's important to remember that react frontend app is not necessary. You can integrate proxy with any other type of app. It's just REST API!
The process is pretty simple.
Display Deepgram component
Ask permission to microphone (browser)
Record your voice
Sending audio to proxy
Proxy ask Deepgram for transcription
Proxy responses with transcription, error or warring about no transcription (eg. user need to repeat louder)
User receives transcription and apply to input
Proxy from technical aspect.
It's node.js + express.js to handle API requests and communicate with Deepgram (using Deepgram SDK)
// endpoint to upload and transcript
app.post("/audiotranscript", upload.single("file"), async (req, res) => {
let filepath = req.file.path
let language = req.body.language;
let transcript = await deepgramTranscript(deepgramApiKey, filepath, language);
res.send({ transcript: transcript });
});
Enter fullscreen mode
Exit fullscreen mode
Fronted from technical aspect.
It's simple handler you pass proxy url and setter for your value.
I prepared two visualizations of usage. I am using antd design as component library.
Using wrapper that creates popover to any React component.
<DeepgramHandlerPopover
setValue={setNotepadValue}
proxyUploadUrl={proxyUploadUrl}
>
<Input.TextArea
rows={20}
value={value}
onChange={(ev) => {
setValue(ev.target.value);
}}
/>
</DeepgramHandlerPopover>
Enter fullscreen mode
Exit fullscreen mode
Using modal (small devices friendly)
<DeepgramHandlerModalButton
setValue={setNotepadValue}
proxyUploadUrl={proxyUploadUrl}
buttonProps={{
type: "primary",
}}
/>
Enter fullscreen mode
Exit fullscreen mode
If you have other idea you can easy just wrap handler:
<MyCoolComponent>
<DeepgramHandler
setValue={setValue}
proxyUploadUrl={proxyUploadUrl}
/>
</MyCoolComponent>
Enter fullscreen mode
Exit fullscreen mode
Final thoughts
I really appreciate what Deepgram has created. It is really well working, even with my unclear English. Amazing how tech industry improving our lives! <3
Paweł Ciosek
